00000001:02000400:3.0F:1788880135.789971:0:201082:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: == conf-sanity test 119: writeconf on slave mdt shouldn't duplicate mdc/osp and crash ========================================================== 11:08:55 (1788880135) 00000001:00000040:3.0:1788880138.117854:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: adler32 (adler32-zlib) speed 0 MB/s 00000001:00000040:3.0:1788880138.118597:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: adler32 (adler32-zlib) speed 0 MB/s 00000001:00000040:2.0:1788880138.542662:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.543281:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.543874:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.544477:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.545074:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.545793:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.546388:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.546973:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.547560:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.548156:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.548735:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.549343:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.549894:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.550469:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.550998:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.551505:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.552030:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.552786:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.553373:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.553895:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.554471:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.555052:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.555653:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.556257:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.556747:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.557342:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.557865:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.558396:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.558923:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.559445:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.560017:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.560635:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.561225:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.561708:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.562287:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.562854:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.563454:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.564048:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.564668:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.565303:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.565954:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.566565:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.567353:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.567932:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.568451:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.568936:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.569491:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.570026:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.570651:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.571277:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.571826:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.572432:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.572959:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.573558:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.574124:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.574676:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.575271:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.575868:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.576488:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.577060:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.577597:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.578159:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.578778:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.579411:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.580039:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.580666:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.581378:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.581986:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.582608:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:00000040:2.0:1788880138.583219:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32 (crc32-pclmul) speed 0 MB/s 00000001:01000000:2.0:1788880138.583870:0:201223:0:(lnet-crypto.c:359:cfs_crypto_performance_test()) Crypto hash algorithm crc32 speed = 1651 MB/s 00000001:00000040:2.0:1788880138.583888:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.583989:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.584085:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.584193:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.584307:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.584405:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.584500:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.584595:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.584703:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.584799:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.584898:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.584995:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.585095:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.585194:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.585386:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.585483:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.585586:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.585705:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.585803:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.585899:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.585993:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.586089:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.586186:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.586304:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.586399:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.586490:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.586580:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.586673:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.586780:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.586875:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.586973:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.587070:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.587168:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.587285:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.587383:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.587474:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.587572:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.587667:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.587781:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.587879:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.587976:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.588073:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.588181:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.588300:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.588398:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.588495:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.588601:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.588708:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.588803:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.588898:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.588993:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.589083:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.589176:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.589355:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.589448:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.589542:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.589637:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.589743:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.589842:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.589940:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.590037:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.590128:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.590223:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.590342:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.590438:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.590532:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.590627:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.590735:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.590830:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.590922:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.591015:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.591108:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.591204:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.591315:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.591407:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.591516:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.591614:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.591720:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.591818:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.591916:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.592014:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.592124:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.592223:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.592353:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.592451:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.592549:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.592651:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.592761:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.592861:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.592960:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.593058:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.593159:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.593346:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.593444:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.593539:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.593636:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.593742:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.593838:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.593935:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.594035:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.594128:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.594225:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.594388:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.594481:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.594604:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.594705:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.594795:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.594886:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.594979:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.595071:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.595166:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.595285:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.595379:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.595472:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.595567:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.595663:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.595769:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.595863:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.595959:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.596057:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.596150:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.596267:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.596362:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.596456:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.596548:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.596645:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.596753:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.596850:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.596945:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.597040:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.597139:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.597266:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.597360:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.597517:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.597611:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.597721:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.597816:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.597912:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.598005:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.598098:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.598193:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.598305:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.598403:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.598494:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.598584:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.598690:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.598786:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.598882:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.598978:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.599075:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.599168:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.599281:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.599381:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.599483:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.599576:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.599674:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.599784:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.599882:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.599981:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.600079:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.600176:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.600300:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.600409:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.600504:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.600597:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.600707:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.600808:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.600904:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.601000:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.601095:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.601190:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.601376:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.601470:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.601562:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.601653:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.601759:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.601847:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.601943:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.602034:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.602129:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.602225:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.602458:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.602538:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.602634:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.602741:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.602837:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.602935:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.603032:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.603130:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.603227:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.603341:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.603447:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.603544:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.603639:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.603761:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.603860:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.603958:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.604056:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.604152:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.604274:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.604372:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.604470:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.604566:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.604665:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.604776:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.604878:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.604979:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.605075:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.605171:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.605300:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.605402:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.605503:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.605601:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.605715:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.605809:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.605898:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.605993:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.606092:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.606189:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.606309:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.606414:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.606509:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.606607:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.606718:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.606814:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.606909:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.607006:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.607102:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.607200:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.607319:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.607415:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.607513:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.607610:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.607722:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.607819:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.607918:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.608014:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.608111:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.608209:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.608327:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.608422:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.608514:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.608607:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.608715:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.608810:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.608903:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.608997:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.609092:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.609185:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.609335:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.609426:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.609586:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.609695:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.609787:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.609861:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.609933:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610005:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610091:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610174:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610273:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610346:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610418:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610492:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610583:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610658:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610760:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610845:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610920:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.610992:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.611080:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.611172:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.611290:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.611393:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.611495:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.611595:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.611706:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.611807:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.611909:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.612022:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.612123:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.612225:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.612355:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.612457:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.612558:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.612657:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.612769:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.612870:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.612971:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.613071:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.613171:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.613357:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.613460:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.613562:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.613665:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.613778:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.613879:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.613980:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.614077:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.614172:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.614287:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.614382:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.614483:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.614582:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.614693:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.614794:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.614895:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.614997:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.615105:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.615204:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.615326:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.615427:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.615528:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.615622:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.615722:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.615818:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.615921:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.616024:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.616131:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.616229:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.616351:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.616453:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.616553:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.616656:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.616763:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.616864:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.616965:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.617064:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.617164:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.617481:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.617578:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.617669:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.617775:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.617871:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.617967:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618042:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618139:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618247:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618322:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618394:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618469:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618564:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618649:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618753:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618832:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618905:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.618977:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619053:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619144:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619230:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619319:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619392:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619465:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619546:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619638:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619732:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619823:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619896:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.619970:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.620061:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.620156:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.620277:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.620375:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.620471:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.620562:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.620635:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.620747:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.620843:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.620938:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.621028:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.621123:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.621221:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.621403:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.621501:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.621613:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.621719:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.621817:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.621891:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.621964:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.622044:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.622139:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.622256:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.622353:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.622448:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.622545:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.622637:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.622740:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.622836:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.622910:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.622981:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.623059:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.623157:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.623277:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.623375:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.623466:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.623549:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.623638:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.623746:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.623842:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.623930:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.624003:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.624088:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.624185:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.624372:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.624482:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.624578:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.624669:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.624777:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.624851:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.624936:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.625009:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.625094:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.625192:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.625319:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.625392:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.625464:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.625538:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.625617:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.625772:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.625869:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.625964:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.626039:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.626136:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.626257:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.626365:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.626460:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.626554:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.626644:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.626749:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.626830:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.626914:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.626998:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.627082:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.627179:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.627300:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.627395:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.627492:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.627589:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.627693:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.627790:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.627884:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.627981:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.628078:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.628175:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.628295:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.628372:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.628452:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.628558:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.628653:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.628758:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.628861:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.628946:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.629033:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.629141:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.629259:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.629357:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.629446:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.629544:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.629641:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.629751:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.629849:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.629934:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.630020:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.630111:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.630209:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.630311:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.630392:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.630489:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.630580:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.630668:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.630775:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.630869:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.630966:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.631063:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.631161:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.631278:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.631370:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.631465:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.631561:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.631656:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.631764:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.631859:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.631942:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.632028:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.632122:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.632220:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.632332:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.632429:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.632528:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.632623:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.632795:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.632891:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.632986:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.633081:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.633175:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.633287:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.633381:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.633479:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.633641:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.633748:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.633846:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.633941:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.634039:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.634134:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.634229:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.634343:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.634438:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.634535:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.634696:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.634793:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.634889:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.634993:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.635091:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.635184:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.635300:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.635393:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.635488:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.635638:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.635744:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.635840:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.635934:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.636027:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.636100:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.636184:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.636283:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.636356:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.636438:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.636535:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.636692:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.636790:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.636886:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.636981:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.637075:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.637171:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.637285:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.637380:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.637475:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.637570:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.637734:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.637833:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.637939:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.638036:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.638134:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.638227:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.638340:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.638437:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.638527:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.638674:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.638781:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.638867:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.638961:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639044:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639117:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639189:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639279:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639356:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639452:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639537:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639658:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639763:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639839:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639911:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.639982:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640056:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640127:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640199:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640288:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640361:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640444:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640539:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640656:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640760:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640833:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640905:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.640977:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.641051:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.641123:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.641195:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.641604:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.641711:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.641805:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.641902:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.641985:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.642058:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.642130:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.642202:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.642367:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.642455:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.642529:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.642653:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.642760:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.642855:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.642935:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.643017:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.643111:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.643184:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.643507:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.643624:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.643735:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.643810:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.643898:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.644002:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.644102:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.644200:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.644315:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.644411:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.644505:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.644601:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.644710:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.644806:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.644903:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.645001:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.645097:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.645192:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.645315:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.645411:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.645508:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.645606:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.645717:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.645812:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.645911:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.646000:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.646083:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.646162:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.646254:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.646326:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.646428:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.646521:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.646614:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.646721:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.646813:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.647119:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.647209:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.647322:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.647416:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.647505:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.647592:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.647673:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.647780:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.647884:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.647959:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648044:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648141:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648228:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648328:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648416:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648513:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648588:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648660:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648748:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648821:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648895:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.648975:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.649071:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.649168:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.649281:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.649370:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.649460:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.649555:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.649652:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.649736:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.649808:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.649893:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.649990:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.650088:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.650185:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.650302:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.650399:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.650497:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.650595:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.650669:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.650755:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.650827:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.650899:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.650979:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.651076:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.651173:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.651292:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.651384:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.651475:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.651572:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.651670:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.651785:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.651870:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.651958:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.652061:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.652157:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.652272:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.652390:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.652496:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.652569:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.652642:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.652727:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.652800:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.652873:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.652945:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.653034:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.653131:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.653226:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.653352:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.653454:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.653556:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.653649:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.653755:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.653852:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.653943:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654036:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654131:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654222:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654312:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654385:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654475:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654572:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654662:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654758:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654831:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654903:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.654984:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.655075:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.655148:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.655220:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.655325:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.655422:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.655516:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.655621:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.655728:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.655826:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.655922:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.656019:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.656116:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.656211:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.656330:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.656425:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.656523:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.656619:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.656727:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.656823:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.656921:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.657018:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.657115:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.657212:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.657331:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.657429:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.657537:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.657634:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.657749:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.657845:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.657947:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.658043:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.658138:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.658261:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.658358:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.658456:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.658554:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.658651:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.658760:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.658850:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.658938:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.659032:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.659128:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.659218:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.659326:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.659420:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.659517:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.659614:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.659724:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.659820:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.659918:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.660015:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.660112:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.660209:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.660328:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.660425:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.660521:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.660619:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.660723:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.660818:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.660914:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.661011:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.661109:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.661207:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.661325:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.661422:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.661519:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.661617:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.661727:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.661823:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.661920:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.662018:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.662115:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.662205:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.662322:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.662399:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.662494:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.662591:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.662696:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.662790:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.662885:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.662980:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.663077:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.663174:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.663302:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.663398:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.663496:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.663593:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.663700:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.663797:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.663895:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.663991:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.664087:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.664184:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.664300:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.664396:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.664490:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.664585:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.664692:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.664788:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.664883:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.664975:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.665070:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.665166:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.665283:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.665380:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.665479:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.665577:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.665673:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.665778:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.665867:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.665963:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.666058:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.666163:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.666280:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.666378:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.666474:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.666571:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.666665:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.666773:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.666868:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.666964:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.667059:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.667156:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.667274:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.667368:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.667464:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.667562:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.667660:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.667767:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.667850:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.667939:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.668023:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.668118:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.668214:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.668325:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.668421:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.668519:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.668615:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.668720:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.668818:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.668914:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.669010:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.669103:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.669193:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.669305:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.669397:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.669486:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.669559:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.669633:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.669733:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.669828:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.669923:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.670001:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.670098:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.670187:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.670300:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.670391:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.670487:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.670568:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.670660:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.670748:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.670832:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.670928:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.671024:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.671122:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.671220:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.671332:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.671429:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.671528:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.671626:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.671747:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.671845:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.671946:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.672039:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.672137:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.672253:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.672350:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.672441:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.672514:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.672586:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.672661:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.672764:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.672857:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.672950:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.673046:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.673140:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.673213:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.673319:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.673401:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.673487:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.673581:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.673669:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.673777:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.673870:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.673944:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.674038:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.674133:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.674230:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.674346:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.674436:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.674509:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.674581:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.674654:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.674758:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.674843:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.674937:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.675033:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.675120:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.675216:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.675331:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.675429:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.675524:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.675620:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.675718:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.675811:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.675906:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.675999:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.676095:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.676182:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.676279:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.676361:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.676436:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.676519:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.676602:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.676706:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.676800:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.676893:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.676990:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.677097:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.677183:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.677282:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.677360:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.677442:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.677531:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.677627:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.677736:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.677831:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.677926:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.678020:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.678111:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.678206:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.678310:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.678383:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.678465:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.678553:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.678646:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.678741:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.678824:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.678921:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.679019:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.679117:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.679213:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.679332:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.679432:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.679524:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.679622:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.679731:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.679824:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.679897:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.679985:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.680073:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.680164:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.680281:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.680362:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.680437:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.680524:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.680618:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.680710:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.680796:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.680869:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.680953:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.681038:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.681131:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.681226:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.681350:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.681446:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.681535:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.681631:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.681737:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.681829:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.681924:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.682019:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.682115:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.682208:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.682320:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.682412:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.682515:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.682608:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.682713:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.682806:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.682901:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.682997:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.683093:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.683187:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.683300:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.683393:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.683488:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.683579:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.683686:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.683781:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.683878:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.683976:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.684072:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.684169:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.684287:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.684382:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.684478:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.684575:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.684668:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.684780:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.684876:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.684972:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.685068:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.685169:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.685287:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.685395:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.685489:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.685585:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.685695:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.685787:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.685883:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.685978:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.686075:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.686173:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.686289:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.686392:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.686491:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.686588:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.686702:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.686803:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.686902:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.687000:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.687096:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.687196:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.687319:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.687418:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.687519:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.687620:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.687731:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.687830:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.687931:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.688028:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.688124:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.688221:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.688352:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.688454:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.688554:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.688654:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.688770:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.688867:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.688962:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.689056:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.689156:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.689273:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.689368:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.689462:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.689562:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.689660:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.689772:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.689874:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.689973:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.690069:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.690159:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.690273:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.690369:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.690464:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.690559:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.690653:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.690758:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.690853:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.690948:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.691043:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.691127:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.691210:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.691332:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.691421:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.691503:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.691637:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.691721:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.691810:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.691905:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.692001:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.692097:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.692195:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.692315:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.692412:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.692508:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.692605:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.692716:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.692811:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.692906:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.693002:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.693099:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.693193:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.693315:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.693411:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.693507:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.693604:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.693709:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.693784:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.693856:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.693946:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.694058:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.694153:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.694274:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.694348:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.694430:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.694521:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.694611:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.694710:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.694798:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.694890:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.694986:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.695077:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.695166:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.695272:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.695362:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.695451:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.695545:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.695640:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.695750:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.695847:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.695941:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.696037:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.696131:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.696226:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.696340:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.696437:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.696534:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.696630:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.696739:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.696849:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.696945:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.697032:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.697120:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.697212:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.697337:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.697434:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.697527:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.697599:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.697671:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.697779:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.697877:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.697973:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.698070:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.698166:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.698287:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.698382:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.698475:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.698571:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.698666:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.698773:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.698872:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.698974:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.699069:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.699164:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.699282:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.699379:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.699472:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.699567:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.699693:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.699788:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.699891:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.699991:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.700087:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.700187:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.700311:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.700410:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.700510:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.700610:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.700721:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.700819:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.700920:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.701016:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.701117:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.701218:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.701336:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.701436:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.701533:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.701632:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.701743:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.701844:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.701946:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.702044:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.702141:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.702351:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.702448:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.702542:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.702635:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.702902:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.703000:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.703101:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.703200:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.703315:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.703412:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.703509:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.703604:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.703707:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.703802:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.703885:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.703975:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.704068:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.704163:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.704278:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.704373:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.704468:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.704553:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.704638:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.704730:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.704803:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.704875:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.704968:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.705062:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.705156:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.705275:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.705371:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.705468:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.705572:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.705697:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.705790:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.705868:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.705961:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.706056:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.706154:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.706273:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.706360:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.706444:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.706534:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.706607:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.706712:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.706804:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.706895:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.706983:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.707056:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.707150:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.707268:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.707363:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.707458:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.707556:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.707653:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.707759:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.707855:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.707951:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.708046:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.708143:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.708262:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.708361:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.708472:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.708570:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.708671:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.708783:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.708880:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.708975:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.709069:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.709163:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.709282:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.709386:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.709482:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.709578:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.709674:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.709779:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.709873:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.709969:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.710068:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.710167:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.710287:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.710382:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.710477:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.710572:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.710671:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.710776:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.710871:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.710968:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.711226:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.711338:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.711434:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.711536:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.711630:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.711739:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.711836:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.711929:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.712023:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.712119:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.712215:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.712331:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.712424:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.712519:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.712613:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.712727:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.712825:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.712924:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.713020:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.713117:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.713219:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.713342:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.713439:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.713532:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.713627:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.713747:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.713841:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.713937:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.714035:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.714126:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.714199:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.714309:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.714394:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.714466:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.714538:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.714621:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.714730:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.714826:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.714923:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.715019:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.715116:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.715213:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.715332:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.715426:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.715520:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.715617:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.715724:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.715817:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.715911:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.716006:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.716102:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.716196:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.716315:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.716411:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.716508:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.716605:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.716713:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.716810:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.716906:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.717001:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.717097:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.717208:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.717325:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.717419:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.717513:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.717595:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.717670:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.717767:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.717859:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.717946:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.718025:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.718118:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.718191:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.718315:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.718410:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.718506:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.718602:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.718710:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.718808:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.718911:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.719015:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.719114:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.719218:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.719335:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.719430:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.719525:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.719621:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.719729:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.719827:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.719923:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.720032:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.720131:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.720224:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.720342:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.720441:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.720536:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.720634:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.720746:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.720845:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.720947:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.721044:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.721142:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.721257:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.721353:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.721448:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.721543:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.721639:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.721747:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.721848:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.721948:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.722047:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.722153:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.722278:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.722373:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.722474:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.722575:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.722690:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.722787:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.722878:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.722990:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.723084:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.723177:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.723293:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.723387:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.723485:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.723581:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.723691:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.723788:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.723885:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.723983:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.724079:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.724174:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.724300:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.724404:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.724502:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.724600:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.724708:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.724805:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.724902:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.724999:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.725097:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.725194:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.725310:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.725407:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.725506:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.725610:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.725717:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.725811:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.725919:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.726012:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.726109:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.726204:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.726324:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.726420:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.726516:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.726613:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.726723:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.726816:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.726914:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.727013:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.727112:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.727209:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.727329:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.727424:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.727521:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.727615:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.727721:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.727816:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.727912:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.728010:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.728106:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.728206:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.728326:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.728421:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.728522:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.728624:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.728736:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.728845:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.728939:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.729033:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.729125:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.729219:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.729337:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.729411:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.729485:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.729578:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.729672:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.729781:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.729875:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.729972:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.730067:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.730162:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.730279:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.730372:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.730470:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.730564:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.730656:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.730765:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.730861:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.730958:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.731051:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.731146:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.731268:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.731360:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.731450:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.731545:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.731654:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.731763:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.731863:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.731961:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.732057:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.732155:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.732273:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.732369:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.732467:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.732561:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.732653:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.732762:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.732859:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.732955:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.733048:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.733144:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.733260:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.733354:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.733451:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.733546:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.733645:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.733758:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.733847:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.733939:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.734033:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.734130:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.734225:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.734344:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.734440:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.734546:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.734638:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.734745:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.734844:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.734941:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.735037:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.735132:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.735231:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.735351:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.735449:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.735546:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.735643:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.735751:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.735847:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.735928:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.736015:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.736111:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.736203:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.736321:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.736419:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.736518:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.736611:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.736711:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.736806:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.736897:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.736991:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.737087:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.737163:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.737277:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.737374:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.737460:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.737555:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.737650:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.737755:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.737838:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.737916:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.737993:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.738090:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.738184:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.738303:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.738395:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.738490:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.738575:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.738648:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.738746:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.738819:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.738893:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.738966:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.739043:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.739131:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.739219:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.739338:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.739436:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.739534:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.739633:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.739737:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.739834:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.739932:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.740043:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.740141:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.740259:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.740348:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.740423:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.740509:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.740582:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.740686:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.740784:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.740881:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.740977:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.741073:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.741169:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.741280:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.741353:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.741425:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.741516:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.741604:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.741696:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.741790:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.741880:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.741975:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.742071:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.742163:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.742280:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.742375:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.742467:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.742564:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.742661:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.742780:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.742873:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.742965:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.743053:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.743139:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.743258:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.743357:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.743446:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.743543:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.743634:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.743743:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.743835:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.743920:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.744013:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.744110:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.744209:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.744327:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.744417:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.744507:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.744600:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.744702:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.744791:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.744879:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.744968:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.745062:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.745158:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.745276:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.745375:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.745473:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.745583:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.745690:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.745785:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.745882:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.745980:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.746077:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.746172:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.746292:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.746388:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.746484:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.746581:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.746690:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.746788:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.746887:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.746975:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.747073:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.747172:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.747292:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.747391:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.747489:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.747587:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.747698:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.747796:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.747888:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.747984:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.748081:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.748171:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.748280:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.748370:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.748478:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.748575:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.748672:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.748775:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.748871:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.748970:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.749068:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.749164:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.749278:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.749367:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.749455:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.749548:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.749642:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.749743:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.749835:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.749931:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.750028:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.750124:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.750221:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.750338:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.750435:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.750533:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.750630:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.750741:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.750837:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.750935:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.751033:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.751132:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.751229:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.751361:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.751457:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.751556:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.751654:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.751770:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.751867:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.751965:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.752063:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.752162:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.752278:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.752376:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.752473:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.752571:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.752669:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.752779:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.752878:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.752975:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.753072:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.753171:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.753296:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.753394:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.753492:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.753591:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.753701:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.753796:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.753892:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.753998:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.754079:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.754152:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.754266:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.754364:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.754460:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.754557:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.754630:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.754724:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.754818:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.754913:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.755008:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.755099:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.755197:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.755308:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.755398:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.755490:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.755587:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.755695:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.755768:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.755844:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.755934:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.756024:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.756120:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.756216:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.756335:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.756432:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.756529:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.756622:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.756724:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.756822:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.756918:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.757029:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.757127:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.757216:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.757349:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.757438:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.757523:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.757607:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.757716:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.757809:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.757899:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.757993:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.758090:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.758185:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.758300:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.758396:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.758490:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.758596:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.758704:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.758801:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.758898:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.758993:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.759091:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.759187:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.759304:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.759396:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.759484:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.759576:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.759670:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.759777:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.759885:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.759983:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.760082:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.760178:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.760291:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.760387:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.760485:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.760582:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.760693:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.760789:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.760886:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.760983:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.761080:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.761175:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.761289:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.761387:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.761485:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.761580:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.761689:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.761784:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.761874:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.761972:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.762070:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.762167:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.762284:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.762381:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.762479:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.762576:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.762686:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.762796:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.762897:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.762997:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.763095:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.763194:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.763316:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.763411:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.763508:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.763606:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.763713:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.763811:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.763906:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.764003:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.764100:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.764199:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.764312:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.764409:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.764505:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.764602:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.764715:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.764812:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.764909:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.765008:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.765106:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.765204:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.765324:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.765420:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.765516:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.765610:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.765729:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.765823:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.765913:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.766011:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.766108:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.766203:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.766313:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.766404:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.766498:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.766592:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.766697:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.766792:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.766886:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.766981:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.767080:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.767179:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.767295:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.767391:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.767486:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.767580:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.767674:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.767782:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.767878:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.767973:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.768069:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.768167:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.768282:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.768376:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.768472:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.768579:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.768685:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.768782:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.768878:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.768973:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.769070:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.769166:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.769285:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.769380:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.769474:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.769563:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.769652:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.769751:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.769845:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.769934:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.770025:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.770119:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.770214:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.770328:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.770422:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.770512:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.770605:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.770711:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.770807:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.770900:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.770992:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.771085:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.771177:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.771290:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.771397:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.771494:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.771593:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.771705:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.771801:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.771895:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.771991:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.772089:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.772186:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.772303:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.772399:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.772494:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.772590:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.772698:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.772794:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.772891:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.772984:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.773081:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.773174:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.773265:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.773337:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.773422:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.773517:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.773613:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.773720:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.773814:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.773908:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.774001:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.774098:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.774206:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.774306:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.774381:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.774469:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.774565:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.774661:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.774771:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.774866:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.774964:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.775058:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.775154:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.775268:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.775358:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.775446:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.775535:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.775623:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.775724:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.775820:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.775912:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.776007:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.776103:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.776197:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.776313:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.776406:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.776502:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.776595:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.776698:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.776793:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.776889:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.776994:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.777085:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.777177:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.777288:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.777378:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.777474:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.777571:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.777667:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.777776:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.777873:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.777971:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.778047:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.778140:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.778254:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.778352:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.778448:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.778540:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.778631:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.778736:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.778830:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.778924:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.779018:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.779113:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.779201:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.779314:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.779410:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.779505:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.779603:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.779710:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.779817:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.779912:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.780009:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.780102:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.780200:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.780311:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.780400:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.780492:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.780586:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.780692:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.780785:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.780885:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.780982:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.781077:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.781167:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.781291:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.781384:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.781479:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.781571:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.781666:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.781772:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.781868:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.781962:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.782057:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.782153:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.782269:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.782361:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.782460:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.782558:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.782672:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.782865:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.782956:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.783050:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.783141:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.783231:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.783396:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.783494:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.783589:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.783694:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.783790:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.783881:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.783975:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.784067:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.784155:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.784398:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.784494:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.784591:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.784697:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.784792:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.784883:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.784979:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.785894:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.785991:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.786156:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.786339:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.786438:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.786532:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.786644:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.786762:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.786867:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.786959:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.787051:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.787378:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.787475:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.787574:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.787670:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.787776:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.787929:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.788026:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.788187:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.788370:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.788466:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.788562:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.788654:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.789193:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.789364:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.789458:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.789553:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.789648:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.789756:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.789850:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.789944:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.790040:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.790357:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.790455:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.790549:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.790643:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.790764:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.790921:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.791019:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.791179:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.791362:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.791460:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.791556:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.791656:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.791764:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.791875:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.791973:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.792125:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.792222:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.792392:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.792487:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.792583:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.792690:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.792789:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.792888:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.792989:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.793149:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.793325:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.793420:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.793517:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.793613:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.793719:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.793815:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.793912:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.794010:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.794182:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.794381:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.794477:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.794571:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.794666:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.794772:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.794866:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.794962:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.795123:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.795218:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.795396:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.795493:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.795589:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.795694:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.795789:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.795887:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.795982:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.796140:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.796263:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.796354:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.796444:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.796534:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.796628:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.796733:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.796828:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.796918:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.797007:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.797156:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.797265:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.797369:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.797459:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.797548:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.797642:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.797746:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.797836:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.797926:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.798015:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.798160:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.798269:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.798359:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.798450:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.798540:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.798632:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.798736:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.798828:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.798919:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.799011:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.799157:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.799267:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.799356:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.799445:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.799533:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.799624:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.799729:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.799821:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.799911:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.800007:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.800167:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.800299:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.800395:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.800490:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.800582:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.800687:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.800784:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.800878:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.800973:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.801135:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.801228:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.801346:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.801442:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.801535:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.801630:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.801739:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.801834:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.801930:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.802025:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.802183:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.802300:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.802393:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.802489:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.802585:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.802691:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.802788:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.802887:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.802983:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.803141:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.803261:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.803376:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.803476:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.803573:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.803670:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.803776:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.803877:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.803977:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.804139:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.804259:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.804352:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.804449:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.804547:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.804648:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.804756:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.804848:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.804952:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.805053:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.805219:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.805339:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.805436:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.805536:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.805634:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.805742:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.805841:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.805939:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.806033:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.806191:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.806308:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.806402:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.806508:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.806596:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.806699:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.806796:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.806891:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.806988:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.807152:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.807273:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.807370:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.807466:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.807560:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.807654:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.807759:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.807853:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.807947:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808040:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808163:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808252:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808324:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808397:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808468:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808541:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808614:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808720:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808809:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808882:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.808954:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809027:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809145:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809229:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809320:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809393:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809466:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809538:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809611:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809713:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809808:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809899:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.809990:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.810140:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.810259:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.810361:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.810458:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.810539:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.810612:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.810716:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.810813:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.810905:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.810984:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.811147:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.811257:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.811347:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.811421:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.811510:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.811603:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.811710:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.811803:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.811899:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.812001:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.812151:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.812262:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.812355:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.812449:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.812542:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.812639:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.812748:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.812845:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.812931:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.813023:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.813179:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.813295:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.813390:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.813485:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.813582:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.813699:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.813795:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.813892:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.813988:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.814146:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.814225:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.814331:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.814429:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.814527:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.814624:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.814732:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.814829:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.814926:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.815035:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.815193:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.815312:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.815412:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.815507:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.815604:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.815712:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.815809:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.815904:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.816001:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.816155:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.816274:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.816371:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.816466:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.816559:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.816655:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.816762:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.816856:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.816952:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.817049:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.817213:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.817329:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.817425:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.817522:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.817615:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.817722:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.817817:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.817914:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.818009:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.818192:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.818315:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.818416:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.818516:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.818615:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.818723:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.818815:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.818908:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.819004:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.819169:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.819287:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.819384:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.819481:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.819577:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.819673:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.819781:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.819879:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.819975:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.820131:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.820226:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.820342:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.820438:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.820530:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.820624:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.820727:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.820817:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.820908:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.821003:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.821160:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.821293:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.821388:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.821474:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.821547:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.821634:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.821735:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.821830:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.821919:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.822011:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.822104:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.822199:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.822301:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.822398:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.822493:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.822588:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.822697:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.822791:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.822885:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.822977:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.823070:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.823164:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.823283:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.823380:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.823477:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.823573:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.823669:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.823776:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.823850:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.823931:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.824014:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.824101:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.824198:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.824310:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.824408:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.824503:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.824601:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.824703:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.824776:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.824848:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.824942:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.825037:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.825133:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.825228:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.825341:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.825434:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.825530:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.825626:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.825735:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.825830:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.825926:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.826021:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.826117:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.826213:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.826326:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.826422:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.826519:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.826614:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.826718:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.826825:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.826920:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.827013:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.827108:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.827205:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.827395:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.827490:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.827592:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.827700:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.827794:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.827874:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.827954:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.828044:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.828117:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.828206:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.828322:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.828419:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.828513:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.828607:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.828710:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.828785:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.828857:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.828948:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.829043:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.829116:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.829202:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.829315:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.829410:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.829504:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.829611:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.829719:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.829815:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.829898:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.829988:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.830079:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.830185:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.830305:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.830399:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.830491:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.830585:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.830692:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.830784:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.830882:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.830979:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.831072:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.831161:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.831278:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.831374:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.831470:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.831563:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.831653:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.831757:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.831848:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.831943:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.832038:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.832131:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.832230:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.832342:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.832447:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.832544:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.832638:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.832741:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.832832:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.832916:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.833006:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.833102:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:00000040:2.0:1788880138.833198:0:201223:0:(lnet-crypto.c:95:cfs_crypto_hash_alloc()) Using crypto hash: crc32c (crc32c-intel) speed 0 MB/s 00000001:01000000:2.0:1788880138.833315:0:201223:0:(lnet-crypto.c:359:cfs_crypto_performance_test()) Crypto hash algorithm crc32c speed = 9963 MB/s 00000400:00000010:2.0:1788880138.833324:0:201223:0:(lock.c:50:cfs_percpt_lock_create()) alloc '(pcl)': 24 at 00000000b9458df6. 00000400:00000010:2.0:1788880138.833330:0:201223:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000068d73fd6. 00000400:00000010:2.0:1788880138.833334:0:201223:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000009b00b013. 00000400:00000010:2.0:1788880138.833337:0:201223:0:(lock.c:50:cfs_percpt_lock_create()) alloc '(pcl)': 24 at 00000000936a1e94. 00000400:00000010:2.0:1788880138.833338:0:201223:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000079877f70. 00000400:00000010:2.0:1788880138.833340:0:201223:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000037b6be71. 00000400:00000200:2.0:1788880138.833829:0:201223:0:(api-ni.c:978:lnet_register_lnd()) lo LND registered 00000400:00000001:2.0:1788880138.834056:0:201223:0:(module.c:496:lnet_init()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0F:1788880138.890545:0:201241:0:(api-ni.c:978:lnet_register_lnd()) tcp LND registered 00000020:02000000:0.0:1788880138.967016:0:201254:0:(class_obd.c:804:obdclass_init()) Lustre: Build Version: 2.17.55_24_g1bda257 00000020:00000040:0.0:1788880138.969284:0:201254:0:(class_obd.c:593:obd_init_checks()) OBD_OBJECT_EOF = 0xffffffffffffffff 00000020:00000040:0.0:1788880138.969290:0:201254:0:(class_obd.c:596:obd_init_checks()) u64val OBD_OBJECT_EOF = 0xffffffffffffffff 00000020:00000040:0.0:1788880138.969292:0:201254:0:(class_obd.c:609:obd_init_checks()) u64val OBD_OBJECT_EOF = 0xffffffffffffffff 00000020:00000040:0.0:1788880138.969300:0:201254:0:(lprocfs_status.c:2206:string_to_blksize()) unknown suffix 'OpQ' 00000020:00000040:0.0:1788880138.969303:0:201254:0:(lprocfs_status.c:2206:string_to_blksize()) unknown suffix 'badMib' 00000020:00000040:0.0:1788880138.969307:0:201254:0:(lprocfs_status.c:2206:string_to_blksize()) unknown suffix 'badMib' 00000020:00000010:0.0:1788880138.976003:0:201254:0:(lustre_handles.c:164:class_handle_init()) vmalloced 'handle_hash': 8388608 at 00000000d0c8f975. 00000020:00000001:0.0:1788880138.978867:0:201254:0:(genops.c:791:obd_init_caches()) Process entered 00000020:00000001:0.0:1788880138.978926:0:201254:0:(genops.c:800:obd_init_caches()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880138.978929:0:201254:0:(obd_sysfs.c:853:class_procfs_init()) Process entered 00000100:00000001:0.0:1788880138.978965:0:201254:0:(jobid.c:796:jobid_cache_init()) Process entered 00000001:00000001:0.0:1788880138.978966:0:201254:0:(hash.c:1038:cfs_hash_create()) Process entered 00000001:00000010:0.0:1788880138.978969:0:201254:0:(hash.c:1065:cfs_hash_create()) alloc '(hs)': 232 at 0000000075c0ee36. 00000001:00000010:0.0:1788880138.978974:0:201254:0:(hash.c:933:cfs_hash_buckets_realloc()) alloc '(new_bkts)': 32 at 0000000006a9c28b. 00000001:00000010:0.0:1788880138.978976:0:201254:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000033b4d833. 00000001:00000010:0.0:1788880138.978979:0:201254:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 00000000c437a0bb. 00000001:00000010:0.0:1788880138.978981:0:201254:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 00000000845dcf9e. 00000001:00000010:0.0:1788880138.978982:0:201254:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 000000009c34fa6b. 00000100:00000001:0.0:1788880138.978989:0:201254:0:(jobid.c:816:jobid_cache_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880138.979017:0:201254:0:(obd_sysfs.c:891:class_procfs_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880138.979019:0:201254:0:(lu_object.c:2257:lu_global_init()) Lustre LU module (00000000b142328f). 00000020:00000010:0.0:1788880138.979024:0:201254:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000004f70e70f. 00000020:00000010:0.0:1788880138.979027:0:201254:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000c3d2e3ac. 00000020:00000010:0.0:1788880138.979030:0:201254:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000746952d0. 00000020:00000010:0.0:1788880138.979032:0:201254:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 0000000033a1f369. 00000001:00000010:0.0:1788880138.979036:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 0000000090873ff8. 00000020:00000010:0.0:1788880138.979074:0:201254:0:(cl_object.c:1176:cl_global_init()) kmalloced '(cl_envs)': 192 at 000000001ab36cb1. 00000020:00000010:0.0:1788880138.979166:0:201254:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000b63d40ce. 00000020:00000010:0.0:1788880138.979169:0:201254:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000de392594. 00000020:00000010:0.0:1788880138.979171:0:201254:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000a25dffc9. 00000020:00000010:0.0:1788880138.979173:0:201254:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000003365b6de. 00000020:00000010:0.0:1788880138.979175:0:201254:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000008dfb2b1d. 00000020:00000010:0.0:1788880138.979178:0:201254:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000009412bffb. 00000020:00000010:0.0:1788880138.979179:0:201254:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 0000000015088f03. 00000020:00000010:0.0:1788880138.979192:0:201254:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000e0dddc05. 00000020:00000010:0.0:1788880138.979194:0:201254:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000008ab72166. 00000020:00000010:0.0:1788880138.979195:0:201254:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000000dcd5551. 00000020:00000010:0.0:1788880138.979196:0:201254:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 000000004eea5b63. 00000020:00000010:0.0:1788880138.979198:0:201254:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000004eb455ee. 00000020:00000010:0.0:1788880138.979199:0:201254:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000a3cc3921. 00000020:00000010:0.0:1788880138.979200:0:201254:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000494ddd38. 00000020:00000010:0.0:1788880138.979201:0:201254:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000e16ad7f1. 00000020:00000010:0.0:1788880138.979202:0:201254:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000022311ab4. 02000000:00000001:0.0:1788880138.979205:0:201254:0:(page_pools.c:1080:obd_pool_init()) Process entered 02000000:00000010:0.0:1788880138.979207:0:201254:0:(page_pools.c:1088:obd_pool_init()) kmalloced '(page_pools)': 112 at 0000000055d5751c. 02000000:00000010:0.0:1788880138.979208:0:201254:0:(page_pools.c:1092:obd_pool_init()) kmalloced '(pool_shrinkers)': 112 at 00000000bf2659f0. 02000000:00000010:0.0:1788880138.979216:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 000000008608e97b. 02000000:00000010:0.0:1788880138.979219:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 1056 at 00000000fd75469e. 02000000:08000000:0.0:1788880138.979221:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 0 00000001:00000010:0.0:1788880138.979223:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 00000000232e2652. 00000020:00000010:0.0:1788880138.979281:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 000000006bfdeb70. 00000020:00000010:0.0:1788880138.979291:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 000000006bfdeb70. 02000000:00000010:0.0:1788880138.979296:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 00000000aa1f31df. 02000000:00000010:0.0:1788880138.979299:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 528 at 000000003d3de3dc. 02000000:08000000:0.0:1788880138.979300:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 1 00000001:00000010:0.0:1788880138.979302:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 0000000098129532. 00000020:00000010:0.0:1788880138.979326:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 000000006bfdeb70. 00000020:00000010:0.0:1788880138.979333:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 000000006bfdeb70. 02000000:00000010:0.0:1788880138.979336:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 0000000082d745e4. 02000000:00000010:0.0:1788880138.979339:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 264 at 00000000a944fc16. 02000000:08000000:0.0:1788880138.979340:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 2 00000001:00000010:0.0:1788880138.979342:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 0000000004c153e5. 00000020:00000010:0.0:1788880138.979365:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 000000006bfdeb70. 00000020:00000010:0.0:1788880138.979374:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 000000006bfdeb70. 02000000:00000010:0.0:1788880138.979377:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 000000009a26169d. 02000000:00000010:0.0:1788880138.979379:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 136 at 000000006bfdeb70. 02000000:08000000:0.0:1788880138.979386:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 3 00000001:00000010:0.0:1788880138.979388:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 00000000b97bd069. 00000020:00000010:0.0:1788880138.979410:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 00000000c3ae0def. 00000020:00000010:0.0:1788880138.979415:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 00000000c3ae0def. 02000000:00000010:0.0:1788880138.979418:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 00000000b795fca8. 02000000:00000010:0.0:1788880138.979419:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 72 at 0000000056120010. 02000000:08000000:0.0:1788880138.979420:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 4 00000001:00000010:0.0:1788880138.979421:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 00000000b3caf59d. 00000020:00000010:0.0:1788880138.979434:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 00000000c3ae0def. 00000020:00000010:0.0:1788880138.979439:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 00000000c3ae0def. 02000000:00000010:0.0:1788880138.979440:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 00000000e28b7517. 02000000:00000010:0.0:1788880138.979442:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 40 at 00000000d29702fe. 02000000:08000000:0.0:1788880138.979443:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 5 00000001:00000010:0.0:1788880138.979444:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 000000006c274b78. 00000020:00000010:0.0:1788880138.979454:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 00000000c3ae0def. 00000020:00000010:0.0:1788880138.979459:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 00000000c3ae0def. 02000000:00000010:0.0:1788880138.979460:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 00000000b9c52f6a. 02000000:00000010:0.0:1788880138.979461:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 24 at 000000004c9eb5c5. 02000000:08000000:0.0:1788880138.979462:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 6 00000001:00000010:0.0:1788880138.979463:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 0000000038444350. 00000020:00000010:0.0:1788880138.979474:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 00000000c3ae0def. 00000020:00000010:0.0:1788880138.979478:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 00000000c3ae0def. 02000000:00000010:0.0:1788880138.979480:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 00000000e65a3dc6. 02000000:00000010:0.0:1788880138.979481:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 16 at 00000000082e2e0f. 02000000:08000000:0.0:1788880138.979482:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 7 00000001:00000010:0.0:1788880138.979483:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 00000000617b4359. 00000020:00000010:0.0:1788880138.979492:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 00000000c3ae0def. 00000020:00000010:0.0:1788880138.979496:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 00000000c3ae0def. 02000000:00000010:0.0:1788880138.979498:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 000000003d5d873c. 02000000:00000010:0.0:1788880138.979499:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 8 at 00000000448a032f. 02000000:08000000:0.0:1788880138.979500:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 8 00000001:00000010:0.0:1788880138.979501:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 0000000018299351. 00000020:00000010:0.0:1788880138.979515:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 00000000c3ae0def. 00000020:00000010:0.0:1788880138.979520:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 00000000c3ae0def. 02000000:00000010:0.0:1788880138.979522:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 000000008b7cbe6c. 02000000:00000010:0.0:1788880138.979523:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 8 at 00000000903ff59b. 02000000:08000000:0.0:1788880138.979525:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 9 00000001:00000010:0.0:1788880138.979526:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 000000003fee754c. 00000020:00000010:0.0:1788880138.979547:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 00000000c3ae0def. 00000020:00000010:0.0:1788880138.979555:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 00000000c3ae0def. 02000000:00000010:0.0:1788880138.979559:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 00000000d58aff77. 02000000:00000010:0.0:1788880138.979561:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 8 at 00000000b639e6cb. 02000000:08000000:0.0:1788880138.979562:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 10 00000001:00000010:0.0:1788880138.979563:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 00000000d03b85cc. 00000020:00000010:0.0:1788880138.979583:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 00000000c3ae0def. 00000020:00000010:0.0:1788880138.979590:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 00000000c3ae0def. 02000000:00000010:0.0:1788880138.979593:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 00000000c83717b9. 02000000:00000010:0.0:1788880138.979595:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 8 at 00000000c83a6de5. 02000000:08000000:0.0:1788880138.979596:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 11 00000001:00000010:0.0:1788880138.979605:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 000000003d89f726. 00000020:00000010:0.0:1788880138.979628:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 00000000c3ae0def. 00000020:00000010:0.0:1788880138.979637:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 00000000c3ae0def. 02000000:00000010:0.0:1788880138.979640:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 0000000075f4bd35. 02000000:00000010:0.0:1788880138.979642:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 8 at 000000003ec37524. 02000000:08000000:0.0:1788880138.979643:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 12 00000001:00000010:0.0:1788880138.979645:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 00000000401cf1bd. 00000020:00000010:0.0:1788880138.979665:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 00000000c3ae0def. 00000020:00000010:0.0:1788880138.979683:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 00000000c3ae0def. 02000000:00000010:0.0:1788880138.979687:0:201254:0:(page_pools.c:1099:obd_pool_init()) kmalloced '(page_pools[pool_order])': 296 at 000000009ad973a3. 02000000:00000010:0.0:1788880138.979689:0:201254:0:(page_pools.c:1059:pool_ptrs_alloc()) kmalloced '(pool->opp_ptr_pages)': 8 at 00000000db427585. 02000000:08000000:0.0:1788880138.979690:0:201254:0:(page_pools.c:1130:obd_pool_init()) Allocated pool 13 00000001:00000010:0.0:1788880138.979692:0:201254:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 00000000e6f5d83f. 00000020:00000010:0.0:1788880138.979714:0:201254:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 00000000c3ae0def. 00000020:00000010:0.0:1788880138.979722:0:201254:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 00000000c3ae0def. 02000000:00000001:0.0:1788880138.979731:0:201254:0:(page_pools.c:1157:obd_pool_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880139.071808:0:201269:0:(ptlrpc_module.c:28:ptlrpc_init()) Process entered 00000100:00000001:3.0:1788880139.071840:0:201269:0:(service.c:3561:ptlrpc_hr_init()) Process entered 00000400:00000010:3.0:1788880139.071844:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000e4dac910. 00000400:00000010:3.0:1788880139.071852:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000079773a89. 00000100:00000010:3.0:1788880139.071858:0:201269:0:(service.c:3592:ptlrpc_hr_init()) kmalloced '(hrp->hrp_thrs)': 192 at 00000000fc4f601c. 00000100:00000001:3.0:1788880139.071861:0:201269:0:(service.c:3320:ptlrpc_start_hr_threads()) Process entered 00000100:00000010:0.0:1788880139.072171:0:201271:0:(service.c:3235:ptlrpc_hr_main()) kmalloced '(env)': 56 at 0000000017118026. 00000020:00000010:0.0:1788880139.072179:0:201271:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000b7c6d183. 00000020:00000010:0.0:1788880139.072185:0:201271:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000004c056b50. 00000040:00000010:0.0:1788880139.072190:0:201271:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000bc85b003. 00000020:00000010:0.0:1788880139.072195:0:201271:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 0000000038d164df. 00000020:00000010:0.0:1788880139.072200:0:201271:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 00000000bbf28410. 00000100:00000010:3.0:1788880139.072366:0:201272:0:(service.c:3235:ptlrpc_hr_main()) kmalloced '(env)': 56 at 00000000e22cc1fe. 00000020:00000010:3.0:1788880139.072371:0:201272:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000087fd1a45. 00000020:00000010:3.0:1788880139.072373:0:201272:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000b9943d5f. 00000040:00000010:3.0:1788880139.072375:0:201272:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 0000000091b2ffe6. 00000020:00000010:3.0:1788880139.072377:0:201272:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 0000000038274ee2. 00000020:00000010:3.0:1788880139.072379:0:201272:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 00000000fab4c0fd. 00000100:00000001:0.0:1788880139.072416:0:201269:0:(service.c:3351:ptlrpc_start_hr_threads()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.072418:0:201269:0:(service.c:3611:ptlrpc_hr_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880139.072453:0:201269:0:(events.c:596:ptlrpc_ni_init()) My pid is: 3039 00000400:00001000:0.0:1788880139.072455:0:201269:0:(api-ni.c:3274:LNetNIInit()) refs 0 00000400:00000010:0.0:1788880139.072631:0:201269:0:(api-ni.c:675:lnet_create_remote_nets_table()) alloc '(hash)': 1024 at 000000006133e67c. 00000400:00000010:0.0:1788880139.072633:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000ea5a7549. 00000400:00000010:0.0:1788880139.072636:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 128 at 00000000e2ab9ad6. 00000400:00000010:0.0:1788880139.072637:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000fafa100a. 00000400:00000010:0.0:1788880139.072638:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 128 at 00000000309e8cf4. 00000400:00000010:0.0:1788880139.072642:0:201269:0:(peer.c:109:lnet_peer_tables_create()) alloc '(hash)': 8192 at 000000002bc46b4c. 00000400:00000010:0.0:1788880139.072645:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000f6a10192. 00000400:00000010:0.0:1788880139.072646:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 128 at 00000000aee6d07c. 00000400:00000010:0.0:1788880139.072649:0:201269:0:(lib-msg.c:1338:lnet_msg_container_setup()) alloc '(container->msc_finalizers)': 32 at 00000000f91cf735. 00000400:00000010:0.0:1788880139.072651:0:201269:0:(lib-msg.c:1348:lnet_msg_container_setup()) alloc '(container->msc_resenders)': 32 at 0000000054a85b91. 00000400:00000010:0.0:1788880139.072720:0:201269:0:(api-ni.c:1170:lnet_res_container_setup()) alloc '(rec->rec_lh_hash)': 65536 at 00000000b675ac90. 00000400:00000010:0.0:1788880139.072735:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000094baf73b. 00000400:00000010:0.0:1788880139.072736:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000002fddb8c8. 00000400:00000010:0.0:1788880139.072777:0:201269:0:(api-ni.c:1170:lnet_res_container_setup()) alloc '(rec->rec_lh_hash)': 65536 at 00000000d6cffbfc. 00000400:00000010:0.0:1788880139.072792:0:201269:0:(lib-ptl.c:864:lnet_portals_create()) alloc '(the_lnet.ln_portals)': 512 at 000000005eee6da4. 00000400:00000010:0.0:1788880139.072794:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000471e2ed5. 00000400:00000010:0.0:1788880139.072796:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000086aa230e. 00000400:00000010:0.0:1788880139.072797:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000466a960d. 00000400:00000010:0.0:1788880139.072800:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000004f188310. 00000400:00000010:0.0:1788880139.072801:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000001e06c7c1. 00000400:00000010:0.0:1788880139.072802:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000f3d42d92. 00000400:00000010:0.0:1788880139.072803:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000002ea99a86. 00000400:00000010:0.0:1788880139.072809:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000c89d1209. 00000400:00000010:0.0:1788880139.072811:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000008eb3ba1b. 00000400:00000010:0.0:1788880139.072812:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000068307c0d. 00000400:00000010:0.0:1788880139.072813:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000002cc51cd8. 00000400:00000010:0.0:1788880139.072816:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 0000000005b74f7e. 00000400:00000010:0.0:1788880139.072817:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000005c0b1383. 00000400:00000010:0.0:1788880139.072818:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000000a6e3a6a. 00000400:00000010:0.0:1788880139.072819:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000058a4b15. 00000400:00000010:0.0:1788880139.072822:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000b2f8750c. 00000400:00000010:0.0:1788880139.072823:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000c8e6f852. 00000400:00000010:0.0:1788880139.072824:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000cf0c1497. 00000400:00000010:0.0:1788880139.072825:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000e6c53c40. 00000400:00000010:0.0:1788880139.072828:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000aa69cbd0. 00000400:00000010:0.0:1788880139.072829:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000080f59c1f. 00000400:00000010:0.0:1788880139.072830:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000007426d0b9. 00000400:00000010:0.0:1788880139.072831:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000e4ec6638. 00000400:00000010:0.0:1788880139.072833:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 0000000056142682. 00000400:00000010:0.0:1788880139.072835:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000097734427. 00000400:00000010:0.0:1788880139.072836:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000074205620. 00000400:00000010:0.0:1788880139.072836:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000039ce1f8b. 00000400:00000010:0.0:1788880139.072839:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000006a06a484. 00000400:00000010:0.0:1788880139.072840:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000007b0ffb23. 00000400:00000010:0.0:1788880139.072841:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000000f5bb17e. 00000400:00000010:0.0:1788880139.072842:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000008428675c. 00000400:00000010:0.0:1788880139.072845:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000c529280e. 00000400:00000010:0.0:1788880139.072846:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000bae25990. 00000400:00000010:0.0:1788880139.072847:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000053567cb1. 00000400:00000010:0.0:1788880139.072847:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000f4a2acaf. 00000400:00000010:0.0:1788880139.072850:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000a01b57e7. 00000400:00000010:0.0:1788880139.072851:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000006509993a. 00000400:00000010:0.0:1788880139.072852:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000093f78903. 00000400:00000010:0.0:1788880139.072853:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000263b0164. 00000400:00000010:0.0:1788880139.072858:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 0000000083e5140b. 00000400:00000010:0.0:1788880139.072860:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000017723e8d. 00000400:00000010:0.0:1788880139.072861:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000002a7a336f. 00000400:00000010:0.0:1788880139.072862:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000001e8226eb. 00000400:00000010:0.0:1788880139.072865:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000446915da. 00000400:00000010:0.0:1788880139.072866:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000ac680e09. 00000400:00000010:0.0:1788880139.072867:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000002f2c66f3. 00000400:00000010:0.0:1788880139.072868:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000640f0578. 00000400:00000010:0.0:1788880139.072872:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000b378643a. 00000400:00000010:0.0:1788880139.072873:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000d45a68a7. 00000400:00000010:0.0:1788880139.072874:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000717fbbc4. 00000400:00000010:0.0:1788880139.072875:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000007e4b1d41. 00000400:00000010:0.0:1788880139.072877:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000ae6b8f5f. 00000400:00000010:0.0:1788880139.072879:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000cf099662. 00000400:00000010:0.0:1788880139.072880:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000003ae341c. 00000400:00000010:0.0:1788880139.072881:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000d60d6f43. 00000400:00000010:0.0:1788880139.072884:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000009c7aaebe. 00000400:00000010:0.0:1788880139.072886:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000006c8f5b06. 00000400:00000010:0.0:1788880139.072887:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000d846f832. 00000400:00000010:0.0:1788880139.072889:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000071440828. 00000400:00000010:0.0:1788880139.072892:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000002c3a69c9. 00000400:00000010:0.0:1788880139.072895:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000351ac169. 00000400:00000010:0.0:1788880139.072897:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000bb8bb2bc. 00000400:00000010:0.0:1788880139.072898:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000090bd2c28. 00000400:00000010:0.0:1788880139.072903:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000009f6d2c0b. 00000400:00000010:0.0:1788880139.072905:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000008e2f9996. 00000400:00000010:0.0:1788880139.072907:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000001f35e338. 00000400:00000010:0.0:1788880139.072909:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000086602065. 00000400:00000010:0.0:1788880139.072912:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000009f8e2ec8. 00000400:00000010:0.0:1788880139.072915:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000795e8097. 00000400:00000010:0.0:1788880139.072916:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000029a39df6. 00000400:00000010:0.0:1788880139.072918:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000003042bb80. 00000400:00000010:0.0:1788880139.072926:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 0000000090ef9546. 00000400:00000010:0.0:1788880139.072930:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000407a5848. 00000400:00000010:0.0:1788880139.072931:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000020c2c3af. 00000400:00000010:0.0:1788880139.072933:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000008231791a. 00000400:00000010:0.0:1788880139.072937:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000005b7e341b. 00000400:00000010:0.0:1788880139.072939:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000034219bb6. 00000400:00000010:0.0:1788880139.072940:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000002cb55033. 00000400:00000010:0.0:1788880139.072941:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000006edfa698. 00000400:00000010:0.0:1788880139.072945:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000eb563c77. 00000400:00000010:0.0:1788880139.072947:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000038baaf2b. 00000400:00000010:0.0:1788880139.072948:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000020217daa. 00000400:00000010:0.0:1788880139.072949:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000ae206ae6. 00000400:00000010:0.0:1788880139.072953:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 0000000010def11f. 00000400:00000010:0.0:1788880139.072955:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000000d9b34be. 00000400:00000010:0.0:1788880139.072957:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000028d5b251. 00000400:00000010:0.0:1788880139.072958:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000020123669. 00000400:00000010:0.0:1788880139.072962:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 0000000091dccc4c. 00000400:00000010:0.0:1788880139.072964:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000aabff115. 00000400:00000010:0.0:1788880139.072966:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000089e7f023. 00000400:00000010:0.0:1788880139.072967:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000001dba33a1. 00000400:00000010:0.0:1788880139.072971:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000e3072048. 00000400:00000010:0.0:1788880139.072974:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000e95e1722. 00000400:00000010:0.0:1788880139.072975:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000007071d79c. 00000400:00000010:0.0:1788880139.072977:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000b2fa20b8. 00000400:00000010:0.0:1788880139.072982:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000430b3db4. 00000400:00000010:0.0:1788880139.072985:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000031968a76. 00000400:00000010:0.0:1788880139.072986:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000f3cf0b66. 00000400:00000010:0.0:1788880139.072988:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000022ce03a7. 00000400:00000010:0.0:1788880139.072992:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000003355cc69. 00000400:00000010:0.0:1788880139.072994:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000b9d8d666. 00000400:00000010:0.0:1788880139.072996:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000004b090f52. 00000400:00000010:0.0:1788880139.072998:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000d70ed490. 00000400:00000010:0.0:1788880139.073004:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000f682e9bf. 00000400:00000010:0.0:1788880139.073006:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000074b8f492. 00000400:00000010:0.0:1788880139.073007:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000005e8cce53. 00000400:00000010:0.0:1788880139.073008:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000001700fc60. 00000400:00000010:0.0:1788880139.073011:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000b47edba3. 00000400:00000010:0.0:1788880139.073012:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000213c3221. 00000400:00000010:0.0:1788880139.073013:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000b5efac35. 00000400:00000010:0.0:1788880139.073014:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000179b329e. 00000400:00000010:0.0:1788880139.073017:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 0000000085153192. 00000400:00000010:0.0:1788880139.073018:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000088c59b15. 00000400:00000010:0.0:1788880139.073019:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000008b765425. 00000400:00000010:0.0:1788880139.073020:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000d76808fa. 00000400:00000010:0.0:1788880139.073023:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 0000000013058806. 00000400:00000010:0.0:1788880139.073024:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000fd8cf531. 00000400:00000010:0.0:1788880139.073025:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000913780f2. 00000400:00000010:0.0:1788880139.073026:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000025d0ef44. 00000400:00000010:0.0:1788880139.073029:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000008322def0. 00000400:00000010:0.0:1788880139.073030:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000d0566d66. 00000400:00000010:0.0:1788880139.073031:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000006463e206. 00000400:00000010:0.0:1788880139.073032:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000004890a794. 00000400:00000010:0.0:1788880139.073034:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000601de2e5. 00000400:00000010:0.0:1788880139.073035:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000cdfc2cfc. 00000400:00000010:0.0:1788880139.073037:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000e0b182d5. 00000400:00000010:0.0:1788880139.073038:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000b486895c. 00000400:00000010:0.0:1788880139.073041:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000d550bb08. 00000400:00000010:0.0:1788880139.073043:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000a499c75d. 00000400:00000010:0.0:1788880139.073045:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000f8031231. 00000400:00000010:0.0:1788880139.073047:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000a65624c7. 00000400:00000010:0.0:1788880139.073050:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000566b297b. 00000400:00000010:0.0:1788880139.073053:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000d66b7ebf. 00000400:00000010:0.0:1788880139.073054:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000032cd4506. 00000400:00000010:0.0:1788880139.073056:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000002ce29258. 00000400:00000010:0.0:1788880139.073064:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000002e562cd8. 00000400:00000010:0.0:1788880139.073067:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000001e0ee0fd. 00000400:00000010:0.0:1788880139.073069:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000eee9b52c. 00000400:00000010:0.0:1788880139.073071:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000511f444c. 00000400:00000010:0.0:1788880139.073075:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000b06b53a8. 00000400:00000010:0.0:1788880139.073077:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000cdbfe8c9. 00000400:00000010:0.0:1788880139.073078:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000d0ebb38e. 00000400:00000010:0.0:1788880139.073080:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000828a6e54. 00000400:00000010:0.0:1788880139.073085:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000888097e1. 00000400:00000010:0.0:1788880139.073087:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000e94deafd. 00000400:00000010:0.0:1788880139.073088:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000e18e63ab. 00000400:00000010:0.0:1788880139.073089:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000625a50bd. 00000400:00000010:0.0:1788880139.073092:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000ac8b2dfe. 00000400:00000010:0.0:1788880139.073094:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000b37a5920. 00000400:00000010:0.0:1788880139.073096:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000233e2a5e. 00000400:00000010:0.0:1788880139.073098:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000003805b75. 00000400:00000010:0.0:1788880139.073101:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 0000000001bfa385. 00000400:00000010:0.0:1788880139.073103:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000ceda0acc. 00000400:00000010:0.0:1788880139.073105:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000009a80ca11. 00000400:00000010:0.0:1788880139.073107:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000403fecd2. 00000400:00000010:0.0:1788880139.073110:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 0000000041c5b64a. 00000400:00000010:0.0:1788880139.073113:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000001c697d64. 00000400:00000010:0.0:1788880139.073115:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000008afebc6. 00000400:00000010:0.0:1788880139.073117:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000e2551db2. 00000400:00000010:0.0:1788880139.073121:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000941c4168. 00000400:00000010:0.0:1788880139.073123:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000061fb51ad. 00000400:00000010:0.0:1788880139.073124:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000feccf67e. 00000400:00000010:0.0:1788880139.073126:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000d6cffab0. 00000400:00000010:0.0:1788880139.073129:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000008f9b062b. 00000400:00000010:0.0:1788880139.073131:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000fc2fb13f. 00000400:00000010:0.0:1788880139.073133:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000001ca9e4a4. 00000400:00000010:0.0:1788880139.073134:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000008274fdb7. 00000400:00000010:0.0:1788880139.073143:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000003f011e63. 00000400:00000010:0.0:1788880139.073146:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000003f2306bd. 00000400:00000010:0.0:1788880139.073148:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000006f7e78b5. 00000400:00000010:0.0:1788880139.073149:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000079ae6a9f. 00000400:00000010:0.0:1788880139.073161:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 0000000046b40e9c. 00000400:00000010:0.0:1788880139.073164:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000c9be9ea3. 00000400:00000010:0.0:1788880139.073166:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000ff99559d. 00000400:00000010:0.0:1788880139.073168:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000005f344ea7. 00000400:00000010:0.0:1788880139.073171:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000e26108d3. 00000400:00000010:0.0:1788880139.073174:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000028453751. 00000400:00000010:0.0:1788880139.073176:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000a969fa72. 00000400:00000010:0.0:1788880139.073178:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000c3d2ae05. 00000400:00000010:0.0:1788880139.073181:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000c7b5c3bc. 00000400:00000010:0.0:1788880139.073184:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000002070ea55. 00000400:00000010:0.0:1788880139.073186:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000e4823457. 00000400:00000010:0.0:1788880139.073188:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000037bbda8f. 00000400:00000010:0.0:1788880139.073191:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000688c4d24. 00000400:00000010:0.0:1788880139.073193:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000f35f41c6. 00000400:00000010:0.0:1788880139.073195:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000066fbbf21. 00000400:00000010:0.0:1788880139.073197:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000db2e7d92. 00000400:00000010:0.0:1788880139.073205:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000001dade410. 00000400:00000010:0.0:1788880139.073208:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000016dda068. 00000400:00000010:0.0:1788880139.073210:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000c07b1ac5. 00000400:00000010:0.0:1788880139.073212:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000031003832. 00000400:00000010:0.0:1788880139.073216:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000009800865f. 00000400:00000010:0.0:1788880139.073218:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000cb18f2d1. 00000400:00000010:0.0:1788880139.073220:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000532f837e. 00000400:00000010:0.0:1788880139.073222:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000021088ffd. 00000400:00000010:0.0:1788880139.073225:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000006beab1fd. 00000400:00000010:0.0:1788880139.073228:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000b8b60165. 00000400:00000010:0.0:1788880139.073230:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000cb7fe7cb. 00000400:00000010:0.0:1788880139.073557:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000000e1bb9bf. 00000400:00000010:0.0:1788880139.073569:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000002abe7f84. 00000400:00000010:0.0:1788880139.073572:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000a6167f53. 00000400:00000010:0.0:1788880139.073575:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000c29dd124. 00000400:00000010:0.0:1788880139.073577:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000028e6e2f6. 00000400:00000010:0.0:1788880139.073586:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000ce8b3d41. 00000400:00000010:0.0:1788880139.073589:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000009d311a8e. 00000400:00000010:0.0:1788880139.073591:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000df267571. 00000400:00000010:0.0:1788880139.073593:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000148a8d09. 00000400:00000010:0.0:1788880139.073596:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000dc42c099. 00000400:00000010:0.0:1788880139.073599:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000ffee0703. 00000400:00000010:0.0:1788880139.073601:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000fb3b08fa. 00000400:00000010:0.0:1788880139.073603:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000a19ec314. 00000400:00000010:0.0:1788880139.073606:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000d54b1d1c. 00000400:00000010:0.0:1788880139.073609:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000be17e776. 00000400:00000010:0.0:1788880139.073611:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000b4c2eee2. 00000400:00000010:0.0:1788880139.073613:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000000635a2e5. 00000400:00000010:0.0:1788880139.073616:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000f59c5bf6. 00000400:00000010:0.0:1788880139.073619:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000046714d75. 00000400:00000010:0.0:1788880139.073621:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000003f09e354. 00000400:00000010:0.0:1788880139.073622:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000007ffb3e3c. 00000400:00000010:0.0:1788880139.073630:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000763ff8de. 00000400:00000010:0.0:1788880139.073634:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000008354904e. 00000400:00000010:0.0:1788880139.073636:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000029b0289a. 00000400:00000010:0.0:1788880139.073638:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000069978aae. 00000400:00000010:0.0:1788880139.073642:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000113ab045. 00000400:00000010:0.0:1788880139.073644:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000000d7ca70f. 00000400:00000010:0.0:1788880139.073646:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000003703a2ca. 00000400:00000010:0.0:1788880139.073648:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000075d6c75a. 00000400:00000010:0.0:1788880139.073651:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000942bc8df. 00000400:00000010:0.0:1788880139.073654:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000004ca6d06. 00000400:00000010:0.0:1788880139.073655:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000068d8f7bc. 00000400:00000010:0.0:1788880139.073657:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000b822d544. 00000400:00000010:0.0:1788880139.073664:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000ffc66979. 00000400:00000010:0.0:1788880139.073667:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000e20e7b2e. 00000400:00000010:0.0:1788880139.073679:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000015576b8b. 00000400:00000010:0.0:1788880139.073682:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000a88e6cbc. 00000400:00000010:0.0:1788880139.073689:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000b8ded1e6. 00000400:00000010:0.0:1788880139.073691:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000cdd9372c. 00000400:00000010:0.0:1788880139.073692:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000fafad1e5. 00000400:00000010:0.0:1788880139.073693:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000bbc299e5. 00000400:00000010:0.0:1788880139.073696:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 000000005951c18c. 00000400:00000010:0.0:1788880139.073697:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 00000000d0edf61a. 00000400:00000010:0.0:1788880139.073698:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000001f65c003. 00000400:00000010:0.0:1788880139.073699:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000094c90660. 00000400:00000010:0.0:1788880139.073702:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000ec3dd81b. 00000400:00000010:0.0:1788880139.073703:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 0000000080c92cdc. 00000400:00000010:0.0:1788880139.073704:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000004cbdfdb2. 00000400:00000010:0.0:1788880139.073704:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000327a6f29. 00000400:00000010:0.0:1788880139.073707:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000c9683557. 00000400:00000010:0.0:1788880139.073708:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000007642772e. 00000400:00000010:0.0:1788880139.073709:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000007afec3bf. 00000400:00000010:0.0:1788880139.073710:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000039df5d64. 00000400:00000010:0.0:1788880139.073715:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000acf8f2ac. 00000400:00000010:0.0:1788880139.073717:0:201269:0:(lib-ptl.c:871:lnet_portals_create()) alloc '(the_lnet.ln_portals[i])': 84 at 000000009bcd18b1. 00000400:00000010:0.0:1788880139.073718:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000078d66526. 00000400:00000010:0.0:1788880139.073719:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000eee38d66. 00000400:00000010:0.0:1788880139.073722:0:201269:0:(lib-ptl.c:815:lnet_ptl_setup()) alloc '(mhash)': 4112 at 00000000a6238866. 00000400:00000010:0.0:1788880139.073723:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000cff3e641. 00000400:00000010:0.0:1788880139.073724:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000001f195862. 00000400:00000010:0.0:1788880139.073726:0:201269:0:(config.c:343:lnet_net_alloc()) alloc '(net)': 176 at 00000000b92a657c. 00000400:00000010:0.0:1788880139.073729:0:201269:0:(config.c:424:lnet_ni_alloc_common()) alloc '(ni)': 552 at 00000000254b4d2d. 00000400:00000010:0.0:1788880139.073730:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000e4a59c4b. 00000400:00000010:0.0:1788880139.073731:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000feb5f338. 00000400:00000010:0.0:1788880139.073732:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000056ffec6d. 00000400:00000010:0.0:1788880139.073733:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000f4c915b1. 00000400:00000010:0.0:1788880139.073738:0:201269:0:(config.c:703:lnet_parse_networks()) alloc '(tokens)': 4 at 00000000298caa4f. 00000400:00000010:0.0:1788880139.073741:0:201269:0:(config.c:343:lnet_net_alloc()) alloc '(net)': 176 at 000000000a1327aa. 00000400:00000010:0.0:1788880139.073743:0:201269:0:(config.c:424:lnet_ni_alloc_common()) alloc '(ni)': 552 at 0000000000f3e49f. 00000400:00000010:0.0:1788880139.073744:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000021b321bd. 00000400:00000010:0.0:1788880139.073745:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 0000000035b18e01. 00000400:00000010:0.0:1788880139.073746:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 0000000027899f81. 00000400:00000010:0.0:1788880139.073747:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000009387d638. 00000400:00000010:0.0:1788880139.073748:0:201269:0:(config.c:915:lnet_parse_networks()) kfreed 'tokens': 4 at 00000000298caa4f. 00000400:00000010:0.0:1788880139.073758:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 00000000f880d9d6. 00000400:00000010:0.0:1788880139.073759:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 128 at 00000000fc4130ce. 00000800:00000200:3.0:1788880139.074786:0:201269:0:(socklnd.c:2153:ksocknal_device_event()) devevent: status=NETDEV_REGISTER, iface=lo ifindex 1 state 0 ns 000000004951b85b 00000800:00000200:3.0:1788880139.074792:0:201269:0:(socklnd.c:2153:ksocknal_device_event()) devevent: status=NETDEV_UP, iface=lo ifindex 1 state 0 ns 000000004951b85b 00000800:00000200:3.0:1788880139.074795:0:201269:0:(socklnd.c:2153:ksocknal_device_event()) devevent: status=NETDEV_REGISTER, iface=ens2 ifindex 2 state 6 ns 000000004951b85b 00000800:00000200:3.0:1788880139.074797:0:201269:0:(socklnd.c:2153:ksocknal_device_event()) devevent: status=NETDEV_UP, iface=ens2 ifindex 2 state 6 ns 000000004951b85b 00000800:00000010:3.0:1788880139.074805:0:201269:0:(socklnd.c:2622:ksocknal_startup()) alloc '(net)': 208 at 00000000d366d8b4. 00000400:00000010:3.0:1788880139.074815:0:201269:0:(config.c:397:lnet_ni_add_interface()) alloc '(ni->ni_interface)': 5 at 00000000b340f700. 00000800:00000200:3.0:1788880139.074820:0:201269:0:(socklnd_modparams.c:295:ksocklnd_lookup_conns_per_peer()) intf ens2 speed -1 00000400:00000200:3.0:1788880139.075369:0:201269:0:(config.c:1657:lnet_get_link_status_locked()) device idx 1 get_link 2 00000400:00000200:3.0:1788880139.075425:0:201269:0:(acceptor.c:209:lnet_acceptor_add_socket()) Added socket on ens2:988 (IP: 192.168.204.11) 00000400:02000000:3.0:1788880139.075435:0:201269:0:(api-ni.c:2766:lnet_startup_lndni()) Added LNI 192.168.204.11@tcp [8/256/0/180] 00000400:00000010:0.0:1788880139.078397:0:201269:0:(api-ni.c:1817:lnet_ping_buffer_alloc()) alloc '(pbuf)': 57 at 00000000489731b7. 00000400:00000010:0.0:1788880139.078409:0:201269:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000083ad62f9. 00000400:00000010:0.0:1788880139.078425:0:201269:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008fd4a0df. 00000400:00000200:0.0:1788880139.078435:0:201269:0:(lib-ptl.c:918:LNetSetLazyPortal()) Setting portal 0 lazy 00000400:00000010:0.0:1788880139.078439:0:201269:0:(api-ni.c:1817:lnet_ping_buffer_alloc()) alloc '(pbuf)': 281 at 0000000072ffd099. 00000400:00000010:0.0:1788880139.078443:0:201269:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000002045d3a4. 00000400:00000010:0.0:1788880139.078445:0:201269:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f34d0caa. 00000400:00000200:0.0:1788880139.078447:0:201269:0:(api-ni.c:2326:lnet_push_target_post()) posted push target 0000000072ffd099 00000400:00000200:0.0:1788880139.078450:0:201269:0:(api-ni.c:2283:lnet_push_target_resize()) nbytes 272 success 00000400:00000010:0.0:1788880139.078453:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000004131ff73. 00000400:00000010:0.0:1788880139.078457:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 000000005dbdfb4c. 00000400:00000010:0.0:1788880139.078459:0:201269:0:(lib-cpt.c:1355:cfs_percpt_alloc()) alloc '(arr)': 24 at 000000003fcf7330. 00000400:00000010:0.0:1788880139.078461:0:201269:0:(lib-cpt.c:1365:cfs_percpt_alloc()) alloc '(arr->va_ptrs[i])': 64 at 00000000525ff878. 00000400:00000200:3.0:1788880139.078817:0:201269:0:(peer.c:4372:lnet_peer_discovery_start()) discovery start: 0 00000100:00000001:3.0:1788880139.078934:0:201269:0:(ptlrpcd.c:908:ptlrpcd_addref()) Process entered 00000100:00000001:3.0:1788880139.078936:0:201269:0:(ptlrpcd.c:695:ptlrpcd_init()) Process entered 00000100:00000010:3.0:1788880139.078939:0:201269:0:(ptlrpcd.c:749:ptlrpcd_init()) kmalloced '(ptlrpcds)': 8 at 0000000069aeb3d1. 00000100:00000001:3.0:1788880139.078942:0:201269:0:(ptlrpcd.c:483:ptlrpcd_ctl_init()) Process entered 00000100:00000001:3.0:1788880139.078943:0:201269:0:(ptlrpcd.c:500:ptlrpcd_ctl_init()) Process leaving 00000100:00000001:3.0:1788880139.078944:0:201269:0:(ptlrpcd.c:559:ptlrpcd_start()) Process entered 00000400:00000200:2.0:1788880139.078963:0:201283:0:(peer.c:4198:lnet_peer_discovery()) started 00000100:00000001:2.0:1788880139.079161:0:201284:0:(ptlrpcd.c:374:ptlrpcd()) Process entered 00000100:00000001:2.0:1788880139.079168:0:201284:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:2.0:1788880139.079172:0:201284:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 000000001056b5c6. 00000100:00000001:2.0:1788880139.079175:0:201284:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250043001472 : -126823666550144 : ffff8ca79067fa80) 00000020:00000010:2.0:1788880139.079180:0:201284:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000004a1300a5. 00000020:00000010:2.0:1788880139.079185:0:201284:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000816e2c99. 00000020:00000010:2.0:1788880139.079189:0:201284:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 0000000081a618c6. 00000040:00000010:2.0:1788880139.079193:0:201284:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 0000000064e009e6. 00000020:00000010:2.0:1788880139.079197:0:201284:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000fa5c180e. 00000020:00000010:2.0:1788880139.079200:0:201284:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000213004a7. 00000100:00000001:2.0:1788880139.079209:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880139.079211:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880139.079213:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880139.079215:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880139.079254:0:201269:0:(ptlrpcd.c:579:ptlrpcd_start()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880139.079261:0:201269:0:(ptlrpcd.c:846:ptlrpcd_init()) kmalloced '(pd)': 760 at 000000005e771b39. 00000100:00000001:3.0:1788880139.079268:0:201269:0:(ptlrpcd.c:483:ptlrpcd_ctl_init()) Process entered 00000100:00000001:3.0:1788880139.079270:0:201269:0:(ptlrpcd.c:500:ptlrpcd_ctl_init()) Process leaving 00000100:00000001:3.0:1788880139.079272:0:201269:0:(ptlrpcd.c:528:ptlrpcd_partners()) Process entered 00000100:00000010:3.0:1788880139.079274:0:201269:0:(ptlrpcd.c:538:ptlrpcd_partners()) kmalloced '(pc->pc_partners)': 8 at 0000000059d4cdff. 00000100:00000001:3.0:1788880139.079277:0:201269:0:(ptlrpcd.c:551:ptlrpcd_partners()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880139.079278:0:201269:0:(ptlrpcd.c:483:ptlrpcd_ctl_init()) Process entered 00000100:00000001:3.0:1788880139.079280:0:201269:0:(ptlrpcd.c:500:ptlrpcd_ctl_init()) Process leaving 00000100:00000001:3.0:1788880139.079281:0:201269:0:(ptlrpcd.c:528:ptlrpcd_partners()) Process entered 00000100:00000010:3.0:1788880139.079282:0:201269:0:(ptlrpcd.c:538:ptlrpcd_partners()) kmalloced '(pc->pc_partners)': 8 at 000000009b73e834. 00000100:00000001:3.0:1788880139.079285:0:201269:0:(ptlrpcd.c:551:ptlrpcd_partners()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880139.079286:0:201269:0:(ptlrpcd.c:483:ptlrpcd_ctl_init()) Process entered 00000100:00000001:3.0:1788880139.079287:0:201269:0:(ptlrpcd.c:500:ptlrpcd_ctl_init()) Process leaving 00000100:00000001:3.0:1788880139.079289:0:201269:0:(ptlrpcd.c:528:ptlrpcd_partners()) Process entered 00000100:00000010:3.0:1788880139.079290:0:201269:0:(ptlrpcd.c:538:ptlrpcd_partners()) kmalloced '(pc->pc_partners)': 8 at 0000000004ff55c9. 00000100:00000001:3.0:1788880139.079292:0:201269:0:(ptlrpcd.c:551:ptlrpcd_partners()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880139.079293:0:201269:0:(ptlrpcd.c:483:ptlrpcd_ctl_init()) Process entered 00000100:00000001:3.0:1788880139.079295:0:201269:0:(ptlrpcd.c:500:ptlrpcd_ctl_init()) Process leaving 00000100:00000001:3.0:1788880139.079296:0:201269:0:(ptlrpcd.c:528:ptlrpcd_partners()) Process entered 00000100:00000010:3.0:1788880139.079297:0:201269:0:(ptlrpcd.c:538:ptlrpcd_partners()) kmalloced '(pc->pc_partners)': 8 at 000000000952a57a. 00000100:00000001:3.0:1788880139.079298:0:201269:0:(ptlrpcd.c:551:ptlrpcd_partners()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880139.079300:0:201269:0:(ptlrpcd.c:559:ptlrpcd_start()) Process entered 00000100:00000001:3.0:1788880139.079421:0:201285:0:(ptlrpcd.c:374:ptlrpcd()) Process entered 00000100:00000001:3.0:1788880139.079426:0:201285:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:3.0:1788880139.079429:0:201285:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 000000003f91b00c. 00000100:00000001:3.0:1788880139.079431:0:201285:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250280029760 : -126823429521856 : ffff8ca79e88be40) 00000020:00000010:3.0:1788880139.079435:0:201285:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000064581423. 00000020:00000010:3.0:1788880139.079437:0:201285:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000001be7003e. 00000020:00000010:3.0:1788880139.079440:0:201285:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 000000008136bdfa. 00000040:00000010:3.0:1788880139.079442:0:201285:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000c0e0d685. 00000020:00000010:3.0:1788880139.079445:0:201285:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000f88278a5. 00000020:00000010:3.0:1788880139.079448:0:201285:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000b2d51300. 00000100:00000001:3.0:1788880139.079505:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880139.079507:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880139.079509:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880139.079510:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.079536:0:201269:0:(ptlrpcd.c:579:ptlrpcd_start()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.079539:0:201269:0:(ptlrpcd.c:559:ptlrpcd_start()) Process entered 00000100:00000001:3.0:1788880139.079709:0:201286:0:(ptlrpcd.c:374:ptlrpcd()) Process entered 00000100:00000001:3.0:1788880139.079713:0:201286:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:3.0:1788880139.079716:0:201286:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000addf16a4. 00000100:00000001:3.0:1788880139.079722:0:201286:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250280027648 : -126823429523968 : ffff8ca79e88b600) 00000020:00000010:3.0:1788880139.079725:0:201286:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000ecee2262. 00000020:00000010:3.0:1788880139.079727:0:201286:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000a264af9a. 00000020:00000010:3.0:1788880139.079729:0:201286:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 000000000309ce11. 00000040:00000010:3.0:1788880139.079731:0:201286:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 000000007b207697. 00000020:00000010:3.0:1788880139.079734:0:201286:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000b5b1f0c9. 00000020:00000010:3.0:1788880139.079737:0:201286:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000007a2e71b8. 00000100:00000001:3.0:1788880139.079743:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880139.079745:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880139.079747:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880139.079749:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.079755:0:201269:0:(ptlrpcd.c:579:ptlrpcd_start()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.079759:0:201269:0:(ptlrpcd.c:559:ptlrpcd_start()) Process entered 00000100:00000001:3.0:1788880139.080153:0:201287:0:(ptlrpcd.c:374:ptlrpcd()) Process entered 00000100:00000001:3.0:1788880139.080158:0:201287:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:3.0:1788880139.080161:0:201287:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000087a6f6bb. 00000100:00000001:3.0:1788880139.080163:0:201287:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250280029376 : -126823429522240 : ffff8ca79e88bcc0) 00000020:00000010:3.0:1788880139.080166:0:201287:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000002db67478. 00000020:00000010:3.0:1788880139.080169:0:201287:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000c9d75875. 00000020:00000010:3.0:1788880139.080172:0:201287:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000cd0e1822. 00000040:00000010:3.0:1788880139.080174:0:201287:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000d18d3730. 00000020:00000010:3.0:1788880139.080177:0:201287:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 0000000079185e34. 00000020:00000010:3.0:1788880139.080179:0:201287:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000da380b07. 00000100:00000001:3.0:1788880139.080215:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880139.080216:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880139.080218:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880139.080219:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.080316:0:201269:0:(ptlrpcd.c:579:ptlrpcd_start()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.080319:0:201269:0:(ptlrpcd.c:559:ptlrpcd_start()) Process entered 00000100:00000001:3.0:1788880139.080523:0:201288:0:(ptlrpcd.c:374:ptlrpcd()) Process entered 00000100:00000001:3.0:1788880139.080529:0:201288:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:3.0:1788880139.080532:0:201288:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000e9e35f02. 00000100:00000001:3.0:1788880139.080534:0:201288:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250280028224 : -126823429523392 : ffff8ca79e88b840) 00000020:00000010:3.0:1788880139.080538:0:201288:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000fb7c0d85. 00000020:00000010:3.0:1788880139.080541:0:201288:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000002d70c0e5. 00000020:00000010:3.0:1788880139.080543:0:201288:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000356181d6. 00000040:00000010:3.0:1788880139.080546:0:201288:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000b9faaae6. 00000020:00000010:3.0:1788880139.080549:0:201288:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 0000000096f5492c. 00000020:00000010:3.0:1788880139.080551:0:201288:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000002b0aae7f. 00000100:00000001:3.0:1788880139.080566:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880139.080567:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880139.080569:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880139.080571:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.080584:0:201269:0:(ptlrpcd.c:579:ptlrpcd_start()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.080589:0:201269:0:(ptlrpcd.c:901:ptlrpcd_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.080590:0:201269:0:(ptlrpcd.c:917:ptlrpcd_addref()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880139.080621:0:201269:0:(connection.c:186:ptlrpc_connection_init()) kmalloced '(cpus_latency_qos)': 1088 at 0000000037056c79. 00000100:00000040:2.0:1788880139.080715:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 25 (11127) 02000000:08000000:0.0:1788880139.080755:0:201269:0:(sec.c:69:sptlrpc_register_policy()) sec.null: registered 02000000:08000000:0.0:1788880139.080759:0:201269:0:(sec.c:69:sptlrpc_register_policy()) plain: registered 00000100:00000001:0.0:1788880139.080796:0:201269:0:(nrs.c:1742:ptlrpc_nrs_init()) Process entered 00000100:00000001:0.0:1788880139.080803:0:201269:0:(nrs.c:1188:ptlrpc_nrs_policy_register()) Process entered 00000100:00000001:0.0:1788880139.080804:0:201269:0:(nrs.c:1095:nrs_policy_find_desc_locked()) Process entered 00000100:00000001:0.0:1788880139.080806:0:201269:0:(nrs.c:1101:nrs_policy_find_desc_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880139.080808:0:201269:0:(nrs.c:1230:ptlrpc_nrs_policy_register()) kmalloced '(desc)': 72 at 00000000561053cf. 00000100:00000001:0.0:1788880139.080811:0:201269:0:(nrs.c:1325:ptlrpc_nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.080812:0:201269:0:(nrs.c:1188:ptlrpc_nrs_policy_register()) Process entered 00000100:00000001:0.0:1788880139.080813:0:201269:0:(nrs.c:1095:nrs_policy_find_desc_locked()) Process entered 00000100:00000001:0.0:1788880139.080814:0:201269:0:(nrs.c:1101:nrs_policy_find_desc_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880139.080817:0:201269:0:(nrs.c:1230:ptlrpc_nrs_policy_register()) kmalloced '(desc)': 72 at 00000000666c59c9. 00000100:00000001:0.0:1788880139.080818:0:201269:0:(nrs.c:1325:ptlrpc_nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.080820:0:201269:0:(nrs.c:1188:ptlrpc_nrs_policy_register()) Process entered 00000100:00000001:0.0:1788880139.080821:0:201269:0:(nrs.c:1095:nrs_policy_find_desc_locked()) Process entered 00000100:00000001:0.0:1788880139.080822:0:201269:0:(nrs.c:1101:nrs_policy_find_desc_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880139.080824:0:201269:0:(nrs.c:1230:ptlrpc_nrs_policy_register()) kmalloced '(desc)': 72 at 00000000ca09f0dc. 00000100:00000001:0.0:1788880139.080825:0:201269:0:(nrs.c:1325:ptlrpc_nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.080827:0:201269:0:(nrs.c:1188:ptlrpc_nrs_policy_register()) Process entered 00000100:00000001:0.0:1788880139.080828:0:201269:0:(nrs.c:1095:nrs_policy_find_desc_locked()) Process entered 00000100:00000001:0.0:1788880139.080829:0:201269:0:(nrs.c:1101:nrs_policy_find_desc_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880139.080830:0:201269:0:(nrs.c:1230:ptlrpc_nrs_policy_register()) kmalloced '(desc)': 72 at 0000000073a73d35. 00000100:00000001:0.0:1788880139.080832:0:201269:0:(nrs.c:1325:ptlrpc_nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.080833:0:201269:0:(nrs.c:1188:ptlrpc_nrs_policy_register()) Process entered 00000100:00000001:0.0:1788880139.080835:0:201269:0:(nrs.c:1095:nrs_policy_find_desc_locked()) Process entered 00000100:00000001:0.0:1788880139.080836:0:201269:0:(nrs.c:1101:nrs_policy_find_desc_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880139.080837:0:201269:0:(nrs.c:1230:ptlrpc_nrs_policy_register()) kmalloced '(desc)': 72 at 0000000089637602. 00000100:00000001:0.0:1788880139.080838:0:201269:0:(nrs.c:1325:ptlrpc_nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.080839:0:201269:0:(nrs.c:1188:ptlrpc_nrs_policy_register()) Process entered 00000100:00000001:0.0:1788880139.080840:0:201269:0:(nrs.c:1095:nrs_policy_find_desc_locked()) Process entered 00000100:00000001:0.0:1788880139.080841:0:201269:0:(nrs.c:1101:nrs_policy_find_desc_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880139.080843:0:201269:0:(nrs.c:1230:ptlrpc_nrs_policy_register()) kmalloced '(desc)': 72 at 0000000071506404. 00000100:00000001:0.0:1788880139.080844:0:201269:0:(nrs.c:1325:ptlrpc_nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880139.080846:0:201269:0:(nrs.c:1772:ptlrpc_nrs_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880139.080849:0:201269:0:(tgt_main.c:816:tgt_mod_init()) Process entered 00000020:00000001:0.0:1788880139.080995:0:201269:0:(tgt_main.c:841:tgt_mod_init()) Process leaving (rc=0 : 0 : 0) 00000001:00000010:0.0:1788880139.081015:0:201269:0:(nodemap_handler.c:4907:nodemap_config_alloc()) kmalloced '(config)': 488 at 00000000488a4292. 00000001:00000001:0.0:1788880139.081018:0:201269:0:(hash.c:1038:cfs_hash_create()) Process entered 00000001:00000010:0.0:1788880139.081021:0:201269:0:(hash.c:1065:cfs_hash_create()) alloc '(hs)': 232 at 0000000006bf2d16. 00000001:00000010:0.0:1788880139.081024:0:201269:0:(hash.c:933:cfs_hash_buckets_realloc()) alloc '(new_bkts)': 8 at 000000005ba5ed1a. 00000001:00000010:0.0:1788880139.081030:0:201269:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 152 at 00000000426b68f7. 00000001:00000001:0.0:1788880139.081037:0:201269:0:(nodemap_handler.c:3693:nodemap_create()) Process entered 00000001:00000010:0.0:1788880139.081041:0:201269:0:(nodemap_handler.c:3737:nodemap_create()) kmalloced '(nodemap)': 968 at 000000008ee7d9db. 00000001:00000040:0.0:1788880139.081047:0:201269:0:(nodemap_handler.c:99:nodemap_getref()) GETting nodemap default(p=000000008ee7d9db) : new refcount 3 00000001:00000040:0.0:1788880139.081075:0:201269:0:(nodemap_handler.c:99:nodemap_getref()) GETting nodemap default(p=000000008ee7d9db) : new refcount 4 00000001:00000001:0.0:1788880139.081078:0:201269:0:(nodemap_handler.c:3818:nodemap_create()) Process leaving (rc=18446617250086274048 : -126823623277568 : ffff8ca792fc4400) 00000001:00000001:0.0:1788880139.081080:0:201269:0:(nodemap_handler.c:5014:nodemap_config_set_active()) Process entered 00000001:00000001:0.0:1788880139.081083:0:201269:0:(hash.c:1462:cfs_hash_for_each_tight()) Process entered 00000001:00000001:0.0:1788880139.081086:0:201269:0:(hash.c:1502:cfs_hash_for_each_tight()) Process leaving (rc=1 : 1 : 1) 00000001:00000010:0.0:1788880139.081089:0:201269:0:(nodemap_lproc.c:1458:lprocfs_nodemap_register()) kmalloced '(nm_entry)': 48 at 00000000e2a4a519. 00000001:00000001:0.0:1788880139.081184:0:201269:0:(nodemap_handler.c:5074:nodemap_config_set_active()) Process leaving 00000001:00000040:0.0:1788880139.081186:0:201269:0:(nodemap_handler.c:115:nodemap_putref()) PUTting nodemap default(p=000000008ee7d9db) : new refcount 3 00000100:00000001:0.0:1788880139.081189:0:201269:0:(ptlrpc_module.c:93:ptlrpc_init()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880139.149586:0:201300:0:(upcall_cache.c:571:upcall_cache_init()) Process entered 02000000:00000010:0.0:1788880139.149596:0:201300:0:(upcall_cache.c:573:upcall_cache_init()) kmalloced '(cache)': 1208 at 000000006e5ba4b8. 02000000:00000010:0.0:1788880139.149605:0:201300:0:(upcall_cache.c:581:upcall_cache_init()) kmalloced '(cache->uc_hashtable)': 1024 at 00000000b13908ec. 02000000:00000001:0.0:1788880139.149609:0:201300:0:(upcall_cache.c:594:upcall_cache_init()) Process leaving (rc=18446617250071750656 : -126823637800960 : ffff8ca7921ea800) 02000000:00000001:0.0:1788880139.149612:0:201300:0:(upcall_cache.c:571:upcall_cache_init()) Process entered 02000000:00000010:0.0:1788880139.149615:0:201300:0:(upcall_cache.c:573:upcall_cache_init()) kmalloced '(cache)': 1208 at 00000000cf9e59ee. 02000000:00000010:0.0:1788880139.149633:0:201300:0:(upcall_cache.c:581:upcall_cache_init()) kmalloced '(cache->uc_hashtable)': 16384 at 0000000006425cb1. 02000000:00000001:0.0:1788880139.149639:0:201300:0:(upcall_cache.c:594:upcall_cache_init()) Process leaving (rc=18446617250071740416 : -126823637811200 : ffff8ca7921e8000) 02000000:08000000:0.0:1788880140.686369:0:201300:0:(gss_svc_upcall.c:1081:check_gssd_socket()) Failed to connect to socket: -22 02000000:08000000:0.0:1788880140.686426:0:201300:0:(gss_svc_upcall.c:1128:gss_init_svc_upcall()) Init channel not opened by lsvcgssd, GSS might not work on server side until daemon is active 02000000:08000000:0.0:1788880140.686430:0:201300:0:(gss_mech_switch.c:48:lgss_mech_register()) register gssnull mechanism 02000000:08000000:0.0:1788880140.686432:0:201300:0:(gss_mech_switch.c:48:lgss_mech_register()) register krb5 mechanism 02000000:08000000:0.0:1788880140.686434:0:201300:0:(gss_mech_switch.c:48:lgss_mech_register()) register sk mechanism 02000000:08000000:0.0:1788880140.688148:0:201300:0:(sec.c:69:sptlrpc_register_policy()) gss.keyring: registered 00000008:00000001:0.0:1788880140.986516:0:201356:0:(osc_request.c:4225:osc_init()) Process entered 00000008:00000040:0.0:1788880140.986522:0:201356:0:(osc_request.c:4230:osc_init()) Lustre OSC module (000000007d1b7d67). 00000001:00000010:0.0:1788880140.986856:0:201356:0:(shrinker_debug.c:228:ll_shrinker_alloc()) alloc '(s)': 88 at 000000001068eb10. 00000100:00000010:0.0:1788880140.986910:0:201356:0:(client.c:690:ptlrpc_init_rq_pool()) kmalloced '(pool)': 56 at 000000003e44ce8f. 00000020:00000001:0.0:1788880140.986927:0:201356:0:(genops.c:221:class_register_type()) Process entered 00000020:00000010:0.0:1788880140.986959:0:201356:0:(genops.c:236:class_register_type()) kmalloced '(type)': 144 at 000000002d93b3ff. 00000020:00000001:0.0:1788880140.986977:0:201356:0:(genops.c:285:class_register_type()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:0.0:1788880140.986982:0:201356:0:(lprocfs_status.c:118:ldebugfs_add_symlink()) kmalloced '(dest)': 129 at 000000004fefb4f7. 00000020:00000010:0.0:1788880140.986991:0:201356:0:(lprocfs_status.c:128:ldebugfs_add_symlink()) kfreed 'dest': 129 at 000000004fefb4f7. 00000008:00000001:0.0:1788880140.986996:0:201356:0:(osc_request.c:4287:osc_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:0.0:1788880141.058971:0:201369:0:(lov_obd.c:1400:lov_init()) Process entered 00020000:00000040:0.0:1788880141.058977:0:201369:0:(lov_obd.c:1405:lov_init()) Lustre LOV module (000000003e395ac7). 00000020:00000001:0.0:1788880141.059150:0:201369:0:(genops.c:221:class_register_type()) Process entered 00000020:00000010:0.0:1788880141.059156:0:201369:0:(genops.c:236:class_register_type()) kmalloced '(type)': 144 at 000000004fefb4f7. 00000020:00000001:0.0:1788880141.059183:0:201369:0:(genops.c:285:class_register_type()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:0.0:1788880141.059186:0:201369:0:(lov_obd.c:1430:lov_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880141.155145:0:201382:0:(genops.c:221:class_register_type()) Process entered 00000020:00000010:2.0:1788880141.155154:0:201382:0:(genops.c:236:class_register_type()) kmalloced '(type)': 144 at 00000000adebcc59. 00000020:00000001:2.0:1788880141.155197:0:201382:0:(genops.c:285:class_register_type()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880141.235395:0:201396:0:(genops.c:221:class_register_type()) Process entered 00000020:00000010:2.0:1788880141.235404:0:201396:0:(genops.c:236:class_register_type()) kmalloced '(type)': 144 at 00000000f177c200. 00000020:00000001:2.0:1788880141.235437:0:201396:0:(genops.c:285:class_register_type()) Process leaving (rc=0 : 0 : 0) 00040000:00000001:0.0:1788880141.465760:0:201432:0:(lquota_lib.c:571:lquota_init()) Process entered 00040000:00000001:0.0:1788880141.465891:0:201432:0:(qmt_dev.c:572:qmt_glb_init()) Process entered 00000020:00000001:0.0:1788880141.465895:0:201432:0:(genops.c:221:class_register_type()) Process entered 00000020:00000010:0.0:1788880141.465901:0:201432:0:(genops.c:236:class_register_type()) kmalloced '(type)': 144 at 00000000d5b680c7. 00000020:00000001:0.0:1788880141.465934:0:201432:0:(genops.c:285:class_register_type()) Process leaving (rc=0 : 0 : 0) 00040000:00000001:0.0:1788880141.465936:0:201432:0:(qmt_dev.c:576:qmt_glb_init()) Process leaving (rc=0 : 0 : 0) 00040000:00000001:0.0:1788880141.465951:0:201432:0:(lquota_lib.c:592:lquota_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880142.743133:0:201779:0:(genops.c:221:class_register_type()) Process entered 00000020:00000010:3.0:1788880142.743143:0:201779:0:(genops.c:236:class_register_type()) kmalloced '(type)': 144 at 00000000dafe1e6a. 00000020:00000001:3.0:1788880142.743172:0:201779:0:(genops.c:285:class_register_type()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880142.876733:0:201805:0:(genops.c:221:class_register_type()) Process entered 00000020:00000010:3.0:1788880142.876744:0:201805:0:(genops.c:236:class_register_type()) kmalloced '(type)': 144 at 00000000aa5cb4a4. 00000020:00000001:3.0:1788880142.876769:0:201805:0:(genops.c:285:class_register_type()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880142.876771:0:201805:0:(genops.c:221:class_register_type()) Process entered 00000020:00000010:3.0:1788880142.876774:0:201805:0:(genops.c:236:class_register_type()) kmalloced '(type)': 144 at 00000000c63cc8d0. 00000020:00000001:3.0:1788880142.876800:0:201805:0:(genops.c:285:class_register_type()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880142.976882:0:201826:0:(genops.c:221:class_register_type()) Process entered 00000020:00000010:2.0:1788880142.976892:0:201826:0:(genops.c:236:class_register_type()) kmalloced '(type)': 144 at 00000000fcfebbc0. 00000020:00000001:2.0:1788880142.976923:0:201826:0:(genops.c:285:class_register_type()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880143.114097:0:201854:0:(genops.c:221:class_register_type()) Process entered 00000020:00000010:0.0:1788880143.114107:0:201854:0:(genops.c:236:class_register_type()) kmalloced '(type)': 144 at 000000002e0b3a9e. 00000020:00000001:0.0:1788880143.114131:0:201854:0:(genops.c:285:class_register_type()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880143.114133:0:201854:0:(genops.c:221:class_register_type()) Process entered 00000020:00000010:0.0:1788880143.114136:0:201854:0:(genops.c:236:class_register_type()) kmalloced '(type)': 144 at 0000000070e7d562. 00000020:00000001:0.0:1788880143.114150:0:201854:0:(genops.c:285:class_register_type()) Process leaving (rc=0 : 0 : 0) 00000080:00000040:0.0:1788880143.181759:0:201867:0:(super25.c:291:lustre_init()) Lustre client module (00000000e638bcfe). 00000020:00000010:0.0:1788880143.182126:0:201867:0:(cl_object.c:715:cl_env_new()) slab-alloced 'cle': 144 at 00000000c015a5a2. 00000020:00000010:0.0:1788880143.182132:0:201867:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000dfb64915. 00000020:00000010:0.0:1788880143.182136:0:201867:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000003dd8547f. 00000020:00000010:0.0:1788880143.182138:0:201867:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 000000000caf68b9. 00000008:00000010:0.0:1788880143.182150:0:201867:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 0000000041d91a45. 00020000:00000010:0.0:1788880143.182156:0:201867:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 0000000095bd441d. 00008000:00000010:0.0:1788880143.182166:0:201867:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 000000007c78b0bc. 00000080:00000010:0.0:1788880143.182176:0:201867:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 000000006bbb3bab. 00000080:00000010:0.0:1788880143.182186:0:201867:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 00000000e1c6a789. 00000020:00000010:0.0:1788880143.182190:0:201867:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000023cb1ad2. 00000008:00000010:0.0:1788880143.182200:0:201867:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000c9ae7912. 00020000:00000010:0.0:1788880143.182214:0:201867:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 00000000f0501fbe. 00008000:00000010:0.0:1788880143.182296:0:201867:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 000000001ae7abc6. 00000080:00000010:0.0:1788880143.182309:0:201867:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 00000000b1f6b775. 00000020:00001000:0.0:1788880143.182313:0:201867:0:(cl_object.c:856:cl_env_alloc()) 1@00000000c015a5a2 00000080:00000001:0.0:1788880143.182336:0:201867:0:(super25.c:344:lustre_init()) Process leaving (rc=0 : 0 : 0) 00000001:00000001:3.0:1788880163.026660:0:202626:0:(debug.c:653:libcfs_debug_mark_buffer()) ************************************************** 00000001:02000400:3.0:1788880163.026666:0:202626:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg411-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 00000001:00000001:3.0:1788880163.030639:0:202626:0:(debug.c:656:libcfs_debug_mark_buffer()) ************************************************** 00000100:00000040:2.0:1788880164.479300:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 25 (11153) 00000001:00000001:2.0:1788880165.138962:0:202900:0:(debug.c:653:libcfs_debug_mark_buffer()) ************************************************** 00000001:02000400:2.0:1788880165.138969:0:202900:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg411-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 00000001:00000001:2.0:1788880165.143804:0:202900:0:(debug.c:656:libcfs_debug_mark_buffer()) ************************************************** 00000001:00000001:3.0:1788880172.600700:0:203364:0:(debug.c:653:libcfs_debug_mark_buffer()) ************************************************** 00000001:02000400:3.0:1788880172.600705:0:203364:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg411-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 00000001:00000001:3.0:1788880172.605064:0:203364:0:(debug.c:656:libcfs_debug_mark_buffer()) ************************************************** 00000080:00000010:0.0:1788880172.725264:0:203392:0:(super25.c:245:lustre_init_fs_context()) kmalloced '(lmd)': 128 at 00000000d9002745. 00000020:00000001:0.0:1788880172.725277:0:203392:0:(obd_mount.c:1352:lustre_parse_monolithic()) Process entered 00000020:00000010:0.0:1788880172.725283:0:203392:0:(obd_mount.c:1374:lustre_parse_monolithic()) kmalloced '(lmd->lmd_params)': 4096 at 00000000ff27c579. 00000020:00000010:0.0:1788880172.725292:0:203392:0:(obd_mount.c:1576:lustre_parse_monolithic()) kmalloced '(lmd->lmd_dev)': 28 at 0000000078df11ee. 00000020:00000010:0.0:1788880172.725295:0:203392:0:(obd_mount.c:1632:lustre_parse_monolithic()) kmalloced '(lmd->lmd_profile)': 14 at 000000006dc748d7. 00000020:00000010:0.0:1788880172.725298:0:203392:0:(obd_mount.c:1660:lustre_parse_monolithic()) kmalloced '(lmd->lmd_opts)': 17 at 0000000015de8d84. 00000020:01000004:0.0:1788880172.725300:0:203392:0:(obd_mount.c:923:lmd_print()) mount data: 00000020:01000004:0.0:1788880172.725301:0:203392:0:(obd_mount.c:925:lmd_print()) profile: lustre-client 00000020:01000004:0.0:1788880172.725302:0:203392:0:(obd_mount.c:926:lmd_print()) device: 192.168.204.111@tcp:/lustre 00000020:01000004:0.0:1788880172.725303:0:203392:0:(obd_mount.c:929:lmd_print()) options: user_xattr,flock 00000020:00000001:0.0:1788880172.725305:0:203392:0:(obd_mount.c:1672:lustre_parse_monolithic()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:0.0:1788880172.725374:0:203392:0:(super25.c:129:lustre_fill_super()) Process entered 00000080:01200004:0.0:1788880172.725375:0:203392:0:(super25.c:130:lustre_fill_super()) VFS Op: sb 0000000008cb040b 00000020:00000001:0.0:1788880172.725377:0:203392:0:(obd_mount.c:638:lustre_init_lsi()) Process entered 00000020:00000010:0.0:1788880172.725378:0:203392:0:(obd_mount.c:642:lustre_init_lsi()) kmalloced '(lsi)': 1880 at 0000000059c37dc0. 00000020:00000001:0.0:1788880172.725380:0:203392:0:(obd_mount.c:659:lustre_init_lsi()) Process leaving (rc=18446617249890584576 : -126823818967040 : ffff8ca787524800) 00000080:01000004:0.0:1788880172.725384:0:203392:0:(super25.c:161:lustre_fill_super()) Mounting client lustre-client 00000020:00000001:0.0:1788880172.725385:0:203392:0:(obd_mount.c:289:lustre_start_mgc()) Process entered 00000020:00000010:0.0:1788880172.725394:0:203392:0:(obd_mount.c:332:lustre_start_mgc()) kmalloced '(mgcname)': 23 at 0000000093570a07. 00000020:00000010:0.0:1788880172.725396:0:203392:0:(obd_mount.c:340:lustre_start_mgc()) kmalloced '(data)': 192 at 00000000698cd534. 00000020:01000004:0.0:1788880172.725398:0:203392:0:(obd_mount.c:411:lustre_start_mgc()) Start MGC 'MGC192.168.204.111@tcp' 00000100:00000010:0.0:1788880172.725402:0:203392:0:(lustre_peer.c:90:class_add_uuid()) kmalloced '(data)': 704 at 0000000030946e28. 00000100:00000040:0.0:1788880172.725404:0:203392:0:(lustre_peer.c:134:class_add_uuid()) add uuid 192.168.204.111@tcp 192.168.204.111@tcp 00000400:00000200:0.0:1788880172.725406:0:203392:0:(peer.c:1363:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:0.0:1788880172.725450:0:203392:0:(api-ni.c:3274:LNetNIInit()) refs 1 00000400:00000200:0.0:1788880172.725453:0:203392:0:(peer.c:1377:LNetAddPeer()) Adding NID[0]: 192.168.204.111@tcp 00000400:00000010:0.0:1788880172.725461:0:203392:0:(peer.c:230:lnet_peer_alloc()) alloc '(lp)': 360 at 00000000b0c52047. 00000400:00000200:0.0:1788880172.725465:0:203392:0:(peer.c:266:lnet_peer_alloc()) 00000000b0c52047 nid 192.168.204.111@tcp 00000400:00000010:0.0:1788880172.725468:0:203392:0:(peer.c:194:lnet_peer_net_alloc()) alloc '(lpn)': 72 at 00000000e6d52066. 00000400:00000200:0.0:1788880172.725470:0:203392:0:(peer.c:203:lnet_peer_net_alloc()) 00000000e6d52066 net tcp 00000400:00000010:0.0:1788880172.725478:0:203392:0:(peer.c:138:lnet_peer_ni_alloc()) alloc '(lpni)': 480 at 00000000450a9ac6. 00000400:00000200:0.0:1788880172.725482:0:203392:0:(peer.c:184:lnet_peer_ni_alloc()) 00000000450a9ac6 nid 192.168.204.111@tcp 00000400:00000200:0.0:1788880172.725487:0:203392:0:(peer.c:1111:lnet_peer_ni_clr_non_mr_pref_nid()) peer 192.168.204.111@tcp: -2 00000400:00000200:0.0:1788880172.725490:0:203392:0:(peer.c:1719:lnet_peer_attach_peer_ni()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp flags 0x100001 00000400:00000200:0.0:1788880172.725507:0:203392:0:(peer.c:2528:lnet_peer_queue_for_discovery()) Queue peer 192.168.204.111@tcp: 0 00000400:00000200:0.0:1788880172.725509:0:203392:0:(peer.c:2850:lnet_discover_peer_locked()) Discovery attempt # 1 00000400:00000200:0.0:1788880172.725511:0:203392:0:(peer.c:2890:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:0.0:1788880172.725513:0:203392:0:(peer.c:2899:lnet_discover_peer_locked()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp: 0. pending discovery 00000020:00000010:0.0:1788880172.725517:0:203392:0:(obd_mount.c:473:lustre_start_mgc()) kmalloced '(uuid)': 40 at 0000000019f65800. 00000020:00000001:0.0:1788880172.725522:0:203392:0:(obd_mount.c:228:lustre_start_simple()) Process entered 00000020:01000004:0.0:1788880172.725524:0:203392:0:(obd_mount.c:230:lustre_start_simple()) Starting OBD MGC192.168.204.111@tcp (typ=mgc) 00000020:00000001:0.0:1788880172.725526:0:203392:0:(obd_config.c:683:class_attach_name()) Process entered 00000020:00000001:0.0:1788880172.725528:0:203392:0:(genops.c:350:class_newdev()) Process entered 00000400:00000200:3.0:1788880172.725543:0:201283:0:(peer.c:4145:lnet_peer_discovery_wait_for_work()) woken: 0 00000020:00000010:0.0:1788880172.725546:0:203392:0:(genops.c:51:obd_device_alloc()) slab-alloced 'obd': 6592 at 000000002c7588f3. 00000400:00000200:3.0:1788880172.725548:0:201283:0:(peer.c:4257:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x100041 00000020:00000080:0.0:1788880172.725550:0:203392:0:(genops.c:413:class_newdev()) Allocate new device MGC192.168.204.111@tcp (000000002c7588f3) 00000020:00000001:0.0:1788880172.725552:0:203392:0:(genops.c:952:__class_new_export()) Process entered 00000400:00000010:3.0:1788880172.725553:0:201283:0:(api-ni.c:1817:lnet_ping_buffer_alloc()) alloc '(pbuf)': 281 at 00000000cfd9babf. 00000020:00000010:0.0:1788880172.725555:0:203392:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 0000000040f424e8. 00000020:00000001:0.0:1788880172.725557:0:203392:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:0.0:1788880172.725560:0:203392:0:(lustre_handles.c:93:class_handle_hash()) added object 0000000040f424e8 with handle 0x805d548a36a8864a to hash 00000020:00000001:0.0:1788880172.725562:0:203392:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:0.0:1788880172.725565:0:203392:0:(obd_class.h:1016:obd_init_export()) Process entered 00000400:00000010:3.0:1788880172.725565:0:201283:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ad0499ab. 00000020:00000001:0.0:1788880172.725566:0:203392:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880172.725568:0:203392:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617249890596864 : -126823818954752 : ffff8ca787527800) 00000020:00000040:0.0:1788880172.725570:0:203392:0:(genops.c:900:class_export_put()) PUTting export 0000000040f424e8 : new refcount 1 00000020:00000040:0.0:1788880172.725575:0:203392:0:(obd_config.c:994:class_incref()) incref MGC192.168.204.111@tcp (000000002c7588f3) now 2 - obd_device_list 00000020:00000001:0.0:1788880172.725579:0:203392:0:(genops.c:555:class_register_device()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880172.725580:0:201283:0:(lib-lnet.h:443:lnet_rspt_alloc()) alloc 'rspt': 64 at 000000003270ddf2. 00000020:00000080:0.0:1788880172.725581:0:203392:0:(obd_config.c:720:class_attach_name()) OBD: dev 0 attached type mgc with refcount 2 00000020:00000001:0.0:1788880172.725583:0:203392:0:(obd_config.c:722:class_attach_name()) Process leaving (rc=18446617250083687744 : -126823625863872 : ffff8ca792d4cd40) 00000400:00000200:3.0:1788880172.725585:0:201283:0:(lib-move.c:5362:LNetGet()) LNetGet -> 12345-192.168.204.111@tcp 00000020:00000010:0.0:1788880172.725587:0:203392:0:(obd_mount.c:249:lustre_start_simple()) kmalloced '(lcfg)': 104 at 00000000ee2162e1. 00000020:00000001:0.0:1788880172.725589:0:203392:0:(obd_config.c:742:class_setup()) Process entered 00000400:00000200:3.0:1788880172.725595:0:201283:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880172.725600:0:201283:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880172.725603:0:201283:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000001:00000001:0.0:1788880172.725604:0:203392:0:(hash.c:1038:cfs_hash_create()) Process entered 00000400:00000200:3.0:1788880172.725606:0:201283:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000001:00000010:0.0:1788880172.725608:0:203392:0:(hash.c:1065:cfs_hash_create()) alloc '(hs)': 232 at 00000000a6809731. 00000400:00000200:3.0:1788880172.725609:0:201283:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000001:00000010:0.0:1788880172.725612:0:203392:0:(hash.c:933:cfs_hash_buckets_realloc()) alloc '(new_bkts)': 32 at 000000008511b945. 00000400:00000200:3.0:1788880172.725612:0:201283:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [1:1:256:4294967295] 192.168.204.111@tcp LPNI seq info [1:1:8:4294967295] 00000001:00000010:0.0:1788880172.725615:0:203392:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000092a18e1c. 00000400:00000200:3.0:1788880172.725618:0:201283:0:(lib-move.c:1811:lnet_handle_send()) rspt_next_hop_nid = 192.168.204.111@tcp 00000001:00000010:0.0:1788880172.725621:0:203392:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000071dd3fee. 00000001:00000010:0.0:1788880172.725624:0:203392:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000098961df6. 00000400:00000200:3.0:1788880172.725625:0:201283:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : GET try# 0 00000001:00000010:0.0:1788880172.725626:0:203392:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 00000000bd18c039. 00000020:00000001:0.0:1788880172.725629:0:203392:0:(obd_class.h:565:obd_setup()) Process entered 00000020:00000010:0.0:1788880172.725632:0:203392:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000c19d180e. 00000800:00000200:3.0:1788880172.725633:0:201283:0:(socklnd_cb.c:931:ksocknal_send()) sending 0 bytes in 0 frags to 12345-192.168.204.111@tcp 00000020:00000010:0.0:1788880172.725637:0:203392:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 00000000989c86ee. 00000800:00000010:3.0:1788880172.725637:0:201283:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 232 at 000000005c064165. 00000800:00000010:3.0:1788880172.725645:0:201283:0:(socklnd.c:145:ksocknal_create_peer()) alloc '(peer_ni)': 184 at 00000000b4b9d165. 00000020:00000010:0.0:1788880172.725647:0:203392:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 00000000fabadd93. 00000800:00000010:3.0:1788880172.725648:0:201283:0:(socklnd.c:106:ksocknal_create_conn_cb()) alloc '(conn_cb)': 192 at 00000000a3f3eebd. 00000008:00000010:0.0:1788880172.725652:0:203392:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000ea38ec9a. 00000800:00000200:3.0:1788880172.725653:0:201283:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (2) 00020000:00000010:0.0:1788880172.725657:0:203392:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 00000000cb07dfd8. 00000800:00000200:3.1F:1788880172.725657:0:201283:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (2) 00008000:00000010:0.0:1788880172.725660:0:203392:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 00000000570fe7be. 00000080:00000010:0.0:1788880172.725666:0:203392:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 00000000347372f1. 00000020:00000010:0.0:1788880172.725670:0:203392:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000002b843fc3. 00000020:00000010:0.0:1788880172.725673:0:203392:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000fa0f7569. 00000040:00000010:0.0:1788880172.725677:0:203392:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 0000000026532f47. 00000400:00000200:3.0:1788880172.725677:0:201283:0:(peer.c:3937:lnet_peer_send_ping()) peer 192.168.204.111@tcp 00000400:00000200:3.0:1788880172.725679:0:201283:0:(peer.c:4275:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x100241 rc 0 00000020:00000010:0.0:1788880172.725680:0:203392:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000c85a7d49. 00000020:00000010:0.0:1788880172.725684:0:203392:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 0000000058b4ee38. 00040000:00000010:0.0:1788880172.725688:0:203392:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 00000000b899fe8d. 00040000:00000010:0.0:1788880172.725692:0:203392:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 000000002974b115. 10000000:00000001:0.0:1788880172.725695:0:203392:0:(mgc_request.c:850:mgc_device_alloc()) Process entered 10000000:00000010:0.0:1788880172.725698:0:203392:0:(mgc_request.c:852:mgc_device_alloc()) kmalloced '(lu)': 64 at 00000000095ce225. 00000100:00000001:0.0:1788880172.725703:0:203392:0:(ptlrpcd.c:908:ptlrpcd_addref()) Process entered 00000100:00000001:0.0:1788880172.725705:0:203392:0:(ptlrpcd.c:917:ptlrpcd_addref()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.725708:0:203392:0:(ldlm_lib.c:297:client_obd_setup()) Process entered 00010000:00000010:0.0:1788880172.725713:0:203392:0:(ldlm_lib.c:420:client_obd_setup()) kmalloced '(cli->cl_read_io_latency_by_size)': 3920 at 000000002d0152c4. 00010000:00000010:0.0:1788880172.725717:0:203392:0:(ldlm_lib.c:433:client_obd_setup()) kmalloced '(cli->cl_write_io_latency_by_size)': 3920 at 0000000022135324. 00010000:00000001:0.0:1788880172.725720:0:203392:0:(ldlm_lockd.c:3036:ldlm_get_ref()) Process entered 00010000:00000001:0.0:1788880172.725722:0:203392:0:(ldlm_lockd.c:3375:ldlm_setup()) Process entered 00010000:00000010:0.0:1788880172.725724:0:203392:0:(ldlm_lockd.c:3380:ldlm_setup()) kmalloced '(ldlm_state)': 32 at 000000005ccdfb62. 00010000:00000001:0.0:1788880172.725756:0:203392:0:(ldlm_resource.c:68:ldlm_debugfs_setup()) Process entered 00010000:00000001:0.0:1788880172.725783:0:203392:0:(ldlm_resource.c:78:ldlm_debugfs_setup()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.725786:0:203392:0:(service.c:700:ptlrpc_register_service()) Process entered 00000100:00000010:0.0:1788880172.725788:0:203392:0:(service.c:744:ptlrpc_register_service()) kmalloced '(service)': 352 at 00000000e4fc1d42. 00000100:00000010:0.0:1788880172.725792:0:203392:0:(service.c:802:ptlrpc_register_service()) kmalloced '(svcpt)': 784 at 00000000e777da77. 00000100:00000010:0.0:1788880172.725802:0:203392:0:(service.c:626:ptlrpc_service_part_init()) kmalloced '(array->paa_reqs_array)': 12080 at 00000000cfde11a0. 00000100:00000010:0.0:1788880172.725811:0:203392:0:(service.c:634:ptlrpc_service_part_init()) kmalloced '(array->paa_reqs_count)': 3020 at 00000000587fb32e. 00000100:00000010:0.0:1788880172.725816:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000e885baec. 00000100:00000010:0.0:1788880172.725821:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000060d8d6bb. 00000100:00100000:0.0:1788880172.725823:0:203392:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/1 left), rc = 0 00000400:00000200:0.0:1788880172.725827:0:203392:0:(lib-ptl.c:918:LNetSetLazyPortal()) Setting portal 15 lazy 00000020:00000010:0.0:1788880172.725875:0:203392:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 0000000028978f93. 00000020:00000010:0.0:1788880172.725881:0:203392:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 3680 at 000000001873c9f2. 00000100:00000001:0.0:1788880172.725917:0:203392:0:(nrs.c:1006:nrs_svcpt_setup_locked()) Process entered 00000100:00000001:0.0:1788880172.725920:0:203392:0:(nrs.c:924:nrs_register_policies_locked()) Process entered 00000100:00000001:0.0:1788880172.725921:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000100:00000010:0.0:1788880172.725924:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 0000000059c225df. 00000100:00000001:0.0:1788880172.725926:0:203392:0:(nrs.c:211:nrs_policy_start_locked()) Process entered 00000100:00000010:0.0:1788880172.725929:0:203392:0:(nrs_fifo.c:59:nrs_fifo_start()) kmalloced '(head)': 40 at 00000000209881e7. 00000100:00000001:0.0:1788880172.725930:0:203392:0:(nrs.c:336:nrs_policy_start_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.725931:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.725933:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000100:00000010:0.0:1788880172.725934:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 000000009e4c34e2. 00000100:00000001:0.0:1788880172.725935:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.725937:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000100:00000010:0.0:1788880172.725938:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 00000000d3441c31. 00000100:00000001:0.0:1788880172.725938:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.725939:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000100:00000010:0.0:1788880172.725940:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 000000008d5f52e9. 00000100:00000001:0.0:1788880172.725941:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.725942:0:203392:0:(nrs.c:944:nrs_register_policies_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.725943:0:203392:0:(nrs.c:985:__nrs_svcpt_setup_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.725945:0:203392:0:(nrs.c:1022:nrs_svcpt_setup_locked()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880172.725947:0:203392:0:(nrs.c:1034:nrs_svcpt_setup_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.725977:0:203392:0:(nrs.c:1379:ptlrpc_service_nrs_setup()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880172.725980:0:203392:0:(service.c:847:ptlrpc_register_service()) ldlm_cbd: Started, listening on portal 15 00000100:00000001:0.0:1788880172.725982:0:203392:0:(service.c:3421:ptlrpc_start_threads()) Process entered 00000100:00000001:0.0:1788880172.725984:0:203392:0:(service.c:3454:ptlrpc_start_thread()) Process entered 00000100:00100000:0.0:1788880172.725986:0:203392:0:(service.c:3462:ptlrpc_start_thread()) ldlm_cbd[0] started 0 min 2 max 56 00000100:00000010:0.0:1788880172.725990:0:203392:0:(service.c:3473:ptlrpc_start_thread()) kmalloced '(thread)': 328 at 000000008bc22dd0. 00000100:00100000:0.0:1788880172.725993:0:203392:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cb00_000' 00000100:00000001:0.0:1788880172.726266:0:203393:0:(service.c:3019:ptlrpc_main()) Process entered 00000100:00000010:0.0:1788880172.726282:0:203393:0:(service.c:3046:ptlrpc_main()) kmalloced '(env)': 56 at 00000000ae39ad3e. 00000020:00000010:0.0:1788880172.726286:0:203393:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 00000000e44f6a91. 00000020:00000010:0.0:1788880172.726291:0:203393:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000004d26bb28. 00000020:00000010:0.0:1788880172.726293:0:203393:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000007687a45b. 00000040:00000010:0.0:1788880172.726295:0:203393:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000e9bfd20c. 00000020:00000010:0.0:1788880172.726298:0:203393:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000d5dde937. 00000020:00000010:0.0:1788880172.726313:0:203393:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 0000000001020e32. 00000020:00000010:0.0:1788880172.726318:0:203393:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 0000000082b0b134. 80000000:00000010:0.0:1788880172.726321:0:203393:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 0000000065e39822. 40000000:00000010:0.0:1788880172.726325:0:203393:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 000000002794c770. 00100000:00000010:0.0:1788880172.726329:0:203393:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 000000006913536b. 00040000:00000010:0.0:1788880172.726332:0:203393:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 00000000cadb8ebb. 00040000:00000010:0.0:1788880172.726337:0:203393:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 00000000599a18f4. 00040000:00000010:0.0:1788880172.726339:0:203393:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 000000008559d53d. 00000004:00000010:0.0:1788880172.726343:0:203393:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 00000000334f2be2. 00000004:00000010:0.0:1788880172.726347:0:203393:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 000000009b6b5630. 00000004:00000010:0.0:1788880172.726351:0:203393:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 00000000c44681e1. 00002000:00000010:0.0:1788880172.726355:0:203393:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 0000000056de2fd7. 00000004:00000010:0.0:1788880172.726358:0:203393:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 000000008824e783. 00000100:00000200:0.0:1788880172.726362:0:203393:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_cbd: registering portal 15 00000400:00000010:0.0:1788880172.726367:0:203393:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dd218a96. 00000400:00000010:0.0:1788880172.726371:0:203393:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000033227dba. 00000100:00000010:0.0:1788880172.726379:0:203393:0:(service.c:3073:ptlrpc_main()) kmalloced '(rs)': 2048 at 00000000e7fe2602. 00000100:00000200:0.0:1788880172.726394:0:203393:0:(service.c:3107:ptlrpc_main()) service thread 0 (#1) started 00000100:00000001:2.0:1788880172.726409:0:203392:0:(service.c:3549:ptlrpc_start_thread()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.726413:0:203392:0:(service.c:3454:ptlrpc_start_thread()) Process entered 00000100:00100000:2.0:1788880172.726414:0:203392:0:(service.c:3462:ptlrpc_start_thread()) ldlm_cbd[0] started 1 min 2 max 56 00000100:00000010:2.0:1788880172.726419:0:203392:0:(service.c:3473:ptlrpc_start_thread()) kmalloced '(thread)': 328 at 000000007c073719. 00000100:00100000:2.0:1788880172.726423:0:203392:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cb00_001' 00000800:00000010:0.0:1788880172.726537:0:201273:0:(socklnd.c:886:ksocknal_create_conn()) alloc '(conn)': 4720 at 00000000fd8c4485. 00000800:00000010:0.0:1788880172.726543:0:201273:0:(socklnd.c:913:ksocknal_create_conn()) alloc '(hello)': 144 at 0000000026efb8e8. 00000800:00000010:0.0:1788880172.726550:0:201273:0:(socklnd_proto.c:573:ksocknal_send_hello_v2()) alloc '(hello4)': 56 at 000000008c244d33. 00000800:00000010:0.0:1788880172.726559:0:201273:0:(socklnd_proto.c:613:ksocknal_send_hello_v2()) kfreed 'hello4': 56 at 000000008c244d33. 00000100:00000001:3.0:1788880172.726699:0:203394:0:(service.c:3019:ptlrpc_main()) Process entered 00000100:00000010:3.0:1788880172.726726:0:203394:0:(service.c:3046:ptlrpc_main()) kmalloced '(env)': 56 at 0000000002dcf5c4. 00000020:00000010:3.0:1788880172.726734:0:203394:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 00000000c17c2643. 00000020:00000010:3.0:1788880172.726740:0:203394:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000004e90d55f. 00000020:00000010:3.0:1788880172.726744:0:203394:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000bcf81c94. 00000040:00000010:3.0:1788880172.726749:0:203394:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000d5485bcf. 00000020:00000010:3.0:1788880172.726754:0:203394:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 000000007b25471a. 00000020:00000010:3.0:1788880172.726784:0:203394:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 00000000b9573a80. 00000020:00000010:3.0:1788880172.726790:0:203394:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000ec56f3bd. 80000000:00000010:3.0:1788880172.726795:0:203394:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000782bf1ab. 40000000:00000010:3.0:1788880172.726799:0:203394:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 00000000ead89afe. 00100000:00000010:3.0:1788880172.726804:0:203394:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 00000000e5a4e2ab. 00040000:00000010:3.0:1788880172.726808:0:203394:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 000000007abdd095. 00040000:00000010:3.0:1788880172.726812:0:203394:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 00000000d2ffcafb. 00040000:00000010:3.0:1788880172.726816:0:203394:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 000000001a959f0b. 00000004:00000010:3.0:1788880172.726820:0:203394:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 000000000f2e6431. 00000004:00000010:3.0:1788880172.726826:0:203394:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 0000000091d158b8. 00000004:00000010:3.0:1788880172.726927:0:203394:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 0000000031ca5b3e. 00002000:00000010:3.0:1788880172.726933:0:203394:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 00000000290361f0. 00000004:00000010:3.0:1788880172.726937:0:203394:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 0000000083cac0e2. 00000100:00000010:3.0:1788880172.726944:0:203394:0:(service.c:3073:ptlrpc_main()) kmalloced '(rs)': 2048 at 0000000076b1437a. 00000100:00000200:3.0:1788880172.726961:0:203394:0:(service.c:3107:ptlrpc_main()) service thread 1 (#2) started 00000100:00000001:2.0:1788880172.726974:0:203392:0:(service.c:3549:ptlrpc_start_thread()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.726978:0:203392:0:(service.c:3439:ptlrpc_start_threads()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.726979:0:203392:0:(service.c:856:ptlrpc_register_service()) Process leaving (rc=18446617249883676672 : -126823825874944 : ffff8ca786e8e000) 00000100:00000001:2.0:1788880172.726982:0:203392:0:(service.c:700:ptlrpc_register_service()) Process entered 00000100:00000010:2.0:1788880172.726985:0:203392:0:(service.c:744:ptlrpc_register_service()) kmalloced '(service)': 352 at 000000000a5fedd1. 00000100:00000010:2.0:1788880172.726988:0:203392:0:(service.c:802:ptlrpc_register_service()) kmalloced '(svcpt)': 784 at 00000000c1d38e39. 00000100:00000010:2.0:1788880172.727000:0:203392:0:(service.c:626:ptlrpc_service_part_init()) kmalloced '(array->paa_reqs_array)': 12080 at 00000000bebe5c07. 00000100:00000010:2.0:1788880172.727008:0:203392:0:(service.c:634:ptlrpc_service_part_init()) kmalloced '(array->paa_reqs_count)': 3020 at 00000000b8c79f0f. 00000100:00000010:2.0:1788880172.727014:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000050ed2762. 00000100:00000010:2.0:1788880172.727019:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000331f1cf2. 00000100:00000010:2.0:1788880172.727021:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000a99f77d1. 00000100:00000010:2.0:1788880172.727025:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000bbff0964. 00000100:00000010:2.0:1788880172.727038:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000064626411. 00000100:00000010:2.0:1788880172.727042:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000876a666c. 00000100:00000010:2.0:1788880172.727050:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000ba85c875. 00000100:00000010:2.0:1788880172.727054:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000053c1b352. 00000100:00000010:2.0:1788880172.727056:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000d1ca9b49. 00000100:00000010:2.0:1788880172.727061:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000059c1a800. 00000100:00000010:2.0:1788880172.727062:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000005241a643. 00000100:00000010:2.0:1788880172.727066:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000064162764. 00000100:00000010:2.0:1788880172.727068:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000dfb3a657. 00000100:00000010:2.0:1788880172.727071:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000007a30dea6. 00000100:00000010:2.0:1788880172.727073:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000cc01c627. 00000100:00000010:2.0:1788880172.727077:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000fde91079. 00000100:00000010:2.0:1788880172.727080:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000001517f4ee. 00000100:00000010:2.0:1788880172.727083:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000003ddd44a6. 00000100:00000010:2.0:1788880172.727085:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000aa25e28c. 00000100:00000010:2.0:1788880172.727089:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000009c9f9650. 00000100:00000010:2.0:1788880172.727091:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000096a89ed2. 00000100:00000010:2.0:1788880172.727094:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000f48c0fc7. 00000100:00000010:2.0:1788880172.727096:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000006b1a0253. 00000100:00000010:2.0:1788880172.727100:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000000cf01149. 00000100:00000010:2.0:1788880172.727102:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000009079ea97. 00000100:00000010:2.0:1788880172.727106:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000f6303426. 00000100:00000010:2.0:1788880172.727107:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000024a49945. 00000100:00000010:2.0:1788880172.727111:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000251c4486. 00000100:00000010:2.0:1788880172.727113:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000ccbce4b0. 00000100:00000010:2.0:1788880172.727116:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000003bc98bb5. 00000100:00000010:2.0:1788880172.727118:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000002e026617. 00000100:00000010:2.0:1788880172.727121:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000770705aa. 00000100:00000010:2.0:1788880172.727123:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000966ef2d4. 00000100:00000010:2.0:1788880172.727133:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000ca23d30e. 00000100:00000010:2.0:1788880172.727135:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000057a06a78. 00000100:00000010:2.0:1788880172.727139:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000063d99b0f. 00000100:00000010:2.0:1788880172.727141:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000007a96adb8. 00000100:00000010:2.0:1788880172.727148:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000000981b201. 00000100:00000010:2.0:1788880172.727150:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000084bce416. 00000100:00000010:2.0:1788880172.727153:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000b5fa65f9. 00000100:00000010:2.0:1788880172.727155:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000370034c3. 00000100:00000010:2.0:1788880172.727163:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000009953012a. 00000100:00000010:2.0:1788880172.727167:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000ecf7fdd6. 00000100:00000010:2.0:1788880172.727171:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000008f6e3108. 00000100:00000010:2.0:1788880172.727173:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000e4e9486e. 00000100:00000010:2.0:1788880172.727177:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000bca6883f. 00000100:00000010:2.0:1788880172.727179:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000064338522. 00000100:00000010:2.0:1788880172.727183:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000bf51b8ad. 00000100:00000010:2.0:1788880172.727206:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000039bcafa9. 00000100:00000010:2.0:1788880172.727217:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000e1d43e5e. 00000100:00000010:2.0:1788880172.727220:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000110ab002. 00000100:00000010:2.0:1788880172.727224:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000104a1037. 00000100:00000010:2.0:1788880172.727226:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000df5daca9. 00000100:00000010:2.0:1788880172.727229:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000010e2d161. 00000100:00000010:2.0:1788880172.727231:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000044dea53d. 00000800:00000200:0.1F:1788880172.727583:0:201273:0:(socklnd.c:384:ksocknal_incr_conn_count()) Add conn type 1, ksnr_connected 2 ksnr_max_conns 1 00000800:00000200:0.1:1788880172.727599:0:201273:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.1:1788880172.727604:0:201273:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000005c064165 type 2, nob 96 niov 1 nkiov 0 00000800:00000200:0.0:1788880172.727612:0:201273:0:(socklnd.c:1182:ksocknal_create_conn()) New conn 12345-192.168.204.111@tcp p 3.x 192.168.204.11 -> 192.168.204.111:988 incarnation:1788880146239320504 sched[0] 00000800:00000010:0.0:1788880172.727616:0:201273:0:(socklnd.c:1190:ksocknal_create_conn()) kfreed 'hello': 144 at 0000000026efb8e8. 00000400:00000200:0.0:1788880172.727696:0:201273:0:(lib-socket.c:438:lnet_sock_connect()) Error -99 connecting 1023 -> 192.168.204.111:988 00000100:00000010:2.0:1788880172.727832:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000b99b92de. 00000100:00000010:2.0:1788880172.727836:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000002c09fd43. 00000100:00000010:2.0:1788880172.727847:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000002ed7d18e. 00000100:00000010:2.0:1788880172.727851:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000023ebaca1. 00000100:00000010:2.0:1788880172.727855:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000edcac5d7. 00000100:00000010:2.0:1788880172.727858:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000004a5a9001. 00000100:00000010:2.0:1788880172.727861:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000b016f163. 00000100:00000010:2.0:1788880172.727864:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000001f32ca24. 00000100:00000010:2.0:1788880172.727868:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000015539d40. 00000100:00000010:2.0:1788880172.727871:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000078e577f5. 00000100:00000010:2.0:1788880172.727882:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000f8ad7f99. 00000100:00000010:2.0:1788880172.727895:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000008a640c2f. 00000100:00000010:2.0:1788880172.727903:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000c3442c9e. 00000100:00000010:2.0:1788880172.727910:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000002637d61c. 00000100:00000010:2.0:1788880172.727914:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000028d70ab3. 00000100:00000010:2.0:1788880172.727916:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000066ce66a2. 00000100:00000010:2.0:1788880172.727920:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000ab378891. 00000100:00000010:2.0:1788880172.727923:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000068638bed. 00000100:00000010:2.0:1788880172.727932:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000e991bd9f. 00000100:00000010:2.0:1788880172.727935:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000d59a7f4c. 00000100:00000010:2.0:1788880172.727939:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000f29ed6d7. 00000100:00000010:2.0:1788880172.727941:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000008b76c080. 00000100:00000010:2.0:1788880172.727945:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000004222cc67. 00000100:00000010:2.0:1788880172.727947:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000509140ae. 00000100:00000010:2.0:1788880172.727951:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000d3d5e02c. 00000100:00000010:2.0:1788880172.727953:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000005392135d. 00000100:00000010:2.0:1788880172.727961:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000003ed8b9e8. 00000100:00000010:2.0:1788880172.727964:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000b8eac732. 00000100:00000010:2.0:1788880172.727968:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000000b0f99a9. 00000100:00000010:2.0:1788880172.727970:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000033b4d19e. 00000100:00000010:2.0:1788880172.727973:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000c94f4add. 00000100:00000010:2.0:1788880172.727975:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000004d97c5bb. 00000100:00000010:2.0:1788880172.727978:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000007b27c93b. 00000100:00000010:2.0:1788880172.727981:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000007506a8a2. 00000100:00000010:2.0:1788880172.727988:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000075a91ae4. 00000100:00000010:2.0:1788880172.727991:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000ad119b9c. 00000100:00000010:2.0:1788880172.727994:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000421b2be3. 00000100:00000010:2.0:1788880172.727996:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000007f7bce63. 00000100:00000010:2.0:1788880172.728000:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000252c5f70. 00000100:00000010:2.0:1788880172.728002:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000002023558f. 00000100:00000010:2.0:1788880172.728005:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000f03918e8. 00000100:00000010:2.0:1788880172.728007:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000044206597. 00000100:00000010:2.0:1788880172.728015:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000009af2c9d8. 00000100:00000010:2.0:1788880172.728018:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000251ed2e1. 00000100:00000010:2.0:1788880172.728026:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000016c4783c. 00000100:00000010:2.0:1788880172.728039:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000004853ebab. 00000100:00000010:2.0:1788880172.728042:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000011e0a171. 00000100:00000010:2.0:1788880172.728045:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000890581c7. 00000100:00000010:2.0:1788880172.728049:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000098284d42. 00000100:00000010:2.0:1788880172.728052:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000099228fc7. 00000100:00000010:2.0:1788880172.728061:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000005151ee50. 00000100:00000010:2.0:1788880172.728064:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000274f459d. 00000100:00000010:2.0:1788880172.728067:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000005034e357. 00000100:00000010:2.0:1788880172.728070:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000009fbccbf3. 00000100:00000010:2.0:1788880172.728074:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000be02de9a. 00000100:00000010:2.0:1788880172.728078:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000009497a65d. 00000100:00000010:2.0:1788880172.728082:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000009da85fed. 00000100:00000010:2.0:1788880172.728085:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000b687dc7c. 00000100:00000010:2.0:1788880172.728094:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000046c04b2e. 00000100:00000010:2.0:1788880172.728097:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000e2250be2. 00000100:00000010:2.0:1788880172.728101:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000649d0871. 00000100:00000010:2.0:1788880172.728104:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 0000000076592dbe. 00000100:00000010:2.0:1788880172.728107:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000080530b71. 00000100:00000010:2.0:1788880172.728110:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000fe1a96fb. 00000100:00000010:2.0:1788880172.728114:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000ddba919f. 00000100:00000010:2.0:1788880172.728116:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000f9a87171. 00000100:00000010:2.0:1788880172.728125:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000927d674a. 00000100:00000010:2.0:1788880172.728128:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000e277b57b. 00000100:00000010:2.0:1788880172.728132:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 0000000034d0832d. 00000100:00000010:2.0:1788880172.728136:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 00000000e3f94b83. 00000100:00000010:2.0:1788880172.728139:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 000000008282126e. 00000100:00000010:2.0:1788880172.728144:0:203392:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000006a9415dc. 00000100:00000010:2.0:1788880172.728148:0:203392:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000dcf4d021. 00000100:00100000:2.0:1788880172.728150:0:203392:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_canceld: allocate 64 new 8192-byte reqbufs (0/64 left), rc = 0 00000400:00000200:2.0:1788880172.728155:0:203392:0:(lib-ptl.c:918:LNetSetLazyPortal()) Setting portal 17 lazy 00000020:00000010:2.0:1788880172.728206:0:203392:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 000000003cad9c35. 00000020:00000010:2.0:1788880172.728211:0:203392:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 3680 at 00000000cef393af. 00000100:00000001:2.0:1788880172.728280:0:203392:0:(nrs.c:1006:nrs_svcpt_setup_locked()) Process entered 00000100:00000001:2.0:1788880172.728283:0:203392:0:(nrs.c:924:nrs_register_policies_locked()) Process entered 00000800:00000010:0.0:1788880172.728284:0:201273:0:(socklnd.c:886:ksocknal_create_conn()) alloc '(conn)': 4720 at 000000005badf84a. 00000100:00000001:2.0:1788880172.728286:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000100:00000010:2.0:1788880172.728290:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 000000001cdd6d3f. 00000800:00000010:0.0:1788880172.728293:0:201273:0:(socklnd.c:913:ksocknal_create_conn()) alloc '(hello)': 144 at 0000000026efb8e8. 00000100:00000001:2.0:1788880172.728294:0:203392:0:(nrs.c:211:nrs_policy_start_locked()) Process entered 00000800:00000010:0.0:1788880172.728300:0:201273:0:(socklnd_proto.c:573:ksocknal_send_hello_v2()) alloc '(hello4)': 56 at 000000008c244d33. 00000100:00000010:2.0:1788880172.728300:0:203392:0:(nrs_fifo.c:59:nrs_fifo_start()) kmalloced '(head)': 40 at 0000000080ae8f7b. 00000100:00000001:2.0:1788880172.728305:0:203392:0:(nrs.c:336:nrs_policy_start_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728307:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728309:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000800:00000010:0.0:1788880172.728311:0:201273:0:(socklnd_proto.c:613:ksocknal_send_hello_v2()) kfreed 'hello4': 56 at 000000008c244d33. 00000100:00000010:2.0:1788880172.728312:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 00000000dc757cfa. 00000100:00000001:2.0:1788880172.728317:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728319:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000100:00000010:2.0:1788880172.728321:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 000000006ce46881. 00000100:00000001:2.0:1788880172.728323:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728324:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000100:00000010:2.0:1788880172.728326:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 000000005d09ea17. 00000100:00000001:2.0:1788880172.728328:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728329:0:203392:0:(nrs.c:944:nrs_register_policies_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728330:0:203392:0:(nrs.c:985:__nrs_svcpt_setup_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880172.728333:0:203392:0:(nrs.c:1026:nrs_svcpt_setup_locked()) kmalloced '(svcpt->scp_nrs_hp)': 112 at 000000007b3bc1c0. 00000100:00000001:2.0:1788880172.728334:0:203392:0:(nrs.c:924:nrs_register_policies_locked()) Process entered 00000100:00000001:2.0:1788880172.728335:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000100:00000010:2.0:1788880172.728337:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 0000000008ddf3e1. 00000100:00000001:2.0:1788880172.728338:0:203392:0:(nrs.c:211:nrs_policy_start_locked()) Process entered 00000100:00000010:2.0:1788880172.728340:0:203392:0:(nrs_fifo.c:59:nrs_fifo_start()) kmalloced '(head)': 40 at 00000000fb9c1e17. 00000100:00000001:2.0:1788880172.728341:0:203392:0:(nrs.c:336:nrs_policy_start_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728343:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728344:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000100:00000010:2.0:1788880172.728346:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 00000000689756ba. 00000100:00000001:2.0:1788880172.728347:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728348:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000100:00000010:2.0:1788880172.728350:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 0000000047684423. 00000100:00000001:2.0:1788880172.728352:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728354:0:203392:0:(nrs.c:783:nrs_policy_register()) Process entered 00000100:00000010:2.0:1788880172.728355:0:203392:0:(nrs.c:794:nrs_policy_register()) kmalloced '(policy)': 152 at 00000000a7617fbc. 00000100:00000001:2.0:1788880172.728364:0:203392:0:(nrs.c:842:nrs_policy_register()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728365:0:203392:0:(nrs.c:944:nrs_register_policies_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728367:0:203392:0:(nrs.c:985:__nrs_svcpt_setup_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728369:0:203392:0:(nrs.c:1034:nrs_svcpt_setup_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.728404:0:203392:0:(nrs.c:1379:ptlrpc_service_nrs_setup()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880172.728406:0:203392:0:(service.c:847:ptlrpc_register_service()) ldlm_canceld: Started, listening on portal 17 00000100:00000001:2.0:1788880172.728408:0:203392:0:(service.c:3421:ptlrpc_start_threads()) Process entered 00000100:00000001:2.0:1788880172.728410:0:203392:0:(service.c:3454:ptlrpc_start_thread()) Process entered 00000100:00100000:2.0:1788880172.728412:0:203392:0:(service.c:3462:ptlrpc_start_thread()) ldlm_canceld[0] started 0 min 3 max 56 00000100:00000010:2.0:1788880172.728416:0:203392:0:(service.c:3473:ptlrpc_start_thread()) kmalloced '(thread)': 328 at 000000001d7cf3c2. 00000100:00100000:2.0:1788880172.728419:0:203392:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cn00_000' 00000800:00000001:0.0:1788880172.728434:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.728447:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.728450:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.728452:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.728459:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : REPLY - for me 00000400:00000200:0.0:1788880172.728481:0:201280:0:(lib-move.c:4451:lnet_parse_reply()) 192.168.204.11@tcp: Reply from 12345-192.168.204.111@tcp of length 48/48 into md 0x9 00000800:00000001:0.0:1788880172.728487:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.728493:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.728495:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.728500:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: REPLY: OK 00000400:00000200:0.0:1788880172.728505:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ad0499ab 00000400:00000010:0.0:1788880172.728508:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ad0499ab. 00000400:00000200:0.0:1788880172.728511:0:201280:0:(peer.c:3204:lnet_discovery_event_handler()) Received event: 3 00000400:00000200:0.0:1788880172.728515:0:201280:0:(peer.c:3034:lnet_discovery_event_reply()) Peer 192.168.204.111@tcp has discovery enabled 00000400:00000200:0.0:1788880172.728517:0:201280:0:(peer.c:3046:lnet_discovery_event_reply()) peer 192.168.204.111@tcp(00000000b0c52047) is MR 00000400:00000200:0.0:1788880172.728522:0:201280:0:(peer.c:3115:lnet_discovery_event_reply()) peer 192.168.204.111@tcp data present 363512030. state = 0x100241 00000400:00000200:0.0:1788880172.728527:0:201280:0:(peer.c:2528:lnet_peer_queue_for_discovery()) Queue peer 192.168.204.111@tcp: -114 00000800:00000001:0.0:1788880172.728542:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.728546:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880172.728584:0:201283:0:(peer.c:4145:lnet_peer_discovery_wait_for_work()) woken: 0 00000400:00000200:3.0:1788880172.728595:0:201283:0:(peer.c:4257:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x1000c1 00000400:00000010:3.0:1788880172.728605:0:201283:0:(peer.c:3389:lnet_peer_merge_data()) alloc '(curnis)': 40 at 000000002785155f. 00000100:00000001:2.0:1788880172.728616:0:203395:0:(service.c:3019:ptlrpc_main()) Process entered 00000400:00000010:3.0:1788880172.728618:0:201283:0:(peer.c:3390:lnet_peer_merge_data()) alloc '(addnis)': 48 at 00000000022fb4a4. 00000400:00000010:3.0:1788880172.728631:0:201283:0:(peer.c:3391:lnet_peer_merge_data()) alloc '(delnis)': 40 at 00000000fb339cc9. 00000100:00000010:2.0:1788880172.728635:0:203395:0:(service.c:3046:ptlrpc_main()) kmalloced '(env)': 56 at 000000005be18aba. 00000400:00000010:3.0:1788880172.728639:0:201283:0:(peer.c:3538:lnet_peer_merge_data()) kfreed 'curnis': 40 at 000000002785155f. 00000020:00000010:2.0:1788880172.728641:0:203395:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 00000000c0a6c5c6. 00000400:00000010:3.0:1788880172.728644:0:201283:0:(peer.c:3539:lnet_peer_merge_data()) kfreed 'addnis': 48 at 00000000022fb4a4. 00000020:00000010:2.0:1788880172.728649:0:203395:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000001737984d. 00000400:00000010:3.0:1788880172.728650:0:201283:0:(peer.c:3540:lnet_peer_merge_data()) kfreed 'delnis': 40 at 00000000fb339cc9. 00000400:00000010:3.0:1788880172.728658:0:201283:0:(api-ni.c:1835:lnet_ping_buffer_free()) kfreed 'pbuf': 281 at 00000000cfd9babf. 00000020:00000010:2.0:1788880172.728659:0:203395:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 0000000070a29cc3. 00000400:00000200:3.0:1788880172.728667:0:201283:0:(peer.c:3543:lnet_peer_merge_data()) peer 192.168.204.111@tcp (00000000b0c52047): 0 00000020:00000010:2.0:1788880172.728668:0:203395:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000af4c784e. 00000040:00000010:2.0:1788880172.728675:0:203395:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000a7d38de3. 00000400:00000200:3.0:1788880172.728676:0:201283:0:(peer.c:3855:lnet_peer_data_present()) peer 192.168.204.111@tcp(00000000b0c52047): 0. state = 0x100141 00000020:00000010:2.0:1788880172.728682:0:203395:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 000000008da0a49d. 00000400:00000200:3.0:1788880172.728685:0:201283:0:(peer.c:4275:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x100141 rc 1 00000400:00000200:3.0:1788880172.728695:0:201283:0:(peer.c:4257:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x100141 00000400:00000010:3.0:1788880172.728702:0:201283:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000020:00000010:2.0:1788880172.728704:0:203395:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 00000000a16d78ac. 00000400:00000010:3.0:1788880172.728714:0:201283:0:(lib-lnet.h:443:lnet_rspt_alloc()) alloc 'rspt': 64 at 00000000295db731. 00000020:00000010:2.0:1788880172.728715:0:203395:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 000000001f6bef26. 80000000:00000010:2.0:1788880172.728724:0:203395:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 000000009fdd5093. 00000400:00000200:3.0:1788880172.728726:0:201283:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 40000000:00000010:2.0:1788880172.728731:0:203395:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 00000000d6d3ba14. 00000400:00000200:3.0:1788880172.728743:0:201283:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000008:00000010:2.0:1788880172.728747:0:203395:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 000000008f41a84a. 00000400:00000200:3.0:1788880172.728754:0:201283:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [2:2:256:4294967295] 192.168.204.111@tcp LPNI seq info [2:2:8:4294967295] 00020000:00000010:2.0:1788880172.728755:0:203395:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 000000008e37fc82. 00000400:00000200:3.0:1788880172.728769:0:201283:0:(lib-move.c:1811:lnet_handle_send()) rspt_next_hop_nid = 192.168.204.111@tcp 00008000:00000010:2.0:1788880172.728775:0:203395:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000b8830858. 00000400:00000200:3.0:1788880172.728779:0:201283:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00100000:00000010:2.0:1788880172.728787:0:203395:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 00000000ce8bd726. 00000800:00000200:3.0:1788880172.728793:0:201283:0:(socklnd_cb.c:931:ksocknal_send()) sending 48 bytes in 1 frags to 12345-192.168.204.111@tcp 00040000:00000010:2.0:1788880172.728794:0:203395:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 000000001f1e8766. 00000800:00000010:3.0:1788880172.728802:0:201283:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000336bf630. 00040000:00000010:2.0:1788880172.728803:0:203395:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 0000000032e22470. 00040000:00000010:2.0:1788880172.728813:0:203395:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000af4c5baa. 00000800:00000200:3.0:1788880172.728819:0:201283:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (3) 00000004:00000010:2.0:1788880172.728823:0:203395:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 00000000a2ba9f56. 00000004:00000010:2.0:1788880172.728829:0:203395:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 0000000061aebc63. 00000004:00000010:2.0:1788880172.728833:0:203395:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 0000000047a22bb7. 00002000:00000010:2.0:1788880172.728836:0:203395:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 0000000088f5e206. 00000004:00000010:2.0:1788880172.728846:0:203395:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 00000000b527b12e. 00000080:00000010:2.0:1788880172.728856:0:203395:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 000000008771ecc9. 00000080:00000010:2.0:1788880172.728869:0:203395:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 00000000c6ca86db. 00000800:00000200:3.0:1788880172.728871:0:201283:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:2.0:1788880172.728878:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000800:00000200:3.0:1788880172.728879:0:201283:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000336bf630 type 1, nob 144 niov 1 nkiov 1 00000400:00000010:2.0:1788880172.728886:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000abe18438. 00000400:00000010:2.0:1788880172.728893:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000005ca561a8. 00000100:00000200:2.0:1788880172.728897:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.728899:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004f015181. 00000400:00000010:2.0:1788880172.728902:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000abf49607. 00000400:00000200:3.0:1788880172.728905:0:201283:0:(peer.c:4079:lnet_peer_send_push()) peer 192.168.204.111@tcp 00000100:00000200:2.0:1788880172.728907:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000200:3.0:1788880172.728911:0:201283:0:(peer.c:4275:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x100541 rc 0 00000400:00000010:2.0:1788880172.728913:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001531ac25. 00000400:00000010:2.0:1788880172.728919:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000efcdbfa3. 00000100:00000200:2.0:1788880172.728925:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.728928:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000902be105. 00000400:00000010:2.0:1788880172.728931:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000f55761aa. 00000100:00000200:2.0:1788880172.728935:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.728937:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005db6fe6a. 00000400:00000010:2.0:1788880172.728941:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000001e46c24. 00000100:00000200:2.0:1788880172.728944:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.728946:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000040f70814. 00000400:00000010:2.0:1788880172.728948:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000096e2aa42. 00000100:00000200:2.0:1788880172.728950:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.728951:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007fea3e84. 00000400:00000010:2.0:1788880172.728953:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000f62d908a. 00000100:00000200:2.0:1788880172.728955:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.728956:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000279874df. 00000400:00000010:2.0:1788880172.728958:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000004d9005dc. 00000100:00000200:2.0:1788880172.728961:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.728963:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bfdb94dd. 00000400:00000010:2.0:1788880172.728965:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000062032505. 00000100:00000200:2.0:1788880172.728967:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.728969:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000e385f500. 00000400:00000010:2.0:1788880172.728972:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000000aa05dfa. 00000100:00000200:2.0:1788880172.728974:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.728976:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000044bbf538. 00000800:00000200:0.0:1788880172.728980:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880172.728985:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000336bf630. 00000400:00000010:2.0:1788880172.728985:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000d9b4cd0b. 00000100:00000200:2.0:1788880172.728990:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000200:0.0:1788880172.728992:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:2.0:1788880172.728994:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ba68ed13. 00000400:00000010:2.0:1788880172.728997:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000027907bc2. 00000400:00000200:0.0:1788880172.729001:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:2.0:1788880172.729002:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729006:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000002327e2eb. 00000400:00000200:0.0:1788880172.729008:0:201279:0:(peer.c:3204:lnet_discovery_event_handler()) Received event: 5 00000400:00000010:2.0:1788880172.729010:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000059e7c7c1. 00000400:00000200:0.0:1788880172.729014:0:201279:0:(peer.c:3156:lnet_discovery_event_send()) Push Send to 192.168.204.111@tcp: 0 00000100:00000200:2.0:1788880172.729015:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729019:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b77e6339. 00000400:00000010:2.0:1788880172.729022:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000051361f33. 00000100:00000200:2.0:1788880172.729026:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729043:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000cb5409c7. 00000400:00000010:2.0:1788880172.729046:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000ea385cc8. 00000100:00000200:2.0:1788880172.729048:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729051:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f8d92217. 00000400:00000010:2.0:1788880172.729053:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000b2ad85fa. 00000100:00000200:2.0:1788880172.729055:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729057:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ceb453be. 00000400:00000010:2.0:1788880172.729060:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000003ca52b59. 00000100:00000200:2.0:1788880172.729065:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729068:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000003a348e6a. 00000400:00000010:2.0:1788880172.729071:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000008fc6c96c. 00000100:00000200:2.0:1788880172.729073:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729074:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033ce1655. 00000400:00000010:2.0:1788880172.729076:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000003f9ec611. 00000100:00000200:2.0:1788880172.729078:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729080:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000087dd3e5. 00000400:00000010:2.0:1788880172.729082:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000016fa5755. 00000100:00000200:2.0:1788880172.729084:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729086:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c532162d. 00000400:00000010:2.0:1788880172.729088:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000dfe25f08. 00000100:00000200:2.0:1788880172.729089:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729090:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000075fae383. 00000400:00000010:2.0:1788880172.729092:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000002501ebf3. 00000100:00000200:2.0:1788880172.729093:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729101:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000032fdd71. 00000400:00000010:2.0:1788880172.729104:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000006b4cd2e2. 00000100:00000200:2.0:1788880172.729107:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729108:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcc6262c. 00000400:00000010:2.0:1788880172.729110:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000037f85a18. 00000100:00000200:2.0:1788880172.729112:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729114:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000618f2582. 00000400:00000010:2.0:1788880172.729116:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000d9c64659. 00000100:00000200:2.0:1788880172.729117:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729123:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008557a526. 00000400:00000010:2.0:1788880172.729126:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000ece8af28. 00000100:00000200:2.0:1788880172.729128:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729129:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a14f255d. 00000400:00000010:2.0:1788880172.729131:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000027bb0227. 00000100:00000200:2.0:1788880172.729133:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729135:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000224359bf. 00000400:00000010:2.0:1788880172.729137:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000003eee9c4e. 00000100:00000200:2.0:1788880172.729139:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729141:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000002143a046. 00000400:00000010:2.0:1788880172.729143:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000ddef2c90. 00000100:00000200:2.0:1788880172.729144:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729146:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000011a68ce1. 00000400:00000010:2.0:1788880172.729148:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000006df4b93f. 00000100:00000200:2.0:1788880172.729149:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729151:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000076f08651. 00000400:00000010:2.0:1788880172.729154:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000003f905a07. 00000100:00000200:2.0:1788880172.729156:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729157:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000d5448b5f. 00000400:00000010:2.0:1788880172.729159:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000002dba9f5. 00000100:00000200:2.0:1788880172.729161:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729163:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000030b7795c. 00000400:00000010:2.0:1788880172.729165:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000c7dc9986. 00000100:00000200:2.0:1788880172.729167:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729170:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fd36f67b. 00000400:00000010:2.0:1788880172.729173:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000ac7c53d5. 00000100:00000200:2.0:1788880172.729182:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729185:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000003b9421e8. 00000400:00000010:2.0:1788880172.729188:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000063bbc1fa. 00000100:00000200:2.0:1788880172.729190:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729191:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000e7611a22. 00000400:00000010:2.0:1788880172.729194:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000009f841296. 00000100:00000200:2.0:1788880172.729196:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729198:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fd8da712. 00000400:00000010:2.0:1788880172.729200:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000006c73997e. 00000100:00000200:2.0:1788880172.729202:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729204:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a4832130. 00000400:00000010:2.0:1788880172.729206:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000e4b5d4dd. 00000100:00000200:2.0:1788880172.729207:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729209:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f7861a81. 00000400:00000010:2.0:1788880172.729211:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000032a391ab. 00000100:00000200:2.0:1788880172.729212:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729214:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000092355081. 00000400:00000010:2.0:1788880172.729217:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000003b613ead. 00000100:00000200:2.0:1788880172.729219:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729221:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000cebcad03. 00000400:00000010:2.0:1788880172.729224:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000d8ba0780. 00000100:00000200:2.0:1788880172.729228:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729231:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000231f53c9. 00000400:00000010:2.0:1788880172.729273:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000095e8495b. 00000100:00000200:2.0:1788880172.729279:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729283:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007544437e. 00000400:00000010:2.0:1788880172.729287:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000206fa323. 00000800:00000200:0.1:1788880172.729291:0:201273:0:(socklnd.c:384:ksocknal_incr_conn_count()) Add conn type 2, ksnr_connected 6 ksnr_max_conns 1 00000100:00000200:2.0:1788880172.729292:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729304:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000099081ed7. 00000400:00000010:2.0:1788880172.729311:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000000ec89a82. 00000800:00000200:0.0:1788880172.729312:0:201273:0:(socklnd.c:1182:ksocknal_create_conn()) New conn 12345-192.168.204.111@tcp p 3.x 192.168.204.11 -> 192.168.204.111:988 incarnation:1788880146239320504 sched[0] 00000100:00000200:2.0:1788880172.729317:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000800:00000010:0.0:1788880172.729322:0:201273:0:(socklnd.c:1190:ksocknal_create_conn()) kfreed 'hello': 144 at 0000000026efb8e8. 00000400:00000010:2.0:1788880172.729323:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f9754008. 00000400:00000010:2.0:1788880172.729329:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000081a23317. 00000100:00000200:2.0:1788880172.729335:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729338:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000d1c60633. 00000400:00000010:2.0:1788880172.729348:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000ecfd1062. 00000100:00000200:2.0:1788880172.729352:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729355:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000002c6175bf. 00000400:00000010:2.0:1788880172.729358:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000cc990595. 00000100:00000200:2.0:1788880172.729360:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729365:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a747926a. 00000400:00000010:2.0:1788880172.729367:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000b953b80d. 00000100:00000200:2.0:1788880172.729370:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729373:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b97a250d. 00000400:00000010:2.0:1788880172.729377:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000721c880a. 00000100:00000200:2.0:1788880172.729382:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729384:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000061fb1f1d. 00000400:00000010:2.0:1788880172.729386:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000005b352c43. 00000100:00000200:2.0:1788880172.729388:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729390:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000d6e055d7. 00000400:00000010:2.0:1788880172.729394:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000007db0a9ce. 00000100:00000200:2.0:1788880172.729398:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729402:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000e629f3f. 00000400:00000010:2.0:1788880172.729405:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000269fe32d. 00000400:00000200:0.0:1788880172.729412:0:201273:0:(lib-socket.c:438:lnet_sock_connect()) Error -99 connecting 1023 -> 192.168.204.111:988 00000100:00000200:2.0:1788880172.729413:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729419:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000d3dae3b2. 00000400:00000010:2.0:1788880172.729425:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000229f137c. 00000100:00000200:2.0:1788880172.729428:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729431:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004beeea46. 00000400:00000010:2.0:1788880172.729433:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000f49a4c25. 00000100:00000200:2.0:1788880172.729436:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729438:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000058834eb2. 00000400:00000010:2.0:1788880172.729441:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000004a3adb08. 00000100:00000200:2.0:1788880172.729444:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729446:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000099982909. 00000400:00000010:2.0:1788880172.729448:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000015d871f5. 00000100:00000200:2.0:1788880172.729452:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729454:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000003ea90c25. 00000400:00000010:2.0:1788880172.729457:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000fe7ff028. 00000100:00000200:2.0:1788880172.729459:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729468:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000057021538. 00000400:00000200:0.0:1788880172.729471:0:201273:0:(lib-socket.c:438:lnet_sock_connect()) Error -99 connecting 1022 -> 192.168.204.111:988 00000400:00000010:2.0:1788880172.729473:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000053d2b78e. 00000100:00000200:2.0:1788880172.729480:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729482:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a80494a7. 00000400:00000010:2.0:1788880172.729484:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000034ad216. 00000100:00000200:2.0:1788880172.729486:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729488:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006bda2041. 00000400:00000010:2.0:1788880172.729491:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000000a317ea1. 00000100:00000200:2.0:1788880172.729495:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729497:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ad52c5b7. 00000400:00000010:2.0:1788880172.729501:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 000000000ad7bc79. 00000100:00000200:2.0:1788880172.729502:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729505:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f2db28df. 00000400:00000010:2.0:1788880172.729512:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000a1f09744. 00000100:00000200:2.0:1788880172.729527:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729532:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000319c58d9. 00000400:00000010:2.0:1788880172.729536:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000d2b4084c. 00000100:00000200:2.0:1788880172.729538:0:203395:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_canceld: registering portal 17 00000400:00000010:2.0:1788880172.729541:0:203395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000867fc71d. 00000400:00000010:2.0:1788880172.729543:0:203395:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 0000000072fb6879. 00000100:00000010:2.0:1788880172.729547:0:203395:0:(service.c:3073:ptlrpc_main()) kmalloced '(rs)': 2048 at 00000000d0d9c67e. 00000100:00000200:2.0:1788880172.729563:0:203395:0:(service.c:3107:ptlrpc_main()) service thread 0 (#1) started 00000800:00000001:0.0:1788880172.729891:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.729902:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.729904:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.729908:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.729913:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : ACK - for me 00000400:00000200:0.0:1788880172.729926:0:201280:0:(lib-move.c:4504:lnet_parse_ack()) 192.168.204.11@tcp: ACK from 12345-192.168.204.111@tcp into md 0x11 00000400:00000200:0.0:1788880172.729931:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.729934:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: ACK: OK 00000400:00000200:0.0:1788880172.729941:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000400:00000010:0.0:1788880172.729943:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000400:00000200:0.0:1788880172.729946:0:201280:0:(peer.c:3204:lnet_discovery_event_handler()) Received event: 4 00000400:00000200:0.0:1788880172.729958:0:201280:0:(peer.c:2922:lnet_discovery_event_ack()) peer 192.168.204.111@tcp ev->status 0 00000400:00000200:0.0:1788880172.729962:0:201280:0:(peer.c:2528:lnet_peer_queue_for_discovery()) Queue peer 192.168.204.111@tcp: -114 00000800:00000001:0.0:1788880172.729973:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.729976:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.730016:0:203396:0:(service.c:3019:ptlrpc_main()) Process entered 00000100:00000010:2.0:1788880172.730040:0:203396:0:(service.c:3046:ptlrpc_main()) kmalloced '(env)': 56 at 000000003b52b145. 00000020:00000010:2.0:1788880172.730044:0:203396:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 00000000902d0aab. 00000020:00000010:2.0:1788880172.730052:0:203396:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000f94e2dcc. 00000020:00000010:2.0:1788880172.730055:0:203396:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000c96723f8. 00000020:00000010:2.0:1788880172.730059:0:203396:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 000000008783601d. 00000040:00000010:2.0:1788880172.730062:0:203396:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 000000005cd9ea2d. 00000020:00000010:2.0:1788880172.730066:0:203396:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 0000000002902e42. 00000020:00000010:2.0:1788880172.730071:0:203396:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 00000000e5963ff5. 00000020:00000010:2.0:1788880172.730075:0:203396:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 000000005df3e44f. 80000000:00000010:2.0:1788880172.730078:0:203396:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000b2681c2e. 40000000:00000010:2.0:1788880172.730082:0:203396:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 0000000019a908ab. 00000800:00000010:0.0:1788880172.730085:0:201273:0:(socklnd.c:886:ksocknal_create_conn()) alloc '(conn)': 4720 at 00000000373749d4. 00000008:00000010:2.0:1788880172.730087:0:203396:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 0000000033ac9de9. 00000800:00000010:0.0:1788880172.730092:0:201273:0:(socklnd.c:913:ksocknal_create_conn()) alloc '(hello)': 144 at 0000000026efb8e8. 00000800:00000010:0.0:1788880172.730097:0:201273:0:(socklnd_proto.c:573:ksocknal_send_hello_v2()) alloc '(hello4)': 56 at 000000008c244d33. 00020000:00000010:2.0:1788880172.730108:0:203396:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000e32c1431. 00000800:00000010:0.0:1788880172.730110:0:201273:0:(socklnd_proto.c:613:ksocknal_send_hello_v2()) kfreed 'hello4': 56 at 000000008c244d33. 00008000:00000010:2.0:1788880172.730115:0:203396:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000e8f441a3. 00100000:00000010:2.0:1788880172.730122:0:203396:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 00000000168fd172. 00040000:00000010:2.0:1788880172.730125:0:203396:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 0000000003fdebdb. 00040000:00000010:2.0:1788880172.730131:0:203396:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 000000006258d051. 00000400:00000200:3.0:1788880172.730133:0:201283:0:(peer.c:4145:lnet_peer_discovery_wait_for_work()) woken: 0 00040000:00000010:2.0:1788880172.730136:0:203396:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000cfafb0e2. 00000400:00000200:3.0:1788880172.730142:0:201283:0:(peer.c:4257:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x100141 00000004:00000010:2.0:1788880172.730143:0:203396:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 00000000d6156ba6. 00000004:00000010:2.0:1788880172.730154:0:203396:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 00000000a3619df8. 00000400:00000200:3.0:1788880172.730154:0:201283:0:(peer.c:3997:lnet_peer_discovered()) peer 192.168.204.111@tcp 00000004:00000010:2.0:1788880172.730161:0:203396:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 00000000362fcbef. 00002000:00000010:2.0:1788880172.730165:0:203396:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 0000000013942555. 00000004:00000010:2.0:1788880172.730168:0:203396:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 000000004649ef58. 00000080:00000010:2.0:1788880172.730172:0:203396:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 000000002535e9bd. 00000080:00000010:2.0:1788880172.730176:0:203396:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 00000000bda6b294. 00000100:00000010:2.0:1788880172.730184:0:203396:0:(service.c:3073:ptlrpc_main()) kmalloced '(rs)': 2048 at 00000000bac05b68. 00000400:00000200:3.0:1788880172.730203:0:201283:0:(peer.c:4275:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x100111 rc 0 00000400:00000200:3.0:1788880172.730213:0:201283:0:(peer.c:2544:lnet_peer_discovery_complete()) Discovery complete. Dequeue peer 192.168.204.111@tcp 00000100:00000200:2.0:1788880172.730215:0:203396:0:(service.c:3107:ptlrpc_main()) service thread 1 (#2) started 00000800:00000001:2.0:1788880172.730497:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000200:3.0:1788880172.730528:0:201283:0:(peer.c:4145:lnet_peer_discovery_wait_for_work()) woken: 0 00000400:00000010:3.0:1788880172.730537:0:201283:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bbafd6b5. 00000400:00000010:3.0:1788880172.730548:0:201283:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000d4ec363. 00000800:00000001:2.0:1788880172.730552:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.730617:0:203392:0:(service.c:3549:ptlrpc_start_thread()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.730623:0:203392:0:(service.c:3454:ptlrpc_start_thread()) Process entered 00000100:00100000:0.0:1788880172.730625:0:203392:0:(service.c:3462:ptlrpc_start_thread()) ldlm_canceld[0] started 2 min 3 max 56 00000100:00000010:0.0:1788880172.730632:0:203392:0:(service.c:3473:ptlrpc_start_thread()) kmalloced '(thread)': 328 at 000000009e508178. 00000400:00000200:3.0:1788880172.730634:0:201283:0:(api-ni.c:2326:lnet_push_target_post()) posted push target 0000000072ffd099 00000100:00100000:0.0:1788880172.730638:0:203392:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cn00_002' 00000400:00000200:3.0:1788880172.730643:0:201283:0:(peer.c:4257:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x1000d1 00000400:00000010:3.0:1788880172.730651:0:201283:0:(peer.c:3389:lnet_peer_merge_data()) alloc '(curnis)': 40 at 00000000022fb4a4. 00000400:00000010:3.0:1788880172.730654:0:201283:0:(peer.c:3390:lnet_peer_merge_data()) alloc '(addnis)': 48 at 000000002785155f. 00000400:00000010:3.0:1788880172.730655:0:201283:0:(peer.c:3391:lnet_peer_merge_data()) alloc '(delnis)': 40 at 00000000b1db6e49. 00000400:00000010:3.0:1788880172.730659:0:201283:0:(peer.c:3538:lnet_peer_merge_data()) kfreed 'curnis': 40 at 00000000022fb4a4. 00000400:00000010:3.0:1788880172.730661:0:201283:0:(peer.c:3539:lnet_peer_merge_data()) kfreed 'addnis': 48 at 000000002785155f. 00000400:00000010:3.0:1788880172.730662:0:201283:0:(peer.c:3540:lnet_peer_merge_data()) kfreed 'delnis': 40 at 00000000b1db6e49. 00000400:00000010:3.0:1788880172.730664:0:201283:0:(api-ni.c:1835:lnet_ping_buffer_free()) kfreed 'pbuf': 57 at 000000003d0b93e5. 00000400:00000200:3.0:1788880172.730667:0:201283:0:(peer.c:3543:lnet_peer_merge_data()) peer 192.168.204.111@tcp (00000000b0c52047): 0 00000400:00000200:3.0:1788880172.730670:0:201283:0:(peer.c:3855:lnet_peer_data_present()) peer 192.168.204.111@tcp(00000000b0c52047): 0. state = 0x100151 00000400:00000200:3.0:1788880172.730674:0:201283:0:(peer.c:4275:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x100151 rc 1 00000400:00000200:3.0:1788880172.730677:0:201283:0:(peer.c:4257:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x100151 00000400:00000200:3.0:1788880172.730679:0:201283:0:(peer.c:3997:lnet_peer_discovered()) peer 192.168.204.111@tcp 00000400:00000200:3.0:1788880172.730681:0:201283:0:(peer.c:4275:lnet_peer_discovery()) peer 192.168.204.111@tcp(00000000b0c52047) state 0x100111 rc 0 00000400:00000200:3.0:1788880172.730683:0:201283:0:(peer.c:2544:lnet_peer_discovery_complete()) Discovery complete. Dequeue peer 192.168.204.111@tcp 00000100:00000001:3.0:1788880172.730829:0:203397:0:(service.c:3019:ptlrpc_main()) Process entered 00000100:00000010:3.0:1788880172.730844:0:203397:0:(service.c:3046:ptlrpc_main()) kmalloced '(env)': 56 at 00000000b1db6e49. 00000020:00000010:3.0:1788880172.730900:0:203397:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 000000002785155f. 00000020:00000010:3.0:1788880172.730906:0:203397:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000360b9725. 00000020:00000010:3.0:1788880172.730910:0:203397:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000cfd9babf. 00000020:00000010:3.0:1788880172.730914:0:203397:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 000000002d9d5656. 00000040:00000010:3.0:1788880172.730918:0:203397:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 0000000077b4e854. 00000020:00000010:3.0:1788880172.730923:0:203397:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000d25ab788. 00000020:00000010:3.0:1788880172.730928:0:203397:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 000000006aaee213. 00000020:00000010:3.0:1788880172.730996:0:203397:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000c0315058. 80000000:00000010:3.0:1788880172.731001:0:203397:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000bd53c573. 40000000:00000010:3.0:1788880172.731004:0:203397:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 00000000022fb4a4. 00000008:00000010:3.0:1788880172.731020:0:203397:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000a2b6e8bd. 00020000:00000010:3.0:1788880172.731025:0:203397:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 000000001a971abc. 00008000:00000010:3.0:1788880172.731100:0:203397:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 000000004489670e. 00100000:00000010:3.0:1788880172.731108:0:203397:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 0000000068742ac5. 00040000:00000010:3.0:1788880172.731112:0:203397:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 00000000336bf630. 00040000:00000010:3.0:1788880172.731121:0:203397:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 00000000b5f5e930. 00040000:00000010:3.0:1788880172.731125:0:203397:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000759796f2. 00000004:00000010:3.0:1788880172.731129:0:203397:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 00000000e9a70a42. 00000004:00000010:3.0:1788880172.731135:0:203397:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 000000001e6f6ed6. 00000004:00000010:3.0:1788880172.731139:0:203397:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 000000000346f302. 00002000:00000010:3.0:1788880172.731143:0:203397:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 00000000b311c606. 00000004:00000010:3.0:1788880172.731147:0:203397:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 00000000dd97b170. 00000080:00000010:3.0:1788880172.731159:0:203397:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 00000000ab363ee9. 00000080:00000010:3.0:1788880172.731171:0:203397:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 00000000a63852eb. 00000100:00000010:3.0:1788880172.731175:0:203397:0:(service.c:3073:ptlrpc_main()) kmalloced '(rs)': 2048 at 000000009355e9df. 00000100:00000200:3.0:1788880172.731196:0:203397:0:(service.c:3107:ptlrpc_main()) service thread 2 (#3) started 00000800:00000200:0.1:1788880172.731206:0:201273:0:(socklnd.c:384:ksocknal_incr_conn_count()) Add conn type 3, ksnr_connected e ksnr_max_conns 1 00000800:00000200:0.0:1788880172.731217:0:201273:0:(socklnd.c:1182:ksocknal_create_conn()) New conn 12345-192.168.204.111@tcp p 3.x 192.168.204.11 -> 192.168.204.111:988 incarnation:1788880146239320504 sched[0] 00000800:00000010:0.0:1788880172.731224:0:201273:0:(socklnd.c:1190:ksocknal_create_conn()) kfreed 'hello': 144 at 0000000026efb8e8. 00000800:00000001:2.0:1788880172.731326:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880172.731332:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880172.731432:0:203398:0:(ldlm_lockd.c:2965:ldlm_bl_thread_main()) Process entered 00010000:00000010:0.0:1788880172.731436:0:203398:0:(ldlm_lockd.c:2967:ldlm_bl_thread_main()) kmalloced '(env)': 56 at 000000008c244d33. 00000020:00000010:0.0:1788880172.731440:0:203398:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000921c77b7. 00000020:00000010:0.0:1788880172.731443:0:203398:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000880b1a3b. 00000020:00000010:0.0:1788880172.731447:0:203398:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000142b8dba. 00000020:00000010:0.0:1788880172.731450:0:203398:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 0000000054d5c245. 00000020:00000010:0.0:1788880172.731453:0:203398:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 000000008f6d93f0. 80000000:00000010:0.0:1788880172.731456:0:203398:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000ce7a5a27. 40000000:00000010:0.0:1788880172.731467:0:203398:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 00000000ef109210. 00100000:00000010:0.0:1788880172.731471:0:203398:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 000000001174eb2d. 00040000:00000010:0.0:1788880172.731474:0:203398:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 000000009d735d66. 00040000:00000010:0.0:1788880172.731477:0:203398:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000f28b442f. 00002000:00000010:0.0:1788880172.731481:0:203398:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 000000005476f2eb. 00000020:00000010:0.0:1788880172.731484:0:203398:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 0000000081c54058. 00000020:00000001:0.0:1788880172.731494:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880172.731496:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880172.731498:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788880172.731500:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880172.731501:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880172.731502:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788880172.731504:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880172.731505:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880172.731507:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1788880172.731867:0:203400:0:(ldlm_lockd.c:159:expired_lock_main()) Process entered 00000020:00000010:0.0:1788880172.731872:0:203400:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000cc1cf973. 00000020:00000010:0.0:1788880172.731875:0:203400:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000e697e293. 00000040:00000010:0.0:1788880172.731878:0:203400:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000b53d861b. 00000020:00000010:0.0:1788880172.731880:0:203400:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000db8a84a0. 00000020:00000010:0.0:1788880172.731884:0:203400:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 00000000c5c58981. 00000020:00000010:0.0:1788880172.731886:0:203400:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 000000005a5db921. 80000000:00000010:0.0:1788880172.731888:0:203400:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 000000008c6dadec. 40000000:00000010:0.0:1788880172.731890:0:203400:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 00000000cad9d805. 00100000:00000010:0.0:1788880172.731894:0:203400:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 000000001484a6be. 00040000:00000010:0.0:1788880172.731896:0:203400:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 0000000001e51cbd. 00040000:00000010:0.0:1788880172.731900:0:203400:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 00000000d68a96b3. 00040000:00000010:0.0:1788880172.731902:0:203400:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 000000002bcb65b0. 00000004:00000010:0.0:1788880172.731905:0:203400:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 000000002caa6d03. 00000004:00000010:0.0:1788880172.731910:0:203400:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 00000000a48bfaca. 00000004:00000010:0.0:1788880172.731914:0:203400:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 00000000d5ec3418. 00002000:00000010:0.0:1788880172.731917:0:203400:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 0000000070a5d129. 00000004:00000010:0.0:1788880172.731920:0:203400:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 000000004b1366d5. 00000020:00000010:0.0:1788880172.731922:0:203400:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 00000000b01aa9a0. 00010000:00000001:0.0:1788880172.731937:0:203400:0:(ldlm_lockd.c:142:have_expired_locks()) Process entered 00010000:00000001:0.0:1788880172.731939:0:203400:0:(ldlm_lockd.c:147:have_expired_locks()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.731942:0:203400:0:(ldlm_lockd.c:142:have_expired_locks()) Process entered 00010000:00000001:0.0:1788880172.731943:0:203400:0:(ldlm_lockd.c:147:have_expired_locks()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880172.732600:0:203392:0:(mgc_request.c:896:mgc_device_alloc()) Process leaving (rc=18446617250279280704 : -126823430270912 : ffff8ca79e7d5040) 00000020:00000010:2.0:1788880172.732609:0:203392:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000fa0f7569. 00000040:00000010:2.0:1788880172.732614:0:203392:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 0000000026532f47. 00000020:00000010:2.0:1788880172.732619:0:203392:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 00000000c85a7d49. 00000020:00000010:2.0:1788880172.732623:0:203392:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 0000000058b4ee38. 00040000:00000010:2.0:1788880172.732627:0:203392:0:(lquota_lib.c:35:lquota_key_fini()) kfreed 'info': 248 at 00000000b899fe8d. 00040000:00000010:2.0:1788880172.732633:0:203392:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 000000002974b115. 00000020:00000010:2.0:1788880172.732637:0:203392:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 000000002b843fc3. 00000020:00000010:2.0:1788880172.732642:0:203392:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 00000000989c86ee. 00000020:00000010:2.0:1788880172.732647:0:203392:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 00000000fabadd93. 00000008:00000010:2.0:1788880172.732652:0:203392:0:(osc_dev.c:125:osc_session_fini()) slab-freed '(info)': 352 at 00000000ea38ec9a. 00020000:00000010:2.0:1788880172.732656:0:203392:0:(lov_dev.c:136:lov_session_key_fini()) slab-freed '(info)': 584 at 00000000cb07dfd8. 00008000:00000010:2.0:1788880172.732660:0:203392:0:(echo_client.c:449:echo_session_key_fini()) slab-freed '(session)': 8 at 00000000570fe7be. 00000080:00000010:2.0:1788880172.732675:0:203392:0:(vvp_dev.c:105:vvp_session_key_fini()) slab-freed '(session)': 240 at 00000000347372f1. 00000020:00000010:2.0:1788880172.732679:0:203392:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000c19d180e. 00000020:00000001:2.0:1788880172.732690:0:203392:0:(obd_class.h:601:obd_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880172.732694:0:203392:0:(obd_config.c:994:class_incref()) incref MGC192.168.204.111@tcp (000000002c7588f3) now 4 - setup 00000020:00000080:2.0:1788880172.732697:0:203392:0:(obd_config.c:826:class_setup()) finished setup of obd MGC192.168.204.111@tcp (uuid fa4856d5-51a8-499e-b3f8-b477743a3dec) 00000020:00000001:2.0:1788880172.732699:0:203392:0:(obd_config.c:828:class_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880172.732701:0:203392:0:(obd_mount.c:257:lustre_start_simple()) kfreed 'lcfg': 104 at 00000000ee2162e1. 00000020:00000001:2.0:1788880172.732705:0:203392:0:(obd_mount.c:263:lustre_start_simple()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.732711:0:203392:0:(obd_class.h:536:obd_set_info_async()) Process entered 10000000:00000001:2.0:1788880172.732713:0:203392:0:(mgc_request.c:1055:mgc_set_info_async()) Process entered 10000000:00000001:2.0:1788880172.732716:0:203392:0:(mgc_request.c:1127:mgc_set_info_async()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.732717:0:203392:0:(obd_class.h:553:obd_set_info_async()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.732719:0:203392:0:(obd_class.h:860:obd_connect()) Process entered 00010000:00000001:2.0:1788880172.732720:0:203392:0:(ldlm_lib.c:668:client_connect_import()) Process entered 00000020:00000001:2.0:1788880172.732722:0:203392:0:(genops.c:1396:class_connect()) Process entered 00000020:00000001:2.0:1788880172.732724:0:203392:0:(genops.c:952:__class_new_export()) Process entered 00000020:00000010:2.0:1788880172.732727:0:203392:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 00000000ea058c72. 00000020:00000001:2.0:1788880172.732729:0:203392:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880172.732731:0:203392:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000ea058c72 with handle 0x805d548a36a88651 to hash 00000020:00000001:2.0:1788880172.732733:0:203392:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:2.0:1788880172.732735:0:203392:0:(obd_class.h:1016:obd_init_export()) Process entered 00000020:00000001:2.0:1788880172.732737:0:203392:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880172.732740:0:203392:0:(obd_config.c:994:class_incref()) incref MGC192.168.204.111@tcp (000000002c7588f3) now 5 - export 00000020:00000001:2.0:1788880172.732743:0:203392:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000020:00000040:2.0:1788880172.732746:0:203392:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 1 00000020:00000080:2.0:1788880172.732748:0:203392:0:(genops.c:1406:class_connect()) connect: client fa4856d5-51a8-499e-b3f8-b477743a3dec, cookie 0x805d548a36a88651 00000020:00000001:2.0:1788880172.732751:0:203392:0:(genops.c:1407:class_connect()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.732753:0:203392:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880172.732755:0:203392:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000001:2.0:1788880172.732766:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880172.732768:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=2 00000020:00000001:2.0:1788880172.732770:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000020:00000001:2.0:1788880172.732773:0:203392:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000100:00000001:2.0:1788880172.732777:0:203392:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00000001:2.0:1788880172.732781:0:203392:0:(import.c:485:import_select_connection()) Process entered 00000100:00080000:2.0:1788880172.732786:0:203392:0:(import.c:510:import_select_connection()) MGC192.168.204.111@tcp: connect to NID 192.168.204.111@tcp last attempt 0 00000400:00000200:2.0:1788880172.732797:0:203392:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000100:00000001:2.0:1788880172.732800:0:203392:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880172.732802:0:203392:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 2 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880172.732804:0:203392:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000001:2.0:1788880172.732806:0:203392:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880172.732806:0:203392:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000001:2.0:1788880172.732808:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880172.732809:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=3 00000020:00000001:2.0:1788880172.732810:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000020:00000001:2.0:1788880172.732811:0:203392:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000100:00000001:2.0:1788880172.732812:0:203392:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880172.732812:0:203392:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 3 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880172.732813:0:203392:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000040:2.0:1788880172.732814:0:203392:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 2 00000100:00080000:2.0:1788880172.732817:0:203392:0:(import.c:624:import_select_connection()) MGC192.168.204.111@tcp: import 000000003ccc7c76 using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000100:00000001:2.0:1788880172.732818:0:203392:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880172.732820:0:203392:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:2.0:1788880172.732822:0:203392:0:(sec.c:1453:sptlrpc_sec_create()) Process entered 02000000:08000000:2.0:1788880172.732823:0:203392:0:(sec.c:1471:sptlrpc_sec_create()) mgc MGC192.168.204.111@tcp: select security flavor null 02000000:00000001:2.0:1788880172.732826:0:203392:0:(sec.c:1492:sptlrpc_sec_create()) Process leaving (rc=18446744072650609120 : -1058942496 : ffffffffc0e1d1e0) 02000000:00000001:2.0:1788880172.732828:0:203392:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.732829:0:203392:0:(obd_class.h:889:obd_reconnect()) Process entered 00000100:00000001:2.0:1788880172.732831:0:203392:0:(obd_class.h:896:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880172.732836:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000020:00000040:2.0:1788880172.732839:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 02000000:00000001:2.0:1788880172.732843:0:203392:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 02000000:00000001:2.0:1788880172.732845:0:203392:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.732848:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880172.732851:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880172.732854:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880172.732863:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 0000000077b5e929. 00000100:00000001:2.0:1788880172.732872:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880172.732881:0:203392:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fd6b7ee1 x1875776780632192/t0(0) o250->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880172.732897:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880172.732899:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880172.732904:0:203392:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fd6b7ee1] to pc [ptlrpcd_rcv+-1] req@00000000fd6b7ee1 x1875776780632192/t0(0) o250->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.732921:0:203392:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.732924:0:203392:0:(pinger.c:367:ptlrpc_pinger_add_import()) Process entered 00000100:00080000:2.0:1788880172.732928:0:203392:0:(pinger.c:373:ptlrpc_pinger_add_import()) adding pingable import fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS 00000020:00000040:2.0:1788880172.732930:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880172.732959:0:203392:0:(pinger.c:384:ptlrpc_pinger_add_import()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880172.732963:0:203392:0:(ldlm_lib.c:725:client_connect_import()) Process leaving 00000020:00000001:2.0:1788880172.732968:0:203392:0:(obd_class.h:876:obd_connect()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880172.732972:0:203392:0:(obd_mount.c:547:lustre_start_mgc()) kfreed 'uuid': 40 at 0000000019f65800. 00000020:00000010:2.0:1788880172.732977:0:203392:0:(obd_mount.c:548:lustre_start_mgc()) kfreed 'data': 192 at 00000000698cd534. 00000020:00000010:2.0:1788880172.732982:0:203392:0:(obd_mount.c:549:lustre_start_mgc()) kfreed 'mgcname': 23 at 0000000093570a07. 00000020:00000001:2.0:1788880172.732992:0:203392:0:(obd_mount.c:551:lustre_start_mgc()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880172.732993:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 25 (11161) 00000080:00000001:2.0:1788880172.732998:0:203392:0:(llite_lib.c:1512:ll_fill_super()) Process entered 00000080:00200000:2.0:1788880172.733001:0:203392:0:(llite_lib.c:1515:ll_fill_super()) VFS Op: cfg_instance lustre-client-ffff8ca787527000 (sb 0000000008cb040b) 00000080:00000010:2.0:1788880172.733007:0:203392:0:(llite_lib.c:1519:ll_fill_super()) kmalloced '(cfg)': 88 at 00000000f51165b6. 00000080:00000001:2.0:1788880172.733012:0:203392:0:(llite_lib.c:95:ll_init_sbi()) Process entered 00000080:00000010:2.0:1788880172.733017:0:203392:0:(llite_lib.c:97:ll_init_sbi()) kmalloced '(sbi)': 1400 at 00000000f31cb6b5. 00000020:00000001:2.0:1788880172.733094:0:203392:0:(cl_page.c:1299:cl_cache_init()) Process entered 00000020:00000010:2.0:1788880172.733115:0:203392:0:(cl_page.c:1300:cl_cache_init()) kmalloced '(cache)': 152 at 000000007a20bb84. 00000020:00000001:2.0:1788880172.733119:0:203392:0:(cl_page.c:1316:cl_cache_init()) Process leaving (rc=18446617250282258176 : -126823427293440 : ffff8ca79eaabf00) 00000080:00000010:2.0:1788880172.733123:0:203392:0:(llite_lib.c:131:ll_init_sbi()) kmalloced '(sbi->ll_foreign_symlink_prefix)': 6 at 00000000a5959f87. 00000080:00000010:2.0:1788880172.733128:0:203392:0:(llite_lib.c:138:ll_init_sbi()) kmalloced '(sbi->ll_foreign_symlink_upcall)': 5 at 00000000c9155929. 00000080:00000001:2.0:1788880172.733136:0:203392:0:(llite_lib.c:242:ll_init_sbi()) Process leaving (rc=18446617250083401728 : -126823626149888 : ffff8ca792d07000) 00000080:00000001:2.0:1788880172.733139:0:203392:0:(llite_lib.c:1231:ll_options()) Process entered 00000080:01000000:2.0:1788880172.733142:0:203392:0:(llite_lib.c:1241:ll_options()) Parsing opts user_xattr,flock 00000080:00000004:2.0:1788880172.733145:0:203392:0:(llite_lib.c:1251:ll_options()) next opt=user_xattr 00000080:00000004:2.0:1788880172.733149:0:203392:0:(llite_lib.c:1251:ll_options()) next opt=flock 00000080:00000001:2.0:1788880172.733152:0:203392:0:(llite_lib.c:1420:ll_options()) Process leaving (rc=0 : 0 : 0) 00000080:01000000:2.0:1788880172.733158:0:203392:0:(llite_lib.c:1549:ll_fill_super()) llite sb uuid: ea27006f-52a8-4007-b69a-dcac8279445a 02000000:00000001:2.0:1788880172.733170:0:203392:0:(gss_keyring.c:828:gss_rename_sk_key()) Process entered 02000000:00000001:2.0:1788880172.733173:0:203392:0:(gss_keyring.c:832:gss_rename_sk_key()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880172.733359:0:203392:0:(lproc_llite.c:2839:ll_debugfs_register_super()) Process entered 00000020:00000001:0.0:1788880172.733582:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880172.733584:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880172.733586:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000010:2.0:1788880172.733703:0:203392:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 00000000a5ba2225. 00000020:00000010:2.0:1788880172.733708:0:203392:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 1472 at 000000004370ef91. 00000020:00000010:2.0:1788880172.733724:0:203392:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 000000007319f941. 00000020:00000010:2.0:1788880172.733727:0:203392:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 576 at 000000009d96371d. 00000080:00000001:2.0:1788880172.733740:0:203392:0:(lproc_llite.c:2909:ll_debugfs_register_super()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.733744:0:203392:0:(obd_mount.c:61:lustre_process_log()) Process entered 00000020:00000010:2.0:1788880172.733747:0:203392:0:(obd_mount.c:66:lustre_process_log()) kmalloced '(bufs)': 104 at 000000001ca1494e. 00000020:00000010:2.0:1788880172.733751:0:203392:0:(obd_mount.c:75:lustre_process_log()) kmalloced '(lcfg)': 184 at 00000000617a5f88. 00000020:00000001:2.0:1788880172.733754:0:203392:0:(obd_class.h:689:obd_process_config()) Process entered 00000020:00000010:2.0:1788880172.733756:0:203392:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000058b4ee38. 00000020:00000010:2.0:1788880172.733761:0:203392:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 0000000026532f47. 00000040:00000010:2.0:1788880172.733766:0:203392:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000fa0f7569. 00000020:00000010:2.0:1788880172.733769:0:203392:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 0000000068effccb. 00000020:00000010:2.0:1788880172.733773:0:203392:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000c0495ef7. 00040000:00000010:2.0:1788880172.733779:0:203392:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 00000000d0fdfcdc. 00040000:00000010:2.0:1788880172.733784:0:203392:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 0000000078217d0d. 10000000:00000001:2.0:1788880172.733787:0:203392:0:(mgc_request.c:1922:mgc_process_config()) Process entered 10000000:00000001:2.0:1788880172.733789:0:203392:0:(mgc_request_server.c:675:mgc_process_config_server()) Process entered 10000000:01000000:2.0:1788880172.733792:0:203392:0:(mgc_request.c:1943:mgc_process_config()) parse_log lustre-client from 0 10000000:00000001:2.0:1788880172.733794:0:203392:0:(mgc_request.c:326:config_log_add()) Process entered 10000000:01000000:2.0:1788880172.733796:0:203392:0:(mgc_request.c:329:config_log_add()) add config log lustre-client-ffff8ca787527000 10000000:00000001:2.0:1788880172.733798:0:203392:0:(mgc_request.c:159:config_log_find()) Process entered 10000000:00000001:2.0:1788880172.733800:0:203392:0:(mgc_request.c:177:config_log_find()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880172.733801:0:203392:0:(mgc_request.c:191:do_config_log_add()) Process entered 10000000:01000000:2.0:1788880172.733803:0:203392:0:(mgc_request.c:194:do_config_log_add()) do adding config log lustre-sptlrpc-ffff8ca792d4cd40 10000000:00000010:2.0:1788880172.733807:0:203392:0:(mgc_request.c:197:do_config_log_add()) kmalloced '(cld)': 287 at 000000007b8fd88a. 10000000:01000000:2.0:1788880172.733810:0:203392:0:(mgc_request.c:68:mgc_name2resid()) log lustre-sptlrpc to resid 0x65727473756c/0x0 (lustre) 00000020:00000040:2.0:1788880172.733813:0:203392:0:(genops.c:889:class_export_get()) GET export 0000000040f424e8 refcount=3 02000000:00000010:2.0:1788880172.733819:0:203392:0:(sec_config.c:546:sptlrpc_conf_get()) kmalloced '(conf)': 120 at 00000000567e245c. 02000000:08000000:2.0:1788880172.733822:0:203392:0:(sec_config.c:557:sptlrpc_conf_get()) create sptlrpc conf lustre 10000000:00000001:2.0:1788880172.733824:0:203392:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:2.0:1788880172.733832:0:203392:0:(mgc_request.c:1801:mgc_process_log()) Process log lustre-sptlrpc-ffff8ca792d4cd40 from 1 10000000:00000001:2.0:1788880172.733835:0:203392:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:2.0:1788880172.733836:0:203392:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00000010:2.0:1788880172.733841:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000020:00000040:2.0:1788880172.733844:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880172.733848:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880172.733849:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880172.733852:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880172.733858:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000bd87bd8d. 00000100:00000001:2.0:1788880172.733861:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880172.733865:0:203392:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:2.0:1788880172.733867:0:203392:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:2.0:1788880172.733875:0:203392:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at 000000005bab1010. 00010000:00000001:2.0:1788880172.733879:0:203392:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:2.0:1788880172.733883:0:203392:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 0000000081b300c8. 00000020:00000010:2.0:1788880172.733886:0:203392:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 128 at 00000000aa8e4e46. 00000020:00000001:2.0:1788880172.733888:0:203392:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880172.733890:0:203392:0:(lustre_handles.c:93:class_handle_hash()) added object 0000000081b300c8 with handle 0x805d548a36a88658 to hash 00000020:00000001:2.0:1788880172.733891:0:203392:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:2.0:1788880172.733893:0:203392:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617249862489088 : -126823847062528 : ffff8ca785a59400) 00010000:00000001:2.0:1788880172.733896:0:203392:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617249862489088 : -126823847062528 : ffff8ca785a59400) 00010000:00000001:2.0:1788880172.733898:0:203392:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:2.0:1788880172.733901:0:203392:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880172.733903:0:203392:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788880172.733910:0:203392:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788880172.733918:0:203392:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:2.0:1788880172.733922:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:2.0:1788880172.733923:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:2.0:1788880172.733931:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000027960f68. 00000100:00000001:2.0:1788880172.733932:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250282254720 : -126823427296896 : ffff8ca79eaab180) 00000100:00000001:2.0:1788880172.733934:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880172.733934:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 00000100:00000001:2.0:1788880172.733936:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880172.733937:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.733939:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:2.0:1788880172.733940:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880172.733943:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.733948:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880172.733949:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00080000:2.0:1788880172.733951:0:203392:0:(client.c:1832:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.733955:0:203392:0:(client.c:1837:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.733956:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880172.733957:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880172.733958:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 0000000027960f68 going to sleep for 0 seconds 00000100:00000001:2.0:1788880172.733960:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880172.733961:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.733962:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880172.733963:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.733964:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880172.735409:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.735423:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.735426:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.735430:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.735437:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880172.735451:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 416 into portal 25 MB=0x6aa0250b00080 00000400:00000200:0.0:1788880172.735457:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 416/416 into md 0x119 [1] + 0 00000800:00000001:0.0:1788880172.735464:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.735472:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.735474:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.735478:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880172.735484:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.735490:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780632192/t0(0) o250->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 520/544 e 0 to 0 dl 1788880182 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880172.735510:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=416 offset=0 replen=544 req@00000000fd6b7ee1 x1875776780632192/t0(0) o250->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 520/544 e 0 to 0 dl 1788880182 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.735528:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880172.735542:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.735546:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.735553:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880172.735556:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880172.735560:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880172.735562:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880172.735563:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000963814d2. 00000400:00000200:2.0:1788880172.735568:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000400:00000010:2.0:1788880172.735569:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:2.0:1788880172.735571:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880172.735575:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000fd6b7ee1 x1875776780632192/t0(0) o250->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 520/544 e 0 to 0 dl 1788880182 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880172.735583:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000fd6b7ee1 x1875776780632192/t0(0) o250->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 520/544 e 0 to 0 dl 1788880182 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.735598:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880172.735599:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.735601:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880172.735603:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880172.735605:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880172.735606:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880172.735608:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880172.735613:0:201284:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=1788880172 v=5 (0 0 0 0) 00000100:00001000:2.0:1788880172.735615:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=1788880172 v=0 (0 0 0 0) 00000100:00001000:2.0:1788880172.735618:0:201284:0:(import.c:2215:obd_at_measure()) AT 000000003cbfdd57 change: old=0 new=5 delta=5 (val=1) hist 1 0 0 0 00000100:00000001:2.0:1788880172.735619:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880172.735620:0:201284:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880172.735622:0:201284:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880172.735624:0:201284:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.735626:0:201284:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00100000:2.0:1788880172.735627:0:201284:0:(client.c:3112:ptlrpc_free_committed()) MGC192.168.204.111@tcp: committing for last_committed 0 gen 1 00000100:00000040:2.0:1788880172.735630:0:201284:0:(client.c:3222:ptlrpc_free_committed()) MGC192.168.204.111@tcp: normal: skip=0 replay=0/0 committed=0/0 00000100:00000001:2.0:1788880172.735632:0:201284:0:(client.c:3223:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880172.735633:0:201284:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880172.735636:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000fd6b7ee1 x1875776780632192/t0(0) o250->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 520/416 e 0 to 0 dl 1788880182 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.735640:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880172.735642:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880172.735643:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.735644:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00080000:2.0:1788880172.735647:0:201284:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.204.111@tcp: connect to target with instance 0 00000020:00000001:2.0:1788880172.735649:0:201284:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880172.735650:0:201284:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000001:2.0:1788880172.735651:0:201284:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880172.735653:0:201284:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=3 00000020:00000001:2.0:1788880172.735655:0:201284:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000020:00000001:2.0:1788880172.735656:0:201284:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000100:00001000:2.0:1788880172.735658:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=1788880172 v=0 (0 0 0 0) 00000100:00001000:2.0:1788880172.735666:0:201284:0:(import.c:2215:obd_at_measure()) AT 000000003cbfdd57 change: old=0 new=5 delta=5 (val=1) hist 1 0 0 0 00000100:00080000:2.0:1788880172.735676:0:201284:0:(import.c:956:ptlrpc_connect_set_flags()) MGC192.168.204.111@tcp: Resetting ns_connect_flags to server flags: 0xa000011001002060 00000020:00000040:2.0:1788880172.735679:0:201284:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 2 00000100:00000001:2.0:1788880172.735681:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 10000000:01000000:2.0:1788880172.735684:0:201284:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 10000000:00000001:2.0:1788880172.735686:0:201284:0:(mgc_request.c:1201:mgc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.735691:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00080000:2.0:1788880172.735694:0:201284:0:(import.c:65:import_set_state_nolock()) 000000003ccc7c76 MGS: changing import state from CONNECTING to FULL 00000100:00000001:2.0:1788880172.735698:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 10000000:01000000:2.0:1788880172.735700:0:201284:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 10000000:00000040:2.0:1788880172.735701:0:201284:0:(mgc_request.c:1185:mgc_import_event()) MGC192.168.204.111@tcp: Reactivating import 00000100:00080000:2.0:1788880172.735717:0:201284:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 10000000:00000001:2.0:1788880172.735718:0:201284:0:(mgc_request.c:1201:mgc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.735719:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00000001:2.0:1788880172.735720:0:201284:0:(import.c:1241:ptlrpc_connect_interpret()) Process leaving via finish (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880172.735722:0:201284:0:(import.c:1648:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00080000:2.0:1788880172.735727:0:201284:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 0000000027960f68): req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.735742:0:201284:0:(import.c:1754:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.735744:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880172.735746:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776780632192/t0(0) o250->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 520/416 e 0 to 0 dl 1788880182 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880172.735752:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:fa4856d5-51a8-499e-b3f8-b477743a3dec:201284:1875776780632192:192.168.204.111@tcp:250: 00000100:00000001:2.0:1788880172.735755:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.735758:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880172.735760:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000fd6b7ee1 x1875776780632192/t0(0) o250->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 520/416 e 0 to 0 dl 1788880182 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.735765:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880172.735768:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880172.735770:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000b278c5d9. 02000000:00000001:2.0:1788880172.735773:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880172.735774:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880172.735780:0:201284:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880172.735782:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880172.735784:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 0000000077b5e929. 02000000:00000001:2.0:1788880172.735788:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880172.735790:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880172.735792:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000fd6b7ee1. 00000100:00000001:0.0:1788880172.735797:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880172.735797:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880172.735798:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.735801:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.735802:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.735806:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880172.735809:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.735810:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1788880172.735813:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880172.735813:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.735818:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880172.735819:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880172.735822:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.735823:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.735835:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880172.735840:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880172.735842:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.735844:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880172.735849:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000006ece4223. 02000000:00000001:0.0:1788880172.735852:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880172.735856:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f832fa99. 00000400:00000010:0.0:1788880172.735865:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 00000100:00000200:0.0:1788880172.735869:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780632320, portal 25 00000100:00000001:0.0:1788880172.735872:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880172.735873:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 00000100:00000040:0.0:1788880172.735878:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.735885:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880172.735887:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880172.735890:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e9812fb0. 00000100:00000200:0.0:1788880172.735893:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780632320, offset 0 00000400:00000200:0.0:1788880172.735900:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880172.735908:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880172.735918:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880172.735921:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880172.735924:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880172.735927:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880172.735934:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [5:5:256:4294967295] 192.168.204.111@tcp LPNI seq info [5:5:8:4294967295] 00000400:00000200:0.0:1788880172.735949:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880172.735954:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880172.735959:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000800:00000200:0.0:1788880172.735965:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880172.735971:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880172.735975:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:0.0:1788880172.735989:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.735993:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305803844 00000100:00000001:0.0:1788880172.736008:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305803842 00000100:00000001:0.0:1788880172.736012:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305803844 00000100:00000001:0.0:1788880172.736015:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305803842 00000100:00000001:0.0:1788880172.736019:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305803844 00000100:00000001:0.0:1788880172.736022:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305803842 00000100:00000001:0.0:1788880172.736026:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305803844 00000100:00000001:0.0:1788880172.736042:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305803842 00000100:00000001:0.0:1788880172.736045:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880172.736048:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.736051:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880172.736052:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:0.0:1788880172.736055:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 0000000027960f68 going to sleep for 11 seconds 00000100:00000001:0.0:1788880172.736058:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880172.736059:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880172.736092:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880172.736096:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f92e72a2. 00000400:00000200:2.0:1788880172.736101:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880172.736105:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880172.736108:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e9812fb0 00000400:00000010:2.0:1788880172.736110:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e9812fb0. 00000100:00000001:2.0:1788880172.736115:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880172.736119:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.736127:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880172.736129:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.736135:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.736137:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880172.737398:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880172.737405:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880172.737406:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880172.737409:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000f832fa99. 00000400:00000200:0.0:1788880172.737414:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8884ddc 00000400:00000010:0.0:1788880172.737415:0:203392:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8884ddc. 00000100:00000001:0.0:1788880172.737421:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.737425:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880172.737436:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.737444:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880172.737446:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.737448:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880172.737449:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880172.737451:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880172.737453:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.737455:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880172.737473:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=0 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880172.737477:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=0 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880172.737479:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880172.737480:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.737483:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880172.737486:0:203392:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.737488:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880172.737492:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.737498:0:203392:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880172.737501:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880172.737502:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880172.737506:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880172.737513:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780632320:192.168.204.111@tcp:101: 00000100:00000001:0.0:1788880172.737518:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.737522:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.737524:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:0.0:1788880172.737526:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.737529:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.737535:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880172.737538:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000027960f68. 00000100:00000001:0.0:1788880172.737541:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:0.0:1788880172.737543:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.737546:0:203392:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:0.0:1788880172.737548:0:203392:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880172.737550:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880172.737553:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000081b300c8 refcount=4 00000020:00000001:0.0:1788880172.737556:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862489088 : -126823847062528 : ffff8ca785a59400) 00010000:00000001:0.0:1788880172.737559:0:203392:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862489088 : -126823847062528 : ffff8ca785a59400) 00010000:00000040:0.0:1788880172.737566:0:203392:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 0000000081b300c8, remote cookie: 0x92f11317f60a7902, flags: 0x0 00010000:00000001:0.0:1788880172.737576:0:203392:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:0.0:1788880172.737580:0:203392:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:0.0:1788880172.737583:0:203392:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:0.0:1788880172.737584:0:203392:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:0.0:1788880172.737585:0:203392:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:0.0:1788880172.737589:0:203392:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x0:0x0].0x0 (000000005bab1010) refcount = 1 00010000:00010000:0.0:1788880172.737592:0:203392:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7902 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880172.737600:0:203392:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:0.0:1788880172.737602:0:203392:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:0.0:1788880172.737603:0:203392:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:0.0:1788880172.737606:0:203392:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:0.0:1788880172.737608:0:203392:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880172.737611:0:203392:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7902 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880172.737617:0:203392:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:0.0:1788880172.737619:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880172.737622:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880172.737623:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880172.737624:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880172.737625:0:203392:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.737629:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.737634:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780632320/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.737641:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880172.737643:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880172.737645:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000006ece4223. 02000000:00000001:0.0:1788880172.737651:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880172.737654:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880172.737655:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880172.737657:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880172.737659:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000bd87bd8d. 02000000:00000001:0.0:1788880172.737663:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880172.737664:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880172.737672:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000001:0.0:1788880172.737676:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880172.737677:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:0.0:1788880172.737679:0:203392:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788880172.737682:0:203392:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:0.0:1788880172.737684:0:203392:0:(mgc_request.c:108:config_log_get()) log lustre-sptlrpc (000000007b8fd88a) refs 2 10000000:00000001:0.0:1788880172.737686:0:203392:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.737689:0:203392:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880172.737690:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880172.737692:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000081b300c8 refcount=3 00000020:00000001:0.0:1788880172.737694:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862489088 : -126823847062528 : ffff8ca785a59400) 00010000:00000001:0.0:1788880172.737696:0:203392:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862489088 : -126823847062528 : ffff8ca785a59400) 00010000:00000001:0.0:1788880172.737698:0:203392:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:0.0:1788880172.737699:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880172.737701:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880172.737702:0:203392:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788880172.737705:0:203392:0:(mgc_request.c:1668:mgc_process_cfg_log()) Process entered 10000000:00000010:0.0:1788880172.737708:0:203392:0:(mgc_request.c:1682:mgc_process_cfg_log()) kmalloced '(env)': 56 at 000000001fdb1d01. 00000020:00000010:0.0:1788880172.737713:0:203392:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000008e74578. 00000020:00000010:0.0:1788880172.737720:0:203392:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000e0fe5497. 00000040:00000010:0.0:1788880172.737723:0:203392:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000b5c40084. 00000020:00000010:0.0:1788880172.737727:0:203392:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000135c2406. 00000020:00000010:0.0:1788880172.737732:0:203392:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 000000007c59c7b9. 80000000:00000010:0.0:1788880172.737736:0:203392:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000da3f50d1. 00040000:00000010:0.0:1788880172.737740:0:203392:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000b52810a8. 20000000:00000010:0.0:1788880172.737744:0:203392:0:(mgs_handler.c:1787:mgs_key_init()) kmalloced '(value)': 392 at 00000000b405d9c4. 10000000:00000040:0.0:1788880172.737747:0:203392:0:(lustre_log.h:387:llog_ctxt_get()) GETting ctxt 000000006d5d0799 : new refcount 2 00000020:00000001:0.0:1788880172.737750:0:203392:0:(obd_config.c:2120:class_config_parse_llog()) Process entered 00000020:00000040:0.0:1788880172.737752:0:203392:0:(obd_config.c:2122:class_config_parse_llog()) looking up llog lustre-sptlrpc 00000040:00000001:0.0:1788880172.737754:0:203392:0:(llog.c:1538:llog_open()) Process entered 00000040:00000010:0.0:1788880172.737756:0:203392:0:(llog.c:42:llog_alloc_handle()) kmalloced '(loghandle)': 352 at 00000000a4a3549f. 00000040:00000001:0.0:1788880172.737759:0:203392:0:(llog_client.c:70:llog_client_open()) Process entered 00000020:00000040:0.0:1788880172.737761:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:0.0:1788880172.737765:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ff0be673. 00000020:00000040:0.0:1788880172.737768:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880172.737777:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880172.737780:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880172.737784:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880172.737788:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000fdf771c7. 00000100:00000001:0.0:1788880172.737793:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.737799:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:0.0:1788880172.737801:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:0.0:1788880172.737805:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000ed774610. 00000100:00000001:0.0:1788880172.737808:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249982991168 : -126823726560448 : ffff8ca78cd44b40) 00000100:00000001:0.0:1788880172.737810:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880172.737813:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000100:00000001:0.0:1788880172.737818:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880172.737820:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.737822:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:0.0:1788880172.737823:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880172.737829:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ff0be673 x1875776780632448/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.737835:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880172.737836:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.737840:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880172.737843:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880172.737845:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880172.737848:0:203392:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780632448:192.168.204.111@tcp:501: 00000100:00000001:0.0:1788880172.737852:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880172.737855:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880172.737857:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.737860:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880172.737863:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000030a8fe2c. 02000000:00000001:0.0:1788880172.737865:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880172.737870:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f832fa99. 00000400:00000010:0.0:1788880172.737876:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 00000100:00000200:0.0:1788880172.737880:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780632448, portal 25 00000100:00000001:0.0:1788880172.737882:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880172.737883:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000100:00000040:0.0:1788880172.737897:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ff0be673 x1875776780632448/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.737903:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880172.737908:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880172.737912:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000200:0.0:1788880172.737917:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 512 bytes to portal 26, xid 1875776780632448, offset 0 00000400:00000200:0.0:1788880172.737926:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880172.737935:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880172.737940:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880172.737944:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880172.737947:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880172.737951:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880172.737954:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [6:6:256:4294967295] 192.168.204.111@tcp LPNI seq info [6:6:8:4294967295] 00000400:00000200:0.0:1788880172.737964:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880172.737969:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 512 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880172.737973:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000800:00000200:0.0:1788880172.737979:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880172.737985:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880172.737988:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 608 niov 1 nkiov 1 00000100:00000001:0.0:1788880172.738000:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.738002:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305803846 00000800:00000200:0.0:1788880172.738073:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880172.738077:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f92e72a2. 00000400:00000200:0.0:1788880172.738081:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.738085:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880172.738089:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:0.0:1788880172.738090:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:0.0:1788880172.738093:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880172.738096:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ff0be673 x1875776780632448/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.738104:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.738115:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000ff0be673 x1875776780632448/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.738122:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.738124:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880172.738145:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305803844 00000100:00000001:0.0:1788880172.738153:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305803846 00000100:00000001:0.0:1788880172.738157:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305803844 00000100:00000001:0.0:1788880172.738162:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305803846 00000100:00000001:0.0:1788880172.738166:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305803844 00000100:00000001:0.0:1788880172.738174:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305803846 00000100:00000001:0.0:1788880172.738185:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305803844 00000100:00000001:0.0:1788880172.738192:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880172.738195:0:203392:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.738197:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880172.738199:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:0.0:1788880172.738202:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 00000000ed774610 going to sleep for 11 seconds 00000100:00000001:0.0:1788880172.738205:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880172.738206:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880172.739128:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.739139:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.739142:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.739145:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.739152:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880172.739165:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 272 into portal 25 MB=0x6aa0250b00180 00000400:00000200:0.0:1788880172.739171:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 272/272 into md 0x129 [1] + 224 00000800:00000001:0.0:1788880172.739175:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.739181:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.739183:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.739186:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880172.739189:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.739192:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ff0be673 x1875776780632448/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880172.739207:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=272 offset=224 replen=272 req@00000000ff0be673 x1875776780632448/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.739222:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880172.739229:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.739286:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.740994:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.741009:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.741013:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.741016:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.741040:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880172.741059:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b00200 00000400:00000200:0.0:1788880172.741070:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0x131 [1] + 224 00000800:00000001:0.0:1788880172.741078:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.741087:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.741090:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.741101:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880172.741109:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.741116:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780632576/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880172.741128:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@0000000008b25f66 x1875776780632576/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.741148:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880172.741161:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.741165:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880172.741788:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880172.741794:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:0.0:1788880172.741802:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.741807:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880172.741811:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009822ffbc 00000400:00000010:0.0:1788880172.741813:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009822ffbc. 00000100:00000001:0.0:1788880172.741817:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880172.741827:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780632704/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.741835:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.741840:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000008b25f66 x1875776780632704/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.741845:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.741846:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880172.742847:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880172.742854:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880172.742857:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880172.742860:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880172.742867:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880172.742878:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 272 into portal 25 MB=0x6aa0250b00280 00000400:00000200:3.0:1788880172.742886:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 272/272 into md 0x139 [1] + 224 00000800:00000001:3.0:1788880172.742892:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880172.742899:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880172.742902:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880172.742906:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880172.742916:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880172.742921:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780632704/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880172.742931:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=272 offset=224 replen=272 req@0000000008b25f66 x1875776780632704/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.742945:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880172.742954:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880172.742957:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.742969:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880172.742975:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880172.742977:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880172.742988:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000031f5d60b. 00000400:00000200:3.0:1788880172.742993:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005377d7ad 00000400:00000010:3.0:1788880172.742995:0:203392:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005377d7ad. 00000100:00000001:3.0:1788880172.742999:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880172.743002:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780632704/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880172.743011:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780632704/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.743018:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880172.743020:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.743023:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880172.743026:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880172.743039:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880172.743041:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880172.743044:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880172.743050:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=0 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880172.743053:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=0 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880172.743054:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880172.743055:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880172.743057:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880172.743059:0:203392:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.743064:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880172.743067:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780632704/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.743073:0:203392:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880172.743076:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880172.743077:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880172.743080:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780632704/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880172.743084:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780632704:192.168.204.111@tcp:501: 00000100:00000001:3.0:1788880172.743088:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.743092:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.743094:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:3.0:1788880172.743102:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880172.743104:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780632704/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.743108:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880172.743110:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000029c63527. 00000100:00000001:3.0:1788880172.743112:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:3.0:1788880172.743113:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:3.0:1788880172.743117:0:203392:0:(llog_client.c:122:llog_client_open()) Process leaving 00000020:00000001:3.0:1788880172.743120:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880172.743121:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880172.743123:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880172.743124:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880172.743126:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780632704/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880188 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.743133:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880172.743135:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880172.743137:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000c23c60c5. 02000000:00000001:3.0:1788880172.743140:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880172.743141:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880172.743142:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880172.743144:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880172.743146:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000f0470777. 02000000:00000001:3.0:1788880172.743150:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880172.743151:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880172.743153:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:3.0:1788880172.743157:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880172.743158:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000040:00000001:3.0:1788880172.743161:0:203392:0:(llog.c:1563:llog_open()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:3.0:1788880172.743165:0:203392:0:(llog.c:415:llog_init_handle()) Process entered 00000040:00000010:3.0:1788880172.743172:0:203392:0:(llog.c:420:llog_init_handle()) kmalloced '(llh)': 8192 at 000000002063d487. 00000040:00000001:3.0:1788880172.743176:0:203392:0:(llog.c:363:llog_read_header()) Process entered 00000040:00000001:3.0:1788880172.743178:0:203392:0:(llog_client.c:269:llog_client_read_header()) Process entered 00000020:00000040:3.0:1788880172.743181:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:3.0:1788880172.743185:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000004908e872. 00000020:00000040:3.0:1788880172.743188:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880172.743191:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880172.743200:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880172.743204:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880172.743210:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000002b843fc3. 00000100:00000001:3.0:1788880172.743215:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.743219:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:3.0:1788880172.743220:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:3.0:1788880172.743226:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 000000004a0835c7. 00000100:00000001:3.0:1788880172.743228:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250052748800 : -126823656802816 : ffff8ca790fcb600) 00000100:00000001:3.0:1788880172.743231:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880172.743277:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430912 : -126823066120704 : ffff8ca7b431ce00) 00000100:00000001:3.0:1788880172.743282:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880172.743285:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.743287:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:3.0:1788880172.743289:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880172.743293:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.743301:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880172.743303:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880172.743306:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880172.743309:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880172.743312:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880172.743314:0:203392:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000004908e872 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780632832:192.168.204.111@tcp:503: 00000100:00000001:3.0:1788880172.743318:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880172.743323:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880172.743325:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880172.743327:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880172.743345:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 16384 at 000000007fd53b74. 02000000:00000001:3.0:1788880172.743348:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880172.743353:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000067d070dd. 00000400:00000010:3.0:1788880172.743360:0:203392:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 00000000c02ba01c. 00000100:00000200:3.0:1788880172.743366:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 16384 bytes, xid 1875776780632832, portal 25 00000100:00000001:3.0:1788880172.743369:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880172.743370:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430912 : -126823066120704 : ffff8ca7b431ce00) 00000100:00000040:3.0:1788880172.743375:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.743389:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880172.743393:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880172.743397:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000097c86a00. 00000100:00000200:3.0:1788880172.743400:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 272 bytes to portal 26, xid 1875776780632832, offset 0 00000400:00000200:3.0:1788880172.743408:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880172.743418:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880172.743425:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880172.743429:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880172.743433:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880172.743437:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880172.743440:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [9:9:256:4294967295] 192.168.204.111@tcp LPNI seq info [9:9:8:4294967295] 00000400:00000200:3.0:1788880172.743451:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880172.743460:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 272 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880172.743465:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000005c064165. 00000800:00000200:3.0:1788880172.743471:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880172.743478:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880172.743482:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000005c064165 type 1, nob 368 niov 1 nkiov 1 00000100:00000001:3.0:1788880172.743495:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.743498:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305803852 00000100:00000001:3.0:1788880172.743506:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305803850 00000100:00000001:3.0:1788880172.743517:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305803852 00000100:00000001:3.0:1788880172.743527:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305803850 00000100:00000001:3.0:1788880172.743532:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305803852 00000100:00000001:3.0:1788880172.743539:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305803850 00000100:00000001:3.0:1788880172.743542:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305803852 00000100:00000001:3.0:1788880172.743546:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305803850 00000100:00000001:3.0:1788880172.743549:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880172.743551:0:203392:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.743563:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880172.743565:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:3.0:1788880172.743567:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000004a0835c7 going to sleep for 11 seconds 00000100:00000001:3.0:1788880172.743569:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880172.743572:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880172.743606:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880172.743611:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000005c064165. 00000400:00000200:0.0:1788880172.743615:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.743622:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880172.743625:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000097c86a00 00000400:00000010:0.0:1788880172.743627:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000097c86a00. 00000100:00000001:0.0:1788880172.743630:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880172.743634:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.743642:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.743643:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.743648:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.743649:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880172.744725:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.744744:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.744747:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.744750:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.744758:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880172.744770:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 8416 into portal 25 MB=0x6aa0250b00300 00000400:00000200:0.0:1788880172.744774:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 8416/8416 into md 0x141 [4] + 224 00000800:00000001:0.0:1788880172.744778:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.744801:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.744804:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.744814:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880172.744819:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.744832:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880172.744841:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=8416 offset=224 replen=8416 req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.744853:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880172.744862:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.744864:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.744885:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880172.744892:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880172.744893:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880172.744895:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000067d070dd. 00000400:00000200:3.0:1788880172.744899:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c02ba01c 00000400:00000010:3.0:1788880172.744901:0:203392:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 00000000c02ba01c. 00000100:00000001:3.0:1788880172.744904:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880172.744910:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880172.744918:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.744925:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880172.744926:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.744929:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880172.744930:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880172.744932:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880172.744934:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880172.744936:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880172.744940:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=0 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880172.744943:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=0 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880172.744945:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880172.744947:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880172.744949:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880172.744951:0:203392:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.744953:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880172.744958:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.744964:0:203392:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880172.744967:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880172.744969:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880172.744980:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880172.744987:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000004908e872 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780632832:192.168.204.111@tcp:503: 00000100:00000001:3.0:1788880172.744992:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.744995:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.744997:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:3.0:1788880172.744999:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880172.745002:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.745008:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880172.745010:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 000000004a0835c7. 00000100:00000001:3.0:1788880172.745013:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:3.0:1788880172.745015:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:3.0:1788880172.745020:0:203392:0:(llog_client.c:319:llog_client_read_header()) Process leaving 00000100:00000001:3.0:1788880172.745021:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880172.745023:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000004908e872 x1875776780632832/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.745043:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880172.745045:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880172.745047:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 16384 at 000000007fd53b74. 02000000:00000001:3.0:1788880172.745056:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880172.745059:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880172.745060:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880172.745062:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880172.745064:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000002b843fc3. 02000000:00000001:3.0:1788880172.745068:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880172.745070:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880172.745071:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000004908e872. 00000100:00000001:3.0:1788880172.745075:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880172.745076:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880172.745079:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880172.745080:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880172.745082:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 00000040:00000001:3.0:1788880172.745084:0:203392:0:(llog.c:403:llog_read_header()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:3.0:1788880172.745092:0:203392:0:(llog.c:481:llog_init_handle()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:3.0:1788880172.745095:0:203392:0:(llog.c:951:llog_process_or_fork()) Process entered 00000040:00000010:3.0:1788880172.745099:0:203392:0:(llog.c:953:llog_process_or_fork()) kmalloced '(lpi)': 96 at 000000002b2fe58c. 00000040:00001000:3.0:1788880172.745101:0:203392:0:(llog.c:968:llog_process_or_fork()) Processing [0xa:0x3:0x0] flags 0x004 startcat -1 startidx -1 first_idx 0 last_idx 0 read_mode 0 00000040:00000001:2.0:1788880172.745373:0:203405:0:(llog.c:552:llog_process_thread()) Process entered 00000020:00000010:2.0:1788880172.745379:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000b1120e74. 00000020:00000010:2.0:1788880172.745385:0:203405:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 0000000027d4cece. 00000040:00000010:2.0:1788880172.745388:0:203405:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000c7975147. 00000020:00000010:2.0:1788880172.745392:0:203405:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 000000001e393693. 00000020:00000010:2.0:1788880172.745397:0:203405:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 000000004a155362. 00000020:00000010:2.0:1788880172.745401:0:203405:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 000000009a0ddd94. 80000000:00000010:2.0:1788880172.745406:0:203405:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000bdb31c0d. 40000000:00000010:2.0:1788880172.745413:0:203405:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 000000009978e132. 00100000:00000010:2.0:1788880172.745419:0:203405:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 00000000d610634e. 00040000:00000010:2.0:1788880172.745422:0:203405:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 000000009cf93093. 00040000:00000010:2.0:1788880172.745427:0:203405:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 0000000004b8209f. 00040000:00000010:2.0:1788880172.745430:0:203405:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 000000003720a685. 00000004:00000010:2.0:1788880172.745435:0:203405:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 00000000c85a7d49. 00000004:00000010:2.0:1788880172.745440:0:203405:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 000000002af8b51d. 00000004:00000010:2.0:1788880172.745444:0:203405:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 000000001a0c73ca. 00002000:00000010:2.0:1788880172.745448:0:203405:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 00000000b34868d0. 00000004:00000010:2.0:1788880172.745452:0:203405:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 00000000813c171c. 00000020:00000010:2.0:1788880172.745457:0:203405:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 0000000095cce73e. 00000040:00000010:2.0:1788880172.745478:0:203405:0:(llog.c:578:llog_process_thread()) kmalloced '(buf)': 8192 at 00000000c092b841. 00000040:00001000:2.0:1788880172.745482:0:203405:0:(llog.c:615:llog_process_thread()) index: 1 last_index 64767 00000040:00000001:2.0:1788880172.745485:0:203405:0:(lustre_log.h:475:llog_next_block()) Process entered 00000040:00000001:2.0:1788880172.745487:0:203405:0:(llog_client.c:140:llog_client_next_block()) Process entered 00000020:00000040:2.0:1788880172.745489:0:203405:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:2.0:1788880172.745495:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000020:00000040:2.0:1788880172.745497:0:203405:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880172.745500:0:203405:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880172.745502:0:203405:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880172.745506:0:203405:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880172.745510:0:203405:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 0000000029b90874. 00000100:00000001:2.0:1788880172.745514:0:203405:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.745519:0:203405:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:2.0:1788880172.745521:0:203405:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:2.0:1788880172.745526:0:203405:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000db63a97d. 00000100:00000001:2.0:1788880172.745542:0:203405:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250282256448 : -126823427295168 : ffff8ca79eaab840) 00000100:00000001:2.0:1788880172.745545:0:203405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880172.745546:0:203405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000100:00000001:2.0:1788880172.745550:0:203405:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880172.745552:0:203405:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.745554:0:203405:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:2.0:1788880172.745556:0:203405:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880172.745560:0:203405:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.745578:0:203405:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880172.745579:0:203405:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880172.745582:0:203405:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880172.745583:0:203405:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880172.745585:0:203405:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880172.745587:0:203405:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job llog_process_th:fa4856d5-51a8-499e-b3f8-b477743a3dec:203405:1875776780632960:192.168.204.111@tcp:502: 00000100:00000001:2.0:1788880172.745590:0:203405:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880172.745594:0:203405:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880172.745596:0:203405:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880172.745598:0:203405:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880172.745611:0:203405:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 16384 at 00000000786facfb. 02000000:00000001:2.0:1788880172.745614:0:203405:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880172.745619:0:203405:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000400:00000010:2.0:1788880172.745623:0:203405:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 000000008e500b55. 00000100:00000200:2.0:1788880172.745627:0:203405:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 16384 bytes, xid 1875776780632960, portal 25 00000100:00000001:2.0:1788880172.745629:0:203405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880172.745630:0:203405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000100:00000040:2.0:1788880172.745634:0:203405:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.745641:0:203405:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880172.745643:0:203405:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880172.745657:0:203405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c6ecbca9. 00000100:00000200:2.0:1788880172.745660:0:203405:0:(niobuf.c:94:ptl_send_buf()) Sending 272 bytes to portal 26, xid 1875776780632960, offset 0 00000400:00000200:2.0:1788880172.745679:0:203405:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880172.745688:0:203405:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880172.745695:0:203405:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880172.745699:0:203405:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880172.745702:0:203405:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880172.745704:0:203405:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880172.745707:0:203405:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [10:10:256:4294967295] 192.168.204.111@tcp LPNI seq info [10:10:8:4294967295] 00000400:00000200:2.0:1788880172.745715:0:203405:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880172.745723:0:203405:0:(socklnd_cb.c:931:ksocknal_send()) sending 272 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880172.745727:0:203405:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000036dc6dd9. 00000800:00000200:2.0:1788880172.745731:0:203405:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880172.745736:0:203405:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880172.745739:0:203405:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000036dc6dd9 type 1, nob 368 niov 1 nkiov 1 00000100:00000001:2.0:1788880172.745752:0:203405:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.745755:0:203405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305803854 00000100:00000001:2.0:1788880172.745761:0:203405:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305803852 00000100:00000001:2.0:1788880172.745768:0:203405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305803854 00000100:00000001:2.0:1788880172.745773:0:203405:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305803852 00000100:00000001:2.0:1788880172.745776:0:203405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305803854 00000100:00000001:2.0:1788880172.745787:0:203405:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305803852 00000100:00000001:2.0:1788880172.745794:0:203405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305803854 00000100:00000001:2.0:1788880172.745797:0:203405:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305803852 00000100:00000001:2.0:1788880172.745800:0:203405:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880172.745802:0:203405:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.745805:0:203405:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880172.745807:0:203405:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:2.0:1788880172.745810:0:203405:0:(client.c:2702:ptlrpc_set_wait()) set 00000000db63a97d going to sleep for 11 seconds 00000100:00000001:2.0:1788880172.745813:0:203405:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880172.745814:0:203405:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880172.745842:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880172.745846:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000036dc6dd9. 00000400:00000200:3.0:1788880172.745852:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880172.745857:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880172.745861:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c6ecbca9 00000400:00000010:3.0:1788880172.745864:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c6ecbca9. 00000100:00000001:3.0:1788880172.745869:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880172.745873:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.745882:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880172.745885:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.745891:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.745893:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880172.747274:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.747286:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.747290:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.747292:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.747300:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880172.747311:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 8472 into portal 25 MB=0x6aa0250b00380 00000400:00000200:0.0:1788880172.747318:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 8472/8472 into md 0x149 [4] + 224 00000800:00000001:0.0:1788880172.747323:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.747333:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.747336:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.747340:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880172.747347:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.747354:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880172.747364:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=8472 offset=224 replen=8472 req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.747380:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880172.747388:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.747392:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.747407:0:203405:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880172.747415:0:203405:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880172.747417:0:203405:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880172.747418:0:203405:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:2.0:1788880172.747423:0:203405:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008e500b55 00000400:00000010:2.0:1788880172.747425:0:203405:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 000000008e500b55. 00000100:00000001:2.0:1788880172.747430:0:203405:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880172.747436:0:203405:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880172.747449:0:203405:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.747467:0:203405:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880172.747469:0:203405:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.747471:0:203405:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880172.747474:0:203405:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880172.747476:0:203405:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880172.747477:0:203405:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880172.747479:0:203405:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880172.747483:0:203405:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=0 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880172.747485:0:203405:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=0 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880172.747486:0:203405:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880172.747487:0:203405:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880172.747489:0:203405:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880172.747490:0:203405:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.747492:0:203405:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880172.747495:0:203405:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.747499:0:203405:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880172.747501:0:203405:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880172.747503:0:203405:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880172.747505:0:203405:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880172.747509:0:203405:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job llog_process_th:fa4856d5-51a8-499e-b3f8-b477743a3dec:203405:1875776780632960:192.168.204.111@tcp:502: 00000100:00000001:2.0:1788880172.747513:0:203405:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.747514:0:203405:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.747516:0:203405:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:2.0:1788880172.747517:0:203405:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880172.747518:0:203405:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.747523:0:203405:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880172.747524:0:203405:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000db63a97d. 00000100:00000001:2.0:1788880172.747526:0:203405:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:2.0:1788880172.747527:0:203405:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:2.0:1788880172.747536:0:203405:0:(llog_client.c:198:llog_client_next_block()) Process leaving 00000100:00000001:2.0:1788880172.747537:0:203405:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880172.747539:0:203405:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000fd6b7ee1 x1875776780632960/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.747543:0:203405:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880172.747544:0:203405:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880172.747546:0:203405:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 16384 at 00000000786facfb. 02000000:00000001:2.0:1788880172.747553:0:203405:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880172.747555:0:203405:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880172.747556:0:203405:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880172.747558:0:203405:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880172.747559:0:203405:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 0000000029b90874. 02000000:00000001:2.0:1788880172.747561:0:203405:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880172.747563:0:203405:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880172.747564:0:203405:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000fd6b7ee1. 00000100:00000001:2.0:1788880172.747567:0:203405:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880172.747567:0:203405:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788880172.747569:0:203405:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880172.747570:0:203405:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880172.747571:0:203405:0:(genops.c:1251:class_import_put()) Process leaving 00000040:00000001:2.0:1788880172.747572:0:203405:0:(lustre_log.h:485:llog_next_block()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.747574:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2000 type=0x10620000 idx=1 00000040:00001000:2.0:1788880172.747579:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 1 lrh_len: 224 (8192 remains) 00000040:00001000:2.0:1788880172.747580:0:203405:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.747582:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:2.0:1788880172.747585:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:2.0:1788880172.747587:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:2.0:1788880172.747588:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000001:2.0:1788880172.747594:0:203405:0:(obd_mount.c:953:lustre_check_exclusion()) Process entered 00000020:00000001:2.0:1788880172.747595:0:203405:0:(obd_mount.c:958:lustre_check_exclusion()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880172.747599:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 00000000238509cf. 00000020:00000080:2.0:1788880172.747603:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1788880172.747605:0:203405:0:(obd_config.c:1561:class_process_config()) marker 4 (0x1) lustre-clilov lov setup 00000020:00000001:2.0:1788880172.747606:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.747608:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 00000000238509cf. 00000020:00000001:2.0:1788880172.747610:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.747617:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf20e0 type=0x10620000 idx=2 00000040:00001000:2.0:1788880172.747619:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 2 lrh_len: 120 (7968 remains) 00000040:00001000:2.0:1788880172.747621:0:203405:0:(llog.c:805:llog_process_thread()) index: 2, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.747622:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:2.0:1788880172.747626:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 34 at 00000000bfcc82cb. 00000020:01000000:2.0:1788880172.747629:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf001, instance name: lustre-clilov-ffff8ca787527000 00000020:00000010:2.0:1788880172.747632:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 128 at 00000000a9570094. 00000020:00000080:2.0:1788880172.747634:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf001 00000020:00000001:2.0:1788880172.747636:0:203405:0:(obd_config.c:683:class_attach_name()) Process entered 00000020:00000001:2.0:1788880172.747639:0:203405:0:(genops.c:350:class_newdev()) Process entered 00000020:00000010:2.0:1788880172.747657:0:203405:0:(genops.c:51:obd_device_alloc()) slab-alloced 'obd': 6592 at 00000000998f2370. 00000020:00000080:2.0:1788880172.747663:0:203405:0:(genops.c:413:class_newdev()) Allocate new device lustre-clilov-ffff8ca787527000 (00000000998f2370) 00000020:00000001:2.0:1788880172.747666:0:203405:0:(genops.c:952:__class_new_export()) Process entered 00000020:00000010:2.0:1788880172.747669:0:203405:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 00000000e61aab13. 00000020:00000001:2.0:1788880172.747672:0:203405:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880172.747674:0:203405:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000e61aab13 with handle 0x805d548a36a88666 to hash 00000020:00000001:2.0:1788880172.747676:0:203405:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:2.0:1788880172.747678:0:203405:0:(obd_class.h:1016:obd_init_export()) Process entered 00000020:00000001:2.0:1788880172.747680:0:203405:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.747682:0:203405:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617250036436992 : -126823673114624 : ffff8ca79003d000) 00000020:00000040:2.0:1788880172.747684:0:203405:0:(genops.c:900:class_export_put()) PUTting export 00000000e61aab13 : new refcount 1 00000020:00000040:2.0:1788880172.747689:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-clilov-ffff8ca787527000 (00000000998f2370) now 2 - obd_device_list 00000020:00000001:2.0:1788880172.747698:0:203405:0:(genops.c:555:class_register_device()) Process leaving (rc=0 : 0 : 0) 00000020:00000080:2.0:1788880172.747700:0:203405:0:(obd_config.c:720:class_attach_name()) OBD: dev 1 attached type lov with refcount 2 00000020:00000001:2.0:1788880172.747701:0:203405:0:(obd_config.c:722:class_attach_name()) Process leaving (rc=18446617250653057920 : -126823056493696 : ffff8ca7b4c4b380) 00000020:00000001:2.0:1788880172.747703:0:203405:0:(obd_config.c:1479:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.747705:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 128 at 00000000a9570094. 00000020:00000010:2.0:1788880172.747707:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 34 at 00000000bfcc82cb. 00000020:00000001:2.0:1788880172.747709:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.747710:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2158 type=0x10620000 idx=3 00000040:00001000:2.0:1788880172.747712:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 3 lrh_len: 168 (7848 remains) 00000040:00001000:2.0:1788880172.747713:0:203405:0:(llog.c:805:llog_process_thread()) index: 3, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.747714:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:2.0:1788880172.747720:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 34 at 00000000bfcc82cb. 00000020:01000000:2.0:1788880172.747721:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf003, instance name: lustre-clilov-ffff8ca787527000 00000020:00000010:2.0:1788880172.747723:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 160 at 00000000db63a97d. 00000020:00000080:2.0:1788880172.747724:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf003 00000020:00000001:2.0:1788880172.747728:0:203405:0:(obd_config.c:742:class_setup()) Process entered 00000001:00000001:2.0:1788880172.747746:0:203405:0:(hash.c:1038:cfs_hash_create()) Process entered 00000001:00000010:2.0:1788880172.747748:0:203405:0:(hash.c:1065:cfs_hash_create()) alloc '(hs)': 232 at 00000000238509cf. 00000001:00000010:2.0:1788880172.747752:0:203405:0:(hash.c:933:cfs_hash_buckets_realloc()) alloc '(new_bkts)': 32 at 00000000f0170a92. 00000001:00000010:2.0:1788880172.747755:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000029b90874. 00000001:00000010:2.0:1788880172.747758:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000057e7018a. 00000001:00000010:2.0:1788880172.747760:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000040ce4b8f. 00000001:00000010:2.0:1788880172.747762:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 00000000de26210d. 00000020:00000001:2.0:1788880172.747765:0:203405:0:(obd_class.h:565:obd_setup()) Process entered 00000020:00000010:2.0:1788880172.747769:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000097313bcf. 00000020:00000010:2.0:1788880172.747773:0:203405:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 00000000a9570094. 00000020:00000010:2.0:1788880172.747784:0:203405:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 000000006de8747c. 00000008:00000010:2.0:1788880172.747792:0:203405:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000ed7da6b9. 00020000:00000010:2.0:1788880172.747806:0:203405:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 00000000921e4385. 00008000:00000010:2.0:1788880172.747821:0:203405:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 000000009f2e63a1. 00000080:00000010:2.0:1788880172.747833:0:203405:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 0000000014012704. 00000020:00000010:2.0:1788880172.747838:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000006b65b0a7. 00000020:00000010:2.0:1788880172.747841:0:203405:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000f25b7c34. 00000020:00000010:2.0:1788880172.747844:0:203405:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 000000007ff7ab1f. 00000008:00000010:2.0:1788880172.747848:0:203405:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000218bf9e7. 00020000:00000010:2.0:1788880172.747851:0:203405:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 000000003997086d. 00008000:00000010:2.0:1788880172.747854:0:203405:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000e8840d53. 00000080:00000010:2.0:1788880172.747857:0:203405:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 00000000d35adf78. 00000080:00000010:2.0:1788880172.747860:0:203405:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 0000000034e14fab. 00020000:00000010:2.0:1788880172.747865:0:203405:0:(lov_dev.c:598:lov_device_alloc()) kmalloced '(ld)': 632 at 00000000096d0f5b. 00020000:00000001:2.0:1788880172.747871:0:203405:0:(lov_obd.c:686:lov_setup()) Process entered 00000020:00000001:2.0:1788880172.747882:0:203405:0:(lu_tgt_pool.c:44:lu_tgt_pool_init()) Process entered 00000020:00000010:2.0:1788880172.747884:0:203405:0:(lu_tgt_pool.c:52:lu_tgt_pool_init()) kmalloced '(op->op_array)': 8 at 000000005da561ac. 00000020:00000001:2.0:1788880172.747886:0:203405:0:(lu_tgt_pool.c:57:lu_tgt_pool_init()) Process leaving 00020000:00000010:2.0:1788880172.748025:0:203405:0:(lov_dev.c:615:lov_device_alloc()) kmalloced '(ld->ld_md_tgts)': 4096 at 0000000004ae2fd7. 00020000:00000010:2.0:1788880172.748047:0:203405:0:(lov_dev.c:622:lov_device_alloc()) kmalloced '(ld->ld_lov->lov_mdc_tgts)': 4096 at 0000000097268222. 00000020:00000001:2.0:1788880172.748049:0:203405:0:(lu_object.c:1137:lu_site_init()) Process entered 00000020:00000010:2.0:1788880172.748090:0:203405:0:(lu_object.c:1156:lu_site_init()) kmalloced '(s->ls_bkts)': 18432 at 00000000771711c1. 00000020:00000010:2.0:1788880172.748100:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 000000000cc0514a. 00000020:00000010:2.0:1788880172.748105:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 192 at 000000008e500b55. 00000020:00000001:2.0:1788880172.748111:0:203405:0:(lu_object.c:1199:lu_site_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880172.748124:0:203405:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000b3250a89. 00000040:00000010:2.0:1788880172.748131:0:203405:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 0000000092037f0d. 00000020:00000010:2.0:1788880172.748136:0:203405:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 000000008e5896c5. 00000020:00000010:2.0:1788880172.748140:0:203405:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 0000000023cc42b5. 00000020:00000010:2.0:1788880172.748143:0:203405:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 000000005143beb5. 80000000:00000010:2.0:1788880172.748146:0:203405:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000ccaa1308. 40000000:00000010:2.0:1788880172.748151:0:203405:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 000000005fe92bd6. 00000008:00000010:2.0:1788880172.748154:0:203405:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 0000000094fb9372. 00020000:00000010:2.0:1788880172.748157:0:203405:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000454826b1. 00008000:00000010:2.0:1788880172.748160:0:203405:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 000000001add5d07. 00100000:00000010:2.0:1788880172.748164:0:203405:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 00000000a7a12cec. 00040000:00000010:2.0:1788880172.748168:0:203405:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 000000004d999a1e. 00040000:00000010:2.0:1788880172.748172:0:203405:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 00000000e4eeeb47. 00040000:00000010:2.0:1788880172.748176:0:203405:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 000000009430e1e9. 00000004:00000010:2.0:1788880172.748180:0:203405:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 0000000049666ba3. 00000004:00000010:2.0:1788880172.748186:0:203405:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 00000000ad9382b5. 00000004:00000010:2.0:1788880172.748191:0:203405:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 00000000fbea29db. 00002000:00000010:2.0:1788880172.748194:0:203405:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 0000000072169ee5. 00000004:00000010:2.0:1788880172.748199:0:203405:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 000000008b7753d4. 00000080:00000010:2.0:1788880172.748203:0:203405:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 00000000f35c6a34. 00000080:00000010:2.0:1788880172.748206:0:203405:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 00000000dc1ebd24. 00000020:00000010:2.0:1788880172.748211:0:203405:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 000000001da35de8. 00000040:00000010:2.0:1788880172.748216:0:203405:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 000000008b272251. 00000020:00000010:2.0:1788880172.748221:0:203405:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 0000000038914574. 00000020:00000010:2.0:1788880172.748224:0:203405:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 00000000de97f17e. 00000020:00000010:2.0:1788880172.748227:0:203405:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000d809056b. 80000000:00000010:2.0:1788880172.748230:0:203405:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 000000000d6a84d3. 40000000:00000010:2.0:1788880172.748404:0:203405:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 00000000b1f6b24f. 00000008:00000010:2.0:1788880172.748410:0:203405:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000639bce5d. 00020000:00000010:2.0:1788880172.748416:0:203405:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 000000006c287ad9. 00008000:00000010:2.0:1788880172.748420:0:203405:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 000000008dfb476e. 00100000:00000010:2.0:1788880172.748423:0:203405:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 0000000036d5d0e0. 00040000:00000010:2.0:1788880172.748424:0:203405:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 000000001082bb71. 00040000:00000010:2.0:1788880172.748426:0:203405:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 000000001411d2ed. 00040000:00000010:2.0:1788880172.748429:0:203405:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 0000000010151198. 00000004:00000010:2.0:1788880172.748431:0:203405:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 00000000e0af237f. 00000004:00000010:2.0:1788880172.748434:0:203405:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 0000000098f4c154. 00000004:00000010:2.0:1788880172.748436:0:203405:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 00000000b24b0e61. 00002000:00000010:2.0:1788880172.748438:0:203405:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 00000000782b422e. 00000004:00000010:2.0:1788880172.748439:0:203405:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 00000000ce656c09. 00000080:00000010:2.0:1788880172.748442:0:203405:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 0000000058ed3670. 00000080:00000010:2.0:1788880172.748444:0:203405:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 00000000cbeed16a. 00020000:00000001:2.0:1788880172.748447:0:203405:0:(lov_dev.c:634:lov_device_alloc()) Process leaving (rc=18446617250094086144 : -126823615465472 : ffff8ca793737800) 00020000:00000001:2.0:1788880172.748450:0:203405:0:(lov_dev.c:266:lov_device_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880172.748452:0:203405:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000f25b7c34. 00000020:00000010:2.0:1788880172.748456:0:203405:0:(cl_object.c:1137:cl_key_fini()) kfreed 'info': 72 at 000000007ff7ab1f. 00000008:00000010:2.0:1788880172.748459:0:203405:0:(osc_dev.c:100:osc_key_fini()) slab-freed '(info)': 2864 at 00000000218bf9e7. 00020000:00000010:2.0:1788880172.748461:0:203405:0:(lov_dev.c:95:lov_key_fini()) slab-freed '(info)': 192 at 000000003997086d. 00008000:00000010:2.0:1788880172.748464:0:203405:0:(echo_client.c:424:echo_thread_key_fini()) slab-freed '(info)': 904 at 00000000e8840d53. 00000080:00000010:2.0:1788880172.748466:0:203405:0:(vvp_dev.c:80:ll_thread_key_fini()) slab-freed '(lti)': 168 at 00000000d35adf78. 00000080:00000010:2.0:1788880172.748468:0:203405:0:(vvp_dev.c:130:vvp_thread_key_fini()) slab-freed '(vti)': 592 at 0000000034e14fab. 00000020:00000010:2.0:1788880172.748471:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 000000006b65b0a7. 00000020:00000010:2.0:1788880172.748475:0:203405:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 00000000a9570094. 00000020:00000010:2.0:1788880172.748477:0:203405:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 000000006de8747c. 00000008:00000010:2.0:1788880172.748480:0:203405:0:(osc_dev.c:125:osc_session_fini()) slab-freed '(info)': 352 at 00000000ed7da6b9. 00020000:00000010:2.0:1788880172.748483:0:203405:0:(lov_dev.c:136:lov_session_key_fini()) slab-freed '(info)': 584 at 00000000921e4385. 00008000:00000010:2.0:1788880172.748486:0:203405:0:(echo_client.c:449:echo_session_key_fini()) slab-freed '(session)': 8 at 000000009f2e63a1. 00000080:00000010:2.0:1788880172.748489:0:203405:0:(vvp_dev.c:105:vvp_session_key_fini()) slab-freed '(session)': 240 at 0000000014012704. 00000020:00000010:2.0:1788880172.748491:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000097313bcf. 00000020:00000001:2.0:1788880172.748493:0:203405:0:(obd_class.h:601:obd_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880172.748502:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-clilov-ffff8ca787527000 (00000000998f2370) now 3 - setup 00000020:00000080:2.0:1788880172.748505:0:203405:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-clilov-ffff8ca787527000 (uuid ea27006f-52a8-4007-b69a-dcac8279445a) 00000020:00000001:2.0:1788880172.748507:0:203405:0:(obd_config.c:828:class_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.748510:0:203405:0:(obd_config.c:1618:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.748512:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 160 at 00000000db63a97d. 00000020:00000010:2.0:1788880172.748516:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 34 at 00000000bfcc82cb. 00000020:00000001:2.0:1788880172.748519:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.748521:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2200 type=0x10620000 idx=4 00000040:00001000:2.0:1788880172.748525:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 4 lrh_len: 224 (7680 remains) 00000040:00001000:2.0:1788880172.748527:0:203405:0:(llog.c:805:llog_process_thread()) index: 4, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.748528:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:2.0:1788880172.748530:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:2.0:1788880172.748531:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:2.0:1788880172.748533:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000010:2.0:1788880172.748536:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 0000000013230def. 00000020:00000080:2.0:1788880172.748538:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1788880172.748539:0:203405:0:(obd_config.c:1561:class_process_config()) marker 4 (0x2) lustre-clilov lov setup 00000020:00000001:2.0:1788880172.748541:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.748544:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 0000000013230def. 00000020:00000001:2.0:1788880172.748546:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.748548:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf22e0 type=0x10620000 idx=5 00000040:00001000:2.0:1788880172.748549:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 5 lrh_len: 224 (7456 remains) 00000040:00001000:2.0:1788880172.748551:0:203405:0:(llog.c:805:llog_process_thread()) index: 5, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.748552:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:2.0:1788880172.748554:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:2.0:1788880172.748555:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:2.0:1788880172.748557:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000001:2.0:1788880172.748559:0:203405:0:(obd_mount.c:953:lustre_check_exclusion()) Process entered 00000020:00000001:2.0:1788880172.748560:0:203405:0:(obd_mount.c:958:lustre_check_exclusion()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880172.748562:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 0000000013230def. 00000020:00000080:2.0:1788880172.748563:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1788880172.748563:0:203405:0:(obd_config.c:1561:class_process_config()) marker 5 (0x1) lustre-clilmv lmv setup 00000020:00000001:2.0:1788880172.748564:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.748569:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 0000000013230def. 00000020:00000001:2.0:1788880172.748570:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.748573:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf23c0 type=0x10620000 idx=6 00000040:00001000:2.0:1788880172.748574:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 6 lrh_len: 120 (7232 remains) 00000040:00001000:2.0:1788880172.748576:0:203405:0:(llog.c:805:llog_process_thread()) index: 6, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.748577:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:2.0:1788880172.748579:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 34 at 00000000bfcc82cb. 00000020:01000000:2.0:1788880172.748580:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf001, instance name: lustre-clilmv-ffff8ca787527000 00000020:00000010:2.0:1788880172.748583:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 128 at 00000000a9570094. 00000020:00000080:2.0:1788880172.748584:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf001 00000020:00000001:2.0:1788880172.748586:0:203405:0:(obd_config.c:683:class_attach_name()) Process entered 00000020:00000001:2.0:1788880172.748588:0:203405:0:(genops.c:350:class_newdev()) Process entered 00000020:00000010:2.0:1788880172.748594:0:203405:0:(genops.c:51:obd_device_alloc()) slab-alloced 'obd': 6592 at 00000000aeea1d9b. 00000020:00000080:2.0:1788880172.748597:0:203405:0:(genops.c:413:class_newdev()) Allocate new device lustre-clilmv-ffff8ca787527000 (00000000aeea1d9b) 00000020:00000001:2.0:1788880172.748598:0:203405:0:(genops.c:952:__class_new_export()) Process entered 00000020:00000010:2.0:1788880172.748600:0:203405:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 00000000256a683f. 00000020:00000001:2.0:1788880172.748601:0:203405:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880172.748603:0:203405:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000256a683f with handle 0x805d548a36a8866d to hash 00000020:00000001:2.0:1788880172.748604:0:203405:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:2.0:1788880172.748606:0:203405:0:(obd_class.h:1016:obd_init_export()) Process entered 00000020:00000001:2.0:1788880172.748607:0:203405:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.748608:0:203405:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617249948770304 : -126823760781312 : ffff8ca78aca2000) 00000020:00000040:2.0:1788880172.748610:0:203405:0:(genops.c:900:class_export_put()) PUTting export 00000000256a683f : new refcount 1 00000020:00000040:2.0:1788880172.748614:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-clilmv-ffff8ca787527000 (00000000aeea1d9b) now 2 - obd_device_list 00000020:00000001:2.0:1788880172.748618:0:203405:0:(genops.c:555:class_register_device()) Process leaving (rc=0 : 0 : 0) 00000020:00000080:2.0:1788880172.748619:0:203405:0:(obd_config.c:720:class_attach_name()) OBD: dev 2 attached type lmv with refcount 2 00000020:00000001:2.0:1788880172.748621:0:203405:0:(obd_config.c:722:class_attach_name()) Process leaving (rc=18446617250653051328 : -126823056500288 : ffff8ca7b4c499c0) 00000020:00000001:2.0:1788880172.748623:0:203405:0:(obd_config.c:1479:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.748624:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 128 at 00000000a9570094. 00000020:00000010:2.0:1788880172.748627:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 34 at 00000000bfcc82cb. 00000020:00000001:2.0:1788880172.748629:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.748631:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2438 type=0x10620000 idx=7 00000040:00001000:2.0:1788880172.748633:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 7 lrh_len: 168 (7112 remains) 00000040:00001000:2.0:1788880172.748640:0:203405:0:(llog.c:805:llog_process_thread()) index: 7, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.748642:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:2.0:1788880172.748645:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 34 at 00000000bfcc82cb. 00000020:01000000:2.0:1788880172.748648:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf003, instance name: lustre-clilmv-ffff8ca787527000 00000020:00000010:2.0:1788880172.748651:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 160 at 00000000db63a97d. 00000020:00000080:2.0:1788880172.748653:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf003 00000020:00000001:2.0:1788880172.748657:0:203405:0:(obd_config.c:742:class_setup()) Process entered 00000001:00000001:2.0:1788880172.748678:0:203405:0:(hash.c:1038:cfs_hash_create()) Process entered 00000001:00000010:2.0:1788880172.748680:0:203405:0:(hash.c:1065:cfs_hash_create()) alloc '(hs)': 232 at 0000000013230def. 00000001:00000010:2.0:1788880172.748683:0:203405:0:(hash.c:933:cfs_hash_buckets_realloc()) alloc '(new_bkts)': 32 at 00000000c270e7db. 00000001:00000010:2.0:1788880172.748687:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000097313bcf. 00000001:00000010:2.0:1788880172.748690:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 000000006b65b0a7. 00000001:00000010:2.0:1788880172.748692:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 00000000f25b7c34. 00000001:00000010:2.0:1788880172.748697:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 00000000cbd767c1. 00000020:00000001:2.0:1788880172.748699:0:203405:0:(obd_class.h:565:obd_setup()) Process entered 00000020:00000010:2.0:1788880172.748701:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000e6a93e02. 00000020:00000010:2.0:1788880172.748704:0:203405:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 00000000a9570094. 00000020:00000010:2.0:1788880172.748709:0:203405:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 000000006de8747c. 00000008:00000010:2.0:1788880172.748712:0:203405:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000ed7da6b9. 00020000:00000010:2.0:1788880172.748716:0:203405:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 00000000921e4385. 00008000:00000010:2.0:1788880172.748719:0:203405:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 000000009f2e63a1. 00000080:00000010:2.0:1788880172.748723:0:203405:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 0000000014012704. 00000020:00000010:2.0:1788880172.748727:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000005842920. 00000020:00000010:2.0:1788880172.748731:0:203405:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 0000000064b136ca. 00000040:00000010:2.0:1788880172.748733:0:203405:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 000000000c9a81c5. 00000020:00000010:2.0:1788880172.748737:0:203405:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 0000000012119a99. 00000020:00000010:2.0:1788880172.748741:0:203405:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000dadfe130. 00040000:00000010:2.0:1788880172.748746:0:203405:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 00000000553d6f21. 00040000:00000010:2.0:1788880172.748750:0:203405:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000273da661. 00800000:00000001:2.0:1788880172.748753:0:203405:0:(lmv_obd.c:1247:lmv_device_alloc()) Process entered 00800000:00000010:2.0:1788880172.748755:0:203405:0:(lmv_obd.c:1249:lmv_device_alloc()) kmalloced '(lu)': 64 at 000000007961238e. 80000000:00000001:2.0:1788880172.748763:0:203405:0:(fld_request.c:235:fld_client_init()) Process entered 80000000:00000001:2.0:1788880172.748766:0:203405:0:(fld_cache.c:42:fld_cache_init()) Process entered 80000000:00000010:2.0:1788880172.748769:0:203405:0:(fld_cache.c:47:fld_cache_init()) kmalloced '(cache)': 168 at 000000003d92e067. 80000000:00000040:2.0:1788880172.748776:0:203405:0:(fld_cache.c:66:fld_cache_init()) cli-lustre-clilmv-ffff8ca787527000: FLD cache - Size: 18724, Threshold: 1872 80000000:00000001:2.0:1788880172.748778:0:203405:0:(fld_cache.c:68:fld_cache_init()) Process leaving (rc=18446617250282255872 : -126823427295744 : ffff8ca79eaab600) 80000000:00000001:2.0:1788880172.748780:0:203405:0:(fld_request.c:210:fld_client_debugfs_init()) Process entered 80000000:00000001:2.0:1788880172.748834:0:203405:0:(fld_request.c:265:fld_client_init()) Process leaving 80000000:00000040:2.0:1788880172.748836:0:203405:0:(fld_request.c:271:fld_client_init()) cli-lustre-clilmv-ffff8ca787527000: Using "RRB" hash 00800000:00000010:2.0:1788880172.748844:0:203405:0:(lmv_obd.c:1305:lmv_device_alloc()) kmalloced '(pat)': 280 at 00000000d8a2e56b. 00800000:00000010:2.0:1788880172.748849:0:203405:0:(lmv_obd.c:1314:lmv_device_alloc()) kmalloced '(pat)': 280 at 00000000991ecca6. 00000020:00000010:2.0:1788880172.748999:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 000000005dd33871. 00000020:00000010:2.0:1788880172.749004:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 512 at 000000005ee0d4e3. 00800000:00000001:2.0:1788880172.749025:0:203405:0:(lmv_obd.c:1327:lmv_device_alloc()) Process leaving (rc=18446617250282350656 : -126823427200960 : ffff8ca79eac2840) 00000020:00000010:2.0:1788880172.749039:0:203405:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 0000000064b136ca. 00000040:00000010:2.0:1788880172.749046:0:203405:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 000000000c9a81c5. 00000020:00000010:2.0:1788880172.749049:0:203405:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 0000000012119a99. 00000020:00000010:2.0:1788880172.749053:0:203405:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 00000000dadfe130. 00040000:00000010:2.0:1788880172.749058:0:203405:0:(lquota_lib.c:35:lquota_key_fini()) kfreed 'info': 248 at 00000000553d6f21. 00040000:00000010:2.0:1788880172.749061:0:203405:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 00000000273da661. 00000020:00000010:2.0:1788880172.749065:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000005842920. 00000020:00000010:2.0:1788880172.749069:0:203405:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 00000000a9570094. 00000020:00000010:2.0:1788880172.749072:0:203405:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 000000006de8747c. 00000008:00000010:2.0:1788880172.749075:0:203405:0:(osc_dev.c:125:osc_session_fini()) slab-freed '(info)': 352 at 00000000ed7da6b9. 00020000:00000010:2.0:1788880172.749078:0:203405:0:(lov_dev.c:136:lov_session_key_fini()) slab-freed '(info)': 584 at 00000000921e4385. 00008000:00000010:2.0:1788880172.749080:0:203405:0:(echo_client.c:449:echo_session_key_fini()) slab-freed '(session)': 8 at 000000009f2e63a1. 00000080:00000010:2.0:1788880172.749083:0:203405:0:(vvp_dev.c:105:vvp_session_key_fini()) slab-freed '(session)': 240 at 0000000014012704. 00000020:00000010:2.0:1788880172.749085:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000e6a93e02. 00000020:00000001:2.0:1788880172.749089:0:203405:0:(obd_class.h:601:obd_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880172.749091:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-clilmv-ffff8ca787527000 (00000000aeea1d9b) now 3 - setup 00000020:00000080:2.0:1788880172.749093:0:203405:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-clilmv-ffff8ca787527000 (uuid ea27006f-52a8-4007-b69a-dcac8279445a) 00000020:00000001:2.0:1788880172.749095:0:203405:0:(obd_config.c:828:class_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.749097:0:203405:0:(obd_config.c:1618:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.749099:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 160 at 00000000db63a97d. 00000020:00000010:2.0:1788880172.749100:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 34 at 00000000bfcc82cb. 00000020:00000001:2.0:1788880172.749101:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.749109:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf24e0 type=0x10620000 idx=8 00000040:00001000:2.0:1788880172.749110:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 8 lrh_len: 224 (6944 remains) 00000040:00001000:2.0:1788880172.749112:0:203405:0:(llog.c:805:llog_process_thread()) index: 8, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.749114:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:2.0:1788880172.749116:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:2.0:1788880172.749117:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:2.0:1788880172.749118:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000010:2.0:1788880172.749120:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 00000000553d6f21. 00000020:00000080:2.0:1788880172.749122:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1788880172.749123:0:203405:0:(obd_config.c:1561:class_process_config()) marker 5 (0x2) lustre-clilmv lmv setup 00000020:00000001:2.0:1788880172.749124:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.749125:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 00000000553d6f21. 00000020:00000001:2.0:1788880172.749127:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.749128:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf25c0 type=0x10620000 idx=9 00000040:00001000:2.0:1788880172.749130:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 9 lrh_len: 224 (6720 remains) 00000040:00001000:2.0:1788880172.749130:0:203405:0:(llog.c:805:llog_process_thread()) index: 9, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.749131:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:2.0:1788880172.749132:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:2.0:1788880172.749132:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:2.0:1788880172.749133:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000001:2.0:1788880172.749134:0:203405:0:(obd_mount.c:953:lustre_check_exclusion()) Process entered 00000020:00000001:2.0:1788880172.749135:0:203405:0:(obd_mount.c:958:lustre_check_exclusion()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880172.749136:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 00000000553d6f21. 00000020:00000080:2.0:1788880172.749137:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1788880172.749138:0:203405:0:(obd_config.c:1561:class_process_config()) marker 6 (0x1) lustre-client mount opts 00000020:00000001:2.0:1788880172.749139:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.749140:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 00000000553d6f21. 00000020:00000001:2.0:1788880172.749145:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.749146:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf26a0 type=0x10620000 idx=10 00000040:00001000:2.0:1788880172.749146:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 10 lrh_len: 120 (6496 remains) 00000040:00001000:2.0:1788880172.749147:0:203405:0:(llog.c:805:llog_process_thread()) index: 10, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.749149:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:2.0:1788880172.749151:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 96 at 000000007ff7ab1f. 00000020:00000080:2.0:1788880172.749152:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf007 00000020:00000080:2.0:1788880172.749159:0:203405:0:(obd_config.c:1514:class_process_config()) mountopt: profile lustre-client osc lustre-clilov mdc lustre-clilmv 00000020:00000001:2.0:1788880172.749162:0:203405:0:(obd_config.c:1146:class_add_profile()) Process entered 00000020:01000000:2.0:1788880172.749164:0:203405:0:(obd_config.c:1148:class_add_profile()) Add profile lustre-client 00000020:00000010:2.0:1788880172.749167:0:203405:0:(obd_config.c:1150:class_add_profile()) kmalloced '(lprof)': 48 at 00000000bfcc82cb. 00000020:00000010:2.0:1788880172.749170:0:203405:0:(obd_config.c:1156:class_add_profile()) kmalloced '(lprof->lp_profile)': 14 at 000000001eab4064. 00000020:00000010:2.0:1788880172.749173:0:203405:0:(obd_config.c:1162:class_add_profile()) kmalloced '(lprof->lp_dt)': 14 at 000000001fb298d5. 00000020:00000010:2.0:1788880172.749175:0:203405:0:(obd_config.c:1169:class_add_profile()) kmalloced '(lprof->lp_md)': 14 at 000000000dbf06c2. 00000020:00000001:2.0:1788880172.749177:0:203405:0:(obd_config.c:1181:class_add_profile()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.749178:0:203405:0:(obd_config.c:1525:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.749179:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 96 at 000000007ff7ab1f. 00000020:00000001:2.0:1788880172.749181:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.749182:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2718 type=0x10620000 idx=11 00000040:00001000:2.0:1788880172.749184:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 11 lrh_len: 224 (6376 remains) 00000040:00001000:2.0:1788880172.749185:0:203405:0:(llog.c:805:llog_process_thread()) index: 11, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.749186:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:2.0:1788880172.749187:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:2.0:1788880172.749189:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:2.0:1788880172.749190:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000010:2.0:1788880172.749194:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 00000000553d6f21. 00000020:00000080:2.0:1788880172.749196:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1788880172.749197:0:203405:0:(obd_config.c:1561:class_process_config()) marker 6 (0x2) lustre-client mount opts 00000020:00000001:2.0:1788880172.749199:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.749200:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 00000000553d6f21. 00000020:00000001:2.0:1788880172.749204:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.749205:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf27f8 type=0x10620000 idx=12 00000040:00001000:2.0:1788880172.749207:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 12 lrh_len: 224 (6152 remains) 00000040:00001000:2.0:1788880172.749209:0:203405:0:(llog.c:805:llog_process_thread()) index: 12, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.749210:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:2.0:1788880172.749213:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:2.0:1788880172.749214:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:2.0:1788880172.749215:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000001:2.0:1788880172.749220:0:203405:0:(obd_mount.c:953:lustre_check_exclusion()) Process entered 00000020:00000001:2.0:1788880172.749222:0:203405:0:(obd_mount.c:958:lustre_check_exclusion()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880172.749226:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 00000000553d6f21. 00000020:00000080:2.0:1788880172.749287:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1788880172.749291:0:203405:0:(obd_config.c:1561:class_process_config()) marker 7 (0x1) lustre-MDT0000 add mdc 00000020:00000001:2.0:1788880172.749294:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.749296:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 00000000553d6f21. 00000020:00000001:2.0:1788880172.749299:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.749300:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf28d8 type=0x10620000 idx=13 00000040:00001000:2.0:1788880172.749302:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 13 lrh_len: 88 (5928 remains) 00000100:00000040:0.0:1788880172.749304:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305803854, at 4305803855 00000040:00001000:2.0:1788880172.749305:0:203405:0:(llog.c:805:llog_process_thread()) index: 13, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.749317:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000100:00000010:0.0:1788880172.749324:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000040:2.0:1788880172.749336:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305803854, at 4305803855 00000100:00000010:2.0:1788880172.749343:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000020:00000010:2.0:1788880172.749357:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 64 at 00000000b68d2909. 00000020:00000080:2.0:1788880172.749359:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf005 00000020:00000080:2.0:1788880172.749363:0:203405:0:(obd_config.c:1485:class_process_config()) adding mapping from uuid 192.168.204.111@tcp to nid 0x20000c0a8cc6f (192.168.204.111@tcp) 00000100:00000010:2.0:1788880172.749368:0:203405:0:(lustre_peer.c:90:class_add_uuid()) kmalloced '(data)': 704 at 0000000075dcb836. 00000100:00000040:2.0:1788880172.749371:0:203405:0:(lustre_peer.c:130:class_add_uuid()) found uuid 192.168.204.111@tcp 192.168.204.111@tcp cnt=1 00000400:00000200:2.0:1788880172.749374:0:203405:0:(peer.c:1363:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:2.0:1788880172.749377:0:203405:0:(api-ni.c:3274:LNetNIInit()) refs 1 00000400:00000200:2.0:1788880172.749380:0:203405:0:(peer.c:1377:LNetAddPeer()) Adding NID[0]: 192.168.204.111@tcp 00000400:00000200:2.0:1788880172.749385:0:203405:0:(peer.c:1822:lnet_peer_add()) peer 192.168.204.111@tcp NID flags 0x100001: -17 00000400:00000200:2.0:1788880172.749399:0:203405:0:(peer.c:2890:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:2.0:1788880172.749402:0:203405:0:(peer.c:2899:lnet_discover_peer_locked()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp: 0. pending discovery 00000400:00000200:2.0:1788880172.749405:0:203405:0:(peer.c:1431:LNetAddPeer()) A peer exists for NID 192.168.204.111@tcp 00000100:00000010:2.0:1788880172.749407:0:203405:0:(lustre_peer.c:132:class_add_uuid()) kfreed 'data': 704 at 0000000075dcb836. 00000020:00000001:2.0:1788880172.749409:0:203405:0:(obd_config.c:1500:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.749411:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 64 at 00000000b68d2909. 00000020:00000001:2.0:1788880172.749414:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.749415:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2930 type=0x10620000 idx=14 00000040:00001000:2.0:1788880172.749418:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 14 lrh_len: 128 (5840 remains) 00000040:00001000:2.0:1788880172.749419:0:203405:0:(llog.c:805:llog_process_thread()) index: 14, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.749422:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:2.0:1788880172.749424:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 39 at 00000000b68d2909. 00000020:01000000:2.0:1788880172.749426:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf001, instance name: lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000010:2.0:1788880172.749436:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 136 at 00000000db63a97d. 00000020:00000080:2.0:1788880172.749437:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf001 00000020:00000001:2.0:1788880172.749438:0:203405:0:(obd_config.c:683:class_attach_name()) Process entered 00000020:00000001:2.0:1788880172.749439:0:203405:0:(genops.c:350:class_newdev()) Process entered 00000020:00000010:2.0:1788880172.749446:0:203405:0:(genops.c:51:obd_device_alloc()) slab-alloced 'obd': 6592 at 000000009cb5873b. 00000020:00000080:2.0:1788880172.749448:0:203405:0:(genops.c:413:class_newdev()) Allocate new device lustre-MDT0000-mdc-ffff8ca787527000 (000000009cb5873b) 00000020:00000001:2.0:1788880172.749450:0:203405:0:(genops.c:952:__class_new_export()) Process entered 00000020:00000010:2.0:1788880172.749452:0:203405:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 0000000012119a99. 00000020:00000001:2.0:1788880172.749453:0:203405:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880172.749454:0:203405:0:(lustre_handles.c:93:class_handle_hash()) added object 0000000012119a99 with handle 0x805d548a36a88674 to hash 00000020:00000001:2.0:1788880172.749456:0:203405:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:2.0:1788880172.749458:0:203405:0:(obd_class.h:1016:obd_init_export()) Process entered 00000020:00000001:2.0:1788880172.749459:0:203405:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.749461:0:203405:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617249948780544 : -126823760771072 : ffff8ca78aca4800) 00000020:00000040:2.0:1788880172.749463:0:203405:0:(genops.c:900:class_export_put()) PUTting export 0000000012119a99 : new refcount 1 00000020:00000040:2.0:1788880172.749467:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-MDT0000-mdc-ffff8ca787527000 (000000009cb5873b) now 2 - obd_device_list 00000020:00000001:2.0:1788880172.749470:0:203405:0:(genops.c:555:class_register_device()) Process leaving (rc=0 : 0 : 0) 00000020:00000080:2.0:1788880172.749472:0:203405:0:(obd_config.c:720:class_attach_name()) OBD: dev 3 attached type mdc with refcount 2 00000020:00000001:2.0:1788880172.749473:0:203405:0:(obd_config.c:722:class_attach_name()) Process leaving (rc=18446617250653064512 : -126823056487104 : ffff8ca7b4c4cd40) 00000020:00000001:2.0:1788880172.749475:0:203405:0:(obd_config.c:1479:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880172.749476:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 136 at 00000000db63a97d. 00000020:00000010:2.0:1788880172.749479:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 39 at 00000000b68d2909. 00000020:00000001:2.0:1788880172.749480:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880172.749482:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf29b0 type=0x10620000 idx=15 00000040:00001000:2.0:1788880172.749484:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 15 lrh_len: 144 (5712 remains) 00000040:00001000:2.0:1788880172.749485:0:203405:0:(llog.c:805:llog_process_thread()) index: 15, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:2.0:1788880172.749487:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:2.0:1788880172.749490:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 39 at 00000000b68d2909. 00000020:01000000:2.0:1788880172.749492:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf003, instance name: lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000010:2.0:1788880172.749494:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 136 at 00000000db63a97d. 00000020:00000080:2.0:1788880172.749496:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf003 00000020:00000001:2.0:1788880172.749499:0:203405:0:(obd_config.c:742:class_setup()) Process entered 00000001:00000001:2.0:1788880172.749537:0:203405:0:(hash.c:1038:cfs_hash_create()) Process entered 00000001:00000010:2.0:1788880172.749540:0:203405:0:(hash.c:1065:cfs_hash_create()) alloc '(hs)': 232 at 00000000553d6f21. 00000001:00000010:2.0:1788880172.749544:0:203405:0:(hash.c:933:cfs_hash_buckets_realloc()) alloc '(new_bkts)': 32 at 000000009d30c7f7. 00000001:00000010:2.0:1788880172.749548:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 000000009ad603c4. 00000001:00000010:2.0:1788880172.749552:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 000000002de0832d. 00000001:00000010:2.0:1788880172.749556:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 00000000f206b040. 00000001:00000010:2.0:1788880172.749560:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000093779311. 00000020:00000001:2.0:1788880172.749562:0:203405:0:(obd_class.h:565:obd_setup()) Process entered 00000020:00000010:2.0:1788880172.749566:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000c7b4de5d. 00000020:00000010:2.0:1788880172.749571:0:203405:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 00000000a9570094. 00000020:00000010:2.0:1788880172.749575:0:203405:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 000000006de8747c. 00000008:00000010:2.0:1788880172.749578:0:203405:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000ed7da6b9. 00020000:00000010:2.0:1788880172.749581:0:203405:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 00000000921e4385. 00008000:00000010:2.0:1788880172.749584:0:203405:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 000000009f2e63a1. 00000080:00000010:2.0:1788880172.749588:0:203405:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 0000000014012704. 00000020:00000010:2.0:1788880172.749593:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000bd76ed56. 00000020:00000010:2.0:1788880172.749597:0:203405:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 0000000024c0dace. 00000020:00000010:2.0:1788880172.749601:0:203405:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 000000007ff7ab1f. 00000008:00000010:2.0:1788880172.749607:0:203405:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000218bf9e7. 00020000:00000010:2.0:1788880172.749611:0:203405:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 000000003997086d. 00008000:00000010:2.0:1788880172.749614:0:203405:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000e8840d53. 00000080:00000010:2.0:1788880172.749618:0:203405:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 00000000d35adf78. 00000080:00000010:2.0:1788880172.749623:0:203405:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 0000000034e14fab. 00000002:00000010:2.0:1788880172.749629:0:203405:0:(mdc_dev.c:1727:mdc_device_alloc()) kmalloced '(osc)': 104 at 0000000097c1dfe7. 00000002:00000001:2.0:1788880172.749636:0:203405:0:(mdc_request.c:3051:mdc_setup()) Process entered 00000008:00000001:2.0:1788880172.749639:0:203405:0:(osc_request.c:4061:osc_setup_common()) Process entered 00000100:00000001:2.0:1788880172.749642:0:203405:0:(ptlrpcd.c:908:ptlrpcd_addref()) Process entered 00000100:00000001:2.0:1788880172.749645:0:203405:0:(ptlrpcd.c:917:ptlrpcd_addref()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880172.749649:0:203405:0:(ldlm_lib.c:297:client_obd_setup()) Process entered 00010000:00000010:2.0:1788880172.749659:0:203405:0:(ldlm_lib.c:420:client_obd_setup()) kmalloced '(cli->cl_read_io_latency_by_size)': 3920 at 000000009b228e79. 00010000:00000010:2.0:1788880172.749666:0:203405:0:(ldlm_lib.c:433:client_obd_setup()) kmalloced '(cli->cl_write_io_latency_by_size)': 3920 at 000000009a14e474. 00010000:00000010:2.0:1788880172.749677:0:203405:0:(ldlm_lib.c:522:client_obd_setup()) kmalloced '(cli->cl_mod_tag_bitmap)': 64 at 00000000adc95b12. 00010000:00000001:2.0:1788880172.749679:0:203405:0:(ldlm_lockd.c:3036:ldlm_get_ref()) Process entered 00010000:00000001:2.0:1788880172.749681:0:203405:0:(ldlm_lockd.c:3045:ldlm_get_ref()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880172.749691:0:203405:0:(genops.c:1285:class_new_import()) kmalloced '(imp)': 1720 at 0000000095473c20. 00000020:00000040:2.0:1788880172.749693:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-MDT0000-mdc-ffff8ca787527000 (000000009cb5873b) now 3 - import 00000020:00000001:2.0:1788880172.749700:0:203405:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880172.749701:0:203405:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=1 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880172.749703:0:203405:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:2.0:1788880172.749705:0:203405:0:(ldlm_lib.c:49:import_set_conn()) Process entered 00000100:00000200:2.0:1788880172.749717:0:203405:0:(events.c:559:ptlrpc_uuid_to_peer()) 192.168.204.111@tcp->12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880172.749719:0:203405:0:(connection.c:104:ptlrpc_connection_get()) Process entered 00000400:00000200:2.0:1788880172.749724:0:203405:0:(peer.c:1550:LNetPrimaryNID()) NID 192.168.204.111@tcp primary NID 192.168.204.111@tcp rc 0 00000100:00000001:2.0:1788880172.749727:0:203405:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880172.749730:0:203405:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 4 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880172.749732:0:203405:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000001:2.0:1788880172.749734:0:203405:0:(connection.c:110:ptlrpc_connection_get()) Process leaving via out (rc=18446617250405033856 : -126823304517760 : 0xffff8ca7a5fc2780) 00000100:00000040:2.0:1788880172.749736:0:203405:0:(connection.c:149:ptlrpc_connection_get()) conn=000000008a5be7d5 refcount 4 to 192.168.204.111@tcp 00000100:00000040:2.0:1788880172.749738:0:203405:0:(client.c:137:ptlrpc_uuid_to_connection()) 192.168.204.111@tcp -> 000000008a5be7d5 00010000:00000010:2.0:1788880172.749740:0:203405:0:(ldlm_lib.c:70:import_set_conn()) kmalloced '(imp_conn)': 88 at 0000000095cab57e. 00010000:00080000:2.0:1788880172.749742:0:203405:0:(ldlm_lib.c:102:import_set_conn()) imp 0000000095473c20@lustre-MDT0000-mdc-ffff8ca787527000: add connection 192.168.204.111@tcp at head 00010000:00000001:2.0:1788880172.749744:0:203405:0:(ldlm_lib.c:109:import_set_conn()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880172.749746:0:203405:0:(ldlm_resource.c:976:ldlm_namespace_new()) Process entered 00010000:00000001:2.0:1788880172.749747:0:203405:0:(ldlm_lockd.c:3036:ldlm_get_ref()) Process entered 00010000:00000001:2.0:1788880172.749749:0:203405:0:(ldlm_lockd.c:3045:ldlm_get_ref()) Process leaving (rc=0 : 0 : 0) 00010000:00000010:2.0:1788880172.749752:0:203405:0:(ldlm_resource.c:993:ldlm_namespace_new()) kmalloced '(ns)': 848 at 00000000c56413c7. 00000001:00000001:2.0:1788880172.749754:0:203405:0:(hash.c:1038:cfs_hash_create()) Process entered 00000001:00000010:2.0:1788880172.749758:0:203405:0:(hash.c:1065:cfs_hash_create()) alloc '(hs)': 280 at 00000000182548a9. 00000001:00000010:2.0:1788880172.749762:0:203405:0:(hash.c:933:cfs_hash_buckets_realloc()) alloc '(new_bkts)': 256 at 0000000053a79bce. 00000001:00000010:2.0:1788880172.749834:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000006fd194f0. 00000001:00000010:2.0:1788880172.749908:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000004c550e75. 00000001:00000010:2.0:1788880172.749981:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000078993ea2. 00000001:00000010:2.0:1788880172.750046:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000005dcea311. 00000001:00000010:2.0:1788880172.750110:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000477d2be9. 00000001:00000010:2.0:1788880172.750192:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000000ebebdb7. 00000001:00000010:2.0:1788880172.750313:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000027e7d054. 00000001:00000010:2.0:1788880172.750389:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000e0d9b14d. 00000001:00000010:2.0:1788880172.750445:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000ef89d9d2. 00000001:00000010:2.0:1788880172.750544:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000049fe4696. 00000001:00000010:2.0:1788880172.750631:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000cde2cce0. 00000100:00000040:3.0:1788880172.751123:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305803854, at 4305803856 00000100:00000010:3.0:1788880172.751131:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000001:00000010:3.0:1788880172.751163:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000ad235f6e. 00000001:00000010:3.0:1788880172.751219:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000003c2094a0. 00000001:00000010:3.0:1788880172.751303:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000002b4e11b6. 00000001:00000010:3.0:1788880172.751351:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000006100f7c3. 00000001:00000010:3.0:1788880172.751402:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000032617a43. 00000001:00000010:3.0:1788880172.751458:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000002f7e780. 00000001:00000010:3.0:1788880172.751501:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000093f91764. 00000001:00000010:3.0:1788880172.751555:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000b1c6810f. 00000001:00000010:3.0:1788880172.751602:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000bc2a09f4. 00000001:00000010:3.0:1788880172.751654:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000dc55ecfa. 00000001:00000010:3.0:1788880172.751719:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000086c91ec5. 00000001:00000010:3.0:1788880172.751781:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000b81f1b53. 00000001:00000010:3.0:1788880172.751856:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000001d391381. 00000001:00000010:3.0:1788880172.751912:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000072cad877. 00000001:00000010:3.0:1788880172.751972:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000753c99a9. 00000001:00000010:3.0:1788880172.752013:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000009f454f1a. 00000001:00000010:3.0:1788880172.752080:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000061c70119. 00000001:00000010:3.0:1788880172.752120:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000ed29cab6. 00000001:00000010:3.0:1788880172.752157:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000000a9febcf. 00000001:00000010:3.0:1788880172.752195:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000d4eee944. 00000001:00000010:3.0:1788880172.752269:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000f78cf3f6. 00000020:00000010:3.0:1788880172.752298:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 000000005bd0a08a. 00000020:00000010:3.0:1788880172.752303:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 96 at 000000005c6d5eb7. 00010000:00000001:3.0:1788880172.752369:0:203405:0:(ldlm_pool.c:891:ldlm_pool_init()) Process entered 00010000:00000001:3.0:1788880172.752404:0:203405:0:(ldlm_pool.c:810:ldlm_pool_debugfs_init()) Process entered 00000020:00000010:3.0:1788880172.752423:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 00000000ca692e4d. 00000020:00000010:3.0:1788880172.752428:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 352 at 000000002b843fc3. 00010000:00000001:3.0:1788880172.752444:0:203405:0:(ldlm_pool.c:865:ldlm_pool_debugfs_init()) Process leaving 00010000:00010000:3.0:1788880172.752446:0:203405:0:(ldlm_pool.c:925:ldlm_pool_init()) Lock pool ldlm-pool-lustre-MDT0000-mdc-ffff8ca787527000-1 is initialized 00010000:00000001:3.0:1788880172.752448:0:203405:0:(ldlm_pool.c:927:ldlm_pool_init()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880172.752452:0:203405:0:(ldlm_resource.c:1084:ldlm_namespace_new()) Process leaving (rc=18446617250098533376 : -126823611018240 : ffff8ca793b75400) 00010000:00000001:3.0:1788880172.752457:0:203405:0:(ldlm_lib.c:603:client_obd_setup()) Process leaving (rc=0 : 0 : 0) 00000008:00000020:3.0:1788880172.752464:0:203405:0:(osc_request.c:745:osc_update_next_shrink()) next time 12336 to shrink grant 00000008:00000001:3.0:1788880172.752466:0:203405:0:(osc_request.c:4083:osc_setup_common()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.752693:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 00000000dcc601e2. 00000020:00000010:3.0:1788880172.752698:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 512 at 00000000fb93ca9a. 00000020:00000010:3.0:1788880172.752747:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 000000003af4b077. 00000020:00000010:3.0:1788880172.752751:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 3680 at 000000004c6fe24c. 00000002:00000001:3.0:1788880172.752777:0:203405:0:(mdc_request.c:3022:mdc_llog_init()) Process entered 00000040:00000001:3.0:1788880172.752780:0:203405:0:(llog_obd.c:135:llog_setup()) Process entered 00000040:00000010:3.0:1788880172.752785:0:203405:0:(llog_obd.c:26:llog_new_ctxt()) kmalloced '(ctxt)': 168 at 000000008bb9ec9d. 00000020:00000040:3.0:1788880172.752789:0:203405:0:(genops.c:889:class_export_get()) GET export 0000000012119a99 refcount=2 00000040:01000000:3.0:1788880172.752792:0:203405:0:(llog_obd.c:197:llog_setup()) obd lustre-MDT0000-mdc-ffff8ca787527000 ctxt 13 is initialized 00000040:00000001:3.0:1788880172.752795:0:203405:0:(llog_obd.c:201:llog_setup()) Process leaving (rc=0 : 0 : 0) 00000002:00000040:3.0:1788880172.752806:0:203405:0:(lustre_log.h:387:llog_ctxt_get()) GETting ctxt 000000008bb9ec9d : new refcount 2 00000040:00000001:3.0:1788880172.752808:0:203405:0:(llog_net.c:31:llog_initiator_connect()) Process entered 00000020:00000040:3.0:1788880172.752810:0:203405:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=2 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000040:00000001:3.0:1788880172.752811:0:203405:0:(llog_net.c:44:llog_initiator_connect()) Process leaving (rc=0 : 0 : 0) 00000002:00000040:3.0:1788880172.752812:0:203405:0:(lustre_log.h:402:llog_ctxt_put()) PUTting ctxt 000000008bb9ec9d : new refcount 1 00000002:00000001:3.0:1788880172.752814:0:203405:0:(mdc_request.c:3032:mdc_llog_init()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880172.752816:0:203405:0:(mdc_changelog.c:929:mdc_changelog_cdev_init()) Process entered 00000002:00000010:3.0:1788880172.752820:0:203405:0:(mdc_changelog.c:931:mdc_changelog_cdev_init()) kmalloced '(entry)': 1304 at 00000000edcd788c. 00000002:00000001:3.0:1788880172.753124:0:203405:0:(mdc_changelog.c:974:mdc_changelog_cdev_init()) Process leaving via out_unlock (rc=0 : 0 : 0x0) 00000002:00000001:3.0:1788880172.753129:0:203405:0:(mdc_changelog.c:989:mdc_changelog_cdev_init()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880172.753131:0:203405:0:(mdc_request.c:3081:mdc_setup()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880172.753134:0:203405:0:(mdc_dev.c:1752:mdc_device_alloc()) Process leaving (rc=18446617250283047424 : -126823426504192 : ffff8ca79eb6ca00) 00000002:00000001:3.0:1788880172.753138:0:203405:0:(mdc_dev.c:1758:mdc_device_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.753144:0:203405:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 0000000024c0dace. 00000020:00000010:3.0:1788880172.753151:0:203405:0:(cl_object.c:1137:cl_key_fini()) kfreed 'info': 72 at 000000007ff7ab1f. 00000008:00000010:3.0:1788880172.753160:0:203405:0:(osc_dev.c:100:osc_key_fini()) slab-freed '(info)': 2864 at 00000000218bf9e7. 00020000:00000010:3.0:1788880172.753167:0:203405:0:(lov_dev.c:95:lov_key_fini()) slab-freed '(info)': 192 at 000000003997086d. 00008000:00000010:3.0:1788880172.753175:0:203405:0:(echo_client.c:424:echo_thread_key_fini()) slab-freed '(info)': 904 at 00000000e8840d53. 00000080:00000010:3.0:1788880172.753181:0:203405:0:(vvp_dev.c:80:ll_thread_key_fini()) slab-freed '(lti)': 168 at 00000000d35adf78. 00000080:00000010:3.0:1788880172.753187:0:203405:0:(vvp_dev.c:130:vvp_thread_key_fini()) slab-freed '(vti)': 592 at 0000000034e14fab. 00000020:00000010:3.0:1788880172.753192:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000bd76ed56. 00000020:00000010:3.0:1788880172.753198:0:203405:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 00000000a9570094. 00000020:00000010:3.0:1788880172.753203:0:203405:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 000000006de8747c. 00000008:00000010:3.0:1788880172.753208:0:203405:0:(osc_dev.c:125:osc_session_fini()) slab-freed '(info)': 352 at 00000000ed7da6b9. 00020000:00000010:3.0:1788880172.753215:0:203405:0:(lov_dev.c:136:lov_session_key_fini()) slab-freed '(info)': 584 at 00000000921e4385. 00008000:00000010:3.0:1788880172.753219:0:203405:0:(echo_client.c:449:echo_session_key_fini()) slab-freed '(session)': 8 at 000000009f2e63a1. 00000080:00000010:3.0:1788880172.753223:0:203405:0:(vvp_dev.c:105:vvp_session_key_fini()) slab-freed '(session)': 240 at 0000000014012704. 00000020:00000010:3.0:1788880172.753228:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000c7b4de5d. 00000020:00000001:3.0:1788880172.753288:0:203405:0:(obd_class.h:601:obd_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880172.753294:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-MDT0000-mdc-ffff8ca787527000 (000000009cb5873b) now 4 - setup 00000020:00000080:3.0:1788880172.753298:0:203405:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-MDT0000-mdc-ffff8ca787527000 (uuid ea27006f-52a8-4007-b69a-dcac8279445a) 00000020:00000001:3.0:1788880172.753301:0:203405:0:(obd_config.c:828:class_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.753304:0:203405:0:(obd_config.c:1618:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.753315:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 136 at 00000000db63a97d. 00000020:00000010:3.0:1788880172.753319:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 39 at 00000000b68d2909. 00000020:00000001:3.0:1788880172.753323:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.753326:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2a40 type=0x10620000 idx=16 00000040:00001000:3.0:1788880172.753331:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 16 lrh_len: 160 (5568 remains) 00000040:00001000:3.0:1788880172.753334:0:203405:0:(llog.c:805:llog_process_thread()) index: 16, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.753338:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:3.0:1788880172.753364:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 34 at 00000000e7da380a. 00000020:01000000:3.0:1788880172.753368:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf014, instance name: lustre-clilmv-ffff8ca787527000 00000020:00000010:3.0:1788880172.753372:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 152 at 0000000047bb7169. 00000020:00000080:3.0:1788880172.753375:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf014 00000020:00000001:3.0:1788880172.753381:0:203405:0:(obd_class.h:689:obd_process_config()) Process entered 00000020:00000010:3.0:1788880172.753386:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000e49153de. 00000020:00000010:3.0:1788880172.753390:0:203405:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000fe6d4325. 00000040:00000010:3.0:1788880172.753394:0:203405:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 0000000003422132. 00000020:00000010:3.0:1788880172.753398:0:203405:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 000000004a9f1dd1. 00000020:00000010:3.0:1788880172.753403:0:203405:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 000000005fb6640b. 00040000:00000010:3.0:1788880172.753408:0:203405:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 000000005c064165. 00040000:00000010:3.0:1788880172.753413:0:203405:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000b05735d2. 00800000:00000001:3.0:1788880172.753417:0:203405:0:(lmv_obd.c:1377:lmv_process_config()) Process entered 00800000:00000001:3.0:1788880172.753423:0:203405:0:(lmv_obd.c:437:lmv_add_target()) Process entered 00800000:01000000:3.0:1788880172.753425:0:203405:0:(lmv_obd.c:439:lmv_add_target()) Target uuid: lustre-MDT0000_UUID. index 0 00800000:00000010:3.0:1788880172.753433:0:203405:0:(lmv_obd.c:448:lmv_add_target()) kmalloced '(tgt)': 368 at 00000000c7b4de5d. 00020000:00000001:3.0:1788880172.753436:0:203405:0:(lu_tgt_descs.c:352:ltd_add_tgt()) Process entered 00020000:00000010:3.0:1788880172.753439:0:203405:0:(lu_tgt_descs.c:371:ltd_add_tgt()) kmalloced '(ltd->ltd_tgt_idx[index / (((1UL) << 12) / sizeof(void *))])': 4096 at 00000000921c3f50. 00020000:00000001:3.0:1788880172.753442:0:203405:0:(lu_tgt_descs.c:383:ltd_add_tgt()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:3.0:1788880172.753444:0:203405:0:(lmv_obd.c:466:lmv_add_target()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:3.0:1788880172.753446:0:203405:0:(lmv_obd.c:1394:lmv_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00800000:00000001:3.0:1788880172.753448:0:203405:0:(lmv_obd.c:1400:lmv_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.753451:0:203405:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000fe6d4325. 00000040:00000010:3.0:1788880172.753454:0:203405:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 0000000003422132. 00000020:00000010:3.0:1788880172.753455:0:203405:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 000000004a9f1dd1. 00000020:00000010:3.0:1788880172.753458:0:203405:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 000000005fb6640b. 00040000:00000010:3.0:1788880172.753467:0:203405:0:(lquota_lib.c:35:lquota_key_fini()) kfreed 'info': 248 at 000000005c064165. 00040000:00000010:3.0:1788880172.753469:0:203405:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 00000000b05735d2. 00000020:00000010:3.0:1788880172.753470:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000e49153de. 00000020:00000001:3.0:1788880172.753473:0:203405:0:(obd_class.h:704:obd_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.753476:0:203405:0:(obd_class.h:689:obd_process_config()) Process entered 00000020:00000010:3.0:1788880172.753478:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000bd76ed56. 00000020:00000010:3.0:1788880172.753481:0:203405:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 0000000024c0dace. 00000020:00000010:3.0:1788880172.753484:0:203405:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000f6896c3f. 00000008:00000010:3.0:1788880172.753488:0:203405:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000aeeba8e2. 00020000:00000010:3.0:1788880172.753491:0:203405:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000b8c7c243. 00008000:00000010:3.0:1788880172.753494:0:203405:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000d0b9f970. 00000080:00000010:3.0:1788880172.753498:0:203405:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 0000000020ae732f. 00000080:00000010:3.0:1788880172.753501:0:203405:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 000000007ec0a7c6. 00020000:00000001:3.0:1788880172.753506:0:203405:0:(lov_obd.c:814:lov_process_config_base()) Process entered 00020000:00000001:3.0:1788880172.753508:0:203405:0:(lov_obd.c:868:lov_process_config_base()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:3.0:1788880172.753510:0:203405:0:(lov_dev.c:450:lov_add_mdc_target()) Process entered 00000020:00000040:3.0:1788880172.753512:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-clilmv-ffff8ca787527000 (00000000aeea1d9b) now 4 - lov 00020000:00000001:3.0:1788880172.753514:0:203405:0:(lov_dev.c:507:lov_add_mdc_target()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:3.0:1788880172.753517:0:203405:0:(lov_dev.c:573:lov_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.753518:0:203405:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 0000000024c0dace. 00000020:00000010:3.0:1788880172.753521:0:203405:0:(cl_object.c:1137:cl_key_fini()) kfreed 'info': 72 at 00000000f6896c3f. 00000008:00000010:3.0:1788880172.753524:0:203405:0:(osc_dev.c:100:osc_key_fini()) slab-freed '(info)': 2864 at 00000000aeeba8e2. 00020000:00000010:3.0:1788880172.753528:0:203405:0:(lov_dev.c:95:lov_key_fini()) slab-freed '(info)': 192 at 00000000b8c7c243. 00008000:00000010:3.0:1788880172.753530:0:203405:0:(echo_client.c:424:echo_thread_key_fini()) slab-freed '(info)': 904 at 00000000d0b9f970. 00000080:00000010:3.0:1788880172.753535:0:203405:0:(vvp_dev.c:80:ll_thread_key_fini()) slab-freed '(lti)': 168 at 0000000020ae732f. 00000080:00000010:3.0:1788880172.753539:0:203405:0:(vvp_dev.c:130:vvp_thread_key_fini()) slab-freed '(vti)': 592 at 000000007ec0a7c6. 00000020:00000010:3.0:1788880172.753542:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000bd76ed56. 00000020:00000001:3.0:1788880172.753543:0:203405:0:(obd_class.h:704:obd_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.753545:0:203405:0:(obd_config.c:1689:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.753547:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 152 at 0000000047bb7169. 00000020:00000010:3.0:1788880172.753551:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 34 at 00000000e7da380a. 00000020:00000001:3.0:1788880172.753554:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.753556:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2ae0 type=0x10620000 idx=17 00000040:00001000:3.0:1788880172.753557:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 17 lrh_len: 224 (5408 remains) 00000040:00001000:3.0:1788880172.753563:0:203405:0:(llog.c:805:llog_process_thread()) index: 17, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.753565:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:3.0:1788880172.753567:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:3.0:1788880172.753569:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:3.0:1788880172.753569:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000010:3.0:1788880172.753572:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 000000005c064165. 00000020:00000080:3.0:1788880172.753574:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1788880172.753576:0:203405:0:(obd_config.c:1561:class_process_config()) marker 7 (0x2) lustre-MDT0000 add mdc 00000020:00000001:3.0:1788880172.753578:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.753580:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 000000005c064165. 00000020:00000001:3.0:1788880172.753584:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.753586:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2bc0 type=0x10620000 idx=18 00000040:00001000:3.0:1788880172.753588:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 18 lrh_len: 224 (5184 remains) 00000040:00001000:3.0:1788880172.753589:0:203405:0:(llog.c:805:llog_process_thread()) index: 18, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.753592:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:3.0:1788880172.753594:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:3.0:1788880172.753595:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:3.0:1788880172.753597:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x5 00000020:01000000:3.0:1788880172.753600:0:203405:0:(obd_config.c:1899:class_config_llog_handler()) SKIP #8 00000020:01000000:3.0:1788880172.753602:0:203405:0:(obd_config.c:1925:class_config_llog_handler()) skipping 0x6 00000020:00000001:3.0:1788880172.753602:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.753603:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2ca0 type=0x10620000 idx=19 00000040:00001000:3.0:1788880172.753605:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 19 lrh_len: 80 (4960 remains) 00000040:00001000:3.0:1788880172.753608:0:203405:0:(llog.c:805:llog_process_thread()) index: 19, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.753610:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:01000000:3.0:1788880172.753612:0:203405:0:(obd_config.c:1925:class_config_llog_handler()) skipping 0x6 00000020:00000001:3.0:1788880172.753612:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.753613:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2cf0 type=0x10620000 idx=20 00000040:00001000:3.0:1788880172.753614:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 20 lrh_len: 224 (4880 remains) 00000040:00001000:3.0:1788880172.753615:0:203405:0:(llog.c:805:llog_process_thread()) index: 20, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.753617:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:3.0:1788880172.753619:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:3.0:1788880172.753620:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:3.0:1788880172.753621:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x6 mark_flg=0x6 00000020:00000010:3.0:1788880172.753622:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 000000005c064165. 00000020:00000080:3.0:1788880172.753627:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1788880172.753628:0:203405:0:(obd_config.c:1561:class_process_config()) marker 8 (0x6) lustre sys.timeout 00000020:00000001:3.0:1788880172.753629:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.753631:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 000000005c064165. 00000020:00000001:3.0:1788880172.753634:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.753635:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2dd0 type=0x10620000 idx=21 00000040:00001000:3.0:1788880172.753636:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 21 lrh_len: 224 (4656 remains) 00000040:00001000:3.0:1788880172.753638:0:203405:0:(llog.c:805:llog_process_thread()) index: 21, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.753639:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:3.0:1788880172.753640:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:3.0:1788880172.753642:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:3.0:1788880172.753642:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000001:3.0:1788880172.753647:0:203405:0:(obd_mount.c:953:lustre_check_exclusion()) Process entered 00000020:00000001:3.0:1788880172.753648:0:203405:0:(obd_mount.c:958:lustre_check_exclusion()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.753650:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 000000005c064165. 00000020:00000080:3.0:1788880172.753651:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1788880172.753652:0:203405:0:(obd_config.c:1561:class_process_config()) marker 16 (0x1) lustre-MDT0001 add mdc 00000020:00000001:3.0:1788880172.753654:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.753655:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 000000005c064165. 00000020:00000001:3.0:1788880172.753657:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.753657:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2eb0 type=0x10620000 idx=22 00000040:00001000:3.0:1788880172.753658:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 22 lrh_len: 88 (4432 remains) 00000040:00001000:3.0:1788880172.753659:0:203405:0:(llog.c:805:llog_process_thread()) index: 22, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.753660:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:3.0:1788880172.753664:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 64 at 00000000e7da380a. 00000020:00000080:3.0:1788880172.753665:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf005 00000020:00000080:3.0:1788880172.753670:0:203405:0:(obd_config.c:1485:class_process_config()) adding mapping from uuid 192.168.204.111@tcp to nid 0x20000c0a8cc6f (192.168.204.111@tcp) 00000100:00000010:3.0:1788880172.753673:0:203405:0:(lustre_peer.c:90:class_add_uuid()) kmalloced '(data)': 704 at 000000002541ae0f. 00000100:00000040:3.0:1788880172.753677:0:203405:0:(lustre_peer.c:130:class_add_uuid()) found uuid 192.168.204.111@tcp 192.168.204.111@tcp cnt=1 00000400:00000200:3.0:1788880172.753682:0:203405:0:(peer.c:1363:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:3.0:1788880172.753684:0:203405:0:(api-ni.c:3274:LNetNIInit()) refs 1 00000400:00000200:3.0:1788880172.753687:0:203405:0:(peer.c:1377:LNetAddPeer()) Adding NID[0]: 192.168.204.111@tcp 00000400:00000200:3.0:1788880172.753692:0:203405:0:(peer.c:1822:lnet_peer_add()) peer 192.168.204.111@tcp NID flags 0x100001: -17 00000400:00000200:3.0:1788880172.753700:0:203405:0:(peer.c:2890:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:3.0:1788880172.753711:0:203405:0:(peer.c:2899:lnet_discover_peer_locked()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp: 0. pending discovery 00000400:00000200:3.0:1788880172.753715:0:203405:0:(peer.c:1431:LNetAddPeer()) A peer exists for NID 192.168.204.111@tcp 00000100:00000010:3.0:1788880172.753719:0:203405:0:(lustre_peer.c:132:class_add_uuid()) kfreed 'data': 704 at 000000002541ae0f. 00000020:00000001:3.0:1788880172.753723:0:203405:0:(obd_config.c:1500:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.753725:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 64 at 00000000e7da380a. 00000020:00000001:3.0:1788880172.753730:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.753732:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2f08 type=0x10620000 idx=23 00000040:00001000:3.0:1788880172.753735:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 23 lrh_len: 128 (4344 remains) 00000040:00001000:3.0:1788880172.753737:0:203405:0:(llog.c:805:llog_process_thread()) index: 23, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.753739:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:3.0:1788880172.753742:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 39 at 00000000e7da380a. 00000020:01000000:3.0:1788880172.753745:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf001, instance name: lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000010:3.0:1788880172.753748:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 136 at 0000000047bb7169. 00000020:00000080:3.0:1788880172.753749:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf001 00000020:00000001:3.0:1788880172.753751:0:203405:0:(obd_config.c:683:class_attach_name()) Process entered 00000020:00000001:3.0:1788880172.753752:0:203405:0:(genops.c:350:class_newdev()) Process entered 00000020:00000010:3.0:1788880172.753769:0:203405:0:(genops.c:51:obd_device_alloc()) slab-alloced 'obd': 6592 at 000000001b5d8280. 00000020:00000080:3.0:1788880172.753772:0:203405:0:(genops.c:413:class_newdev()) Allocate new device lustre-MDT0001-mdc-ffff8ca787527000 (000000001b5d8280) 00000020:00000001:3.0:1788880172.753775:0:203405:0:(genops.c:952:__class_new_export()) Process entered 00000020:00000010:3.0:1788880172.753777:0:203405:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 000000004a9f1dd1. 00000020:00000001:3.0:1788880172.753779:0:203405:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:3.0:1788880172.753781:0:203405:0:(lustre_handles.c:93:class_handle_hash()) added object 000000004a9f1dd1 with handle 0x805d548a36a8867b to hash 00000020:00000001:3.0:1788880172.753783:0:203405:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:3.0:1788880172.753786:0:203405:0:(obd_class.h:1016:obd_init_export()) Process entered 00000020:00000001:3.0:1788880172.753788:0:203405:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.753789:0:203405:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617249958672384 : -126823750879232 : ffff8ca78b613800) 00000020:00000040:3.0:1788880172.753791:0:203405:0:(genops.c:900:class_export_put()) PUTting export 000000004a9f1dd1 : new refcount 1 00000020:00000040:3.0:1788880172.753798:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-MDT0001-mdc-ffff8ca787527000 (000000001b5d8280) now 2 - obd_device_list 00000020:00000001:3.0:1788880172.753801:0:203405:0:(genops.c:555:class_register_device()) Process leaving (rc=0 : 0 : 0) 00000020:00000080:3.0:1788880172.753803:0:203405:0:(obd_config.c:720:class_attach_name()) OBD: dev 4 attached type mdc with refcount 2 00000020:00000001:3.0:1788880172.753805:0:203405:0:(obd_config.c:722:class_attach_name()) Process leaving (rc=18446617249929212800 : -126823780338816 : ffff8ca7899fb380) 00000020:00000001:3.0:1788880172.753807:0:203405:0:(obd_config.c:1479:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.753809:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 136 at 0000000047bb7169. 00000020:00000010:3.0:1788880172.753816:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 39 at 00000000e7da380a. 00000020:00000001:3.0:1788880172.753817:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.753818:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2f88 type=0x10620000 idx=24 00000040:00001000:3.0:1788880172.753820:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 24 lrh_len: 144 (4216 remains) 00000040:00001000:3.0:1788880172.753820:0:203405:0:(llog.c:805:llog_process_thread()) index: 24, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.753821:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:3.0:1788880172.753825:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 39 at 00000000e7da380a. 00000020:01000000:3.0:1788880172.753827:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf003, instance name: lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000010:3.0:1788880172.753829:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 136 at 0000000047bb7169. 00000020:00000080:3.0:1788880172.753830:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf003 00000020:00000001:3.0:1788880172.753832:0:203405:0:(obd_config.c:742:class_setup()) Process entered 00000001:00000001:3.0:1788880172.753848:0:203405:0:(hash.c:1038:cfs_hash_create()) Process entered 00000001:00000010:3.0:1788880172.753852:0:203405:0:(hash.c:1065:cfs_hash_create()) alloc '(hs)': 232 at 000000005c064165. 00000001:00000010:3.0:1788880172.753857:0:203405:0:(hash.c:933:cfs_hash_buckets_realloc()) alloc '(new_bkts)': 32 at 00000000626f39ad. 00000001:00000010:3.0:1788880172.753860:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 00000000bd76ed56. 00000001:00000010:3.0:1788880172.753862:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000024c0dace. 00000001:00000010:3.0:1788880172.753865:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 00000000e49153de. 00000001:00000010:3.0:1788880172.753868:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 00000000b05735d2. 00000020:00000001:3.0:1788880172.753870:0:203405:0:(obd_class.h:565:obd_setup()) Process entered 00000020:00000010:3.0:1788880172.753874:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000005fb6640b. 00000020:00000010:3.0:1788880172.753879:0:203405:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 0000000014f65f10. 00000020:00000010:3.0:1788880172.753893:0:203405:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 000000000bcc11c1. 00000008:00000010:3.0:1788880172.753906:0:203405:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000ae551350. 00020000:00000010:3.0:1788880172.753921:0:203405:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 000000009b9eb526. 00008000:00000010:3.0:1788880172.753941:0:203405:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 00000000db05d82b. 00000080:00000010:3.0:1788880172.753955:0:203405:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 00000000e9f8c309. 00000020:00000010:3.0:1788880172.753959:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000003422132. 00000020:00000010:3.0:1788880172.753962:0:203405:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000fe6d4325. 00000020:00000010:3.0:1788880172.753966:0:203405:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000f6896c3f. 00000008:00000010:3.0:1788880172.753969:0:203405:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000aeeba8e2. 00020000:00000010:3.0:1788880172.753972:0:203405:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000b8c7c243. 00008000:00000010:3.0:1788880172.753975:0:203405:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000d0b9f970. 00000080:00000010:3.0:1788880172.753979:0:203405:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 0000000020ae732f. 00000080:00000010:3.0:1788880172.753988:0:203405:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 000000007ec0a7c6. 00000002:00000010:3.0:1788880172.753993:0:203405:0:(mdc_dev.c:1727:mdc_device_alloc()) kmalloced '(osc)': 104 at 0000000055903908. 00000002:00000001:3.0:1788880172.753997:0:203405:0:(mdc_request.c:3051:mdc_setup()) Process entered 00000008:00000001:3.0:1788880172.753998:0:203405:0:(osc_request.c:4061:osc_setup_common()) Process entered 00000100:00000001:3.0:1788880172.754001:0:203405:0:(ptlrpcd.c:908:ptlrpcd_addref()) Process entered 00000100:00000001:3.0:1788880172.754003:0:203405:0:(ptlrpcd.c:917:ptlrpcd_addref()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880172.754006:0:203405:0:(ldlm_lib.c:297:client_obd_setup()) Process entered 00010000:00000010:3.0:1788880172.754015:0:203405:0:(ldlm_lib.c:420:client_obd_setup()) kmalloced '(cli->cl_read_io_latency_by_size)': 3920 at 00000000a374cc96. 00010000:00000010:3.0:1788880172.754039:0:203405:0:(ldlm_lib.c:433:client_obd_setup()) kmalloced '(cli->cl_write_io_latency_by_size)': 3920 at 0000000083414525. 00010000:00000010:3.0:1788880172.754056:0:203405:0:(ldlm_lib.c:522:client_obd_setup()) kmalloced '(cli->cl_mod_tag_bitmap)': 64 at 000000000533f34c. 00010000:00000001:3.0:1788880172.754058:0:203405:0:(ldlm_lockd.c:3036:ldlm_get_ref()) Process entered 00010000:00000001:3.0:1788880172.754061:0:203405:0:(ldlm_lockd.c:3045:ldlm_get_ref()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.754068:0:203405:0:(genops.c:1285:class_new_import()) kmalloced '(imp)': 1720 at 00000000b98b8a4e. 00000020:00000040:3.0:1788880172.754071:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-MDT0001-mdc-ffff8ca787527000 (000000001b5d8280) now 3 - import 00000020:00000001:3.0:1788880172.754087:0:203405:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880172.754089:0:203405:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=1 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880172.754092:0:203405:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:3.0:1788880172.754094:0:203405:0:(ldlm_lib.c:49:import_set_conn()) Process entered 00000100:00000200:3.0:1788880172.754107:0:203405:0:(events.c:559:ptlrpc_uuid_to_peer()) 192.168.204.111@tcp->12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880172.754110:0:203405:0:(connection.c:104:ptlrpc_connection_get()) Process entered 00000400:00000200:3.0:1788880172.754116:0:203405:0:(peer.c:1550:LNetPrimaryNID()) NID 192.168.204.111@tcp primary NID 192.168.204.111@tcp rc 0 00000100:00000001:3.0:1788880172.754119:0:203405:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788880172.754121:0:203405:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 5 to 192.168.204.111@tcp 00000100:00000001:3.0:1788880172.754124:0:203405:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000001:3.0:1788880172.754127:0:203405:0:(connection.c:110:ptlrpc_connection_get()) Process leaving via out (rc=18446617250405033856 : -126823304517760 : 0xffff8ca7a5fc2780) 00000100:00000040:3.0:1788880172.754130:0:203405:0:(connection.c:149:ptlrpc_connection_get()) conn=000000008a5be7d5 refcount 5 to 192.168.204.111@tcp 00000100:00000040:3.0:1788880172.754133:0:203405:0:(client.c:137:ptlrpc_uuid_to_connection()) 192.168.204.111@tcp -> 000000008a5be7d5 00010000:00000010:3.0:1788880172.754137:0:203405:0:(ldlm_lib.c:70:import_set_conn()) kmalloced '(imp_conn)': 88 at 0000000027700007. 00010000:00080000:3.0:1788880172.754140:0:203405:0:(ldlm_lib.c:102:import_set_conn()) imp 00000000b98b8a4e@lustre-MDT0001-mdc-ffff8ca787527000: add connection 192.168.204.111@tcp at head 00010000:00000001:3.0:1788880172.754142:0:203405:0:(ldlm_lib.c:109:import_set_conn()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880172.754144:0:203405:0:(ldlm_resource.c:976:ldlm_namespace_new()) Process entered 00010000:00000001:3.0:1788880172.754145:0:203405:0:(ldlm_lockd.c:3036:ldlm_get_ref()) Process entered 00010000:00000001:3.0:1788880172.754147:0:203405:0:(ldlm_lockd.c:3045:ldlm_get_ref()) Process leaving (rc=0 : 0 : 0) 00010000:00000010:3.0:1788880172.754156:0:203405:0:(ldlm_resource.c:993:ldlm_namespace_new()) kmalloced '(ns)': 848 at 0000000090194164. 00000001:00000001:3.0:1788880172.754158:0:203405:0:(hash.c:1038:cfs_hash_create()) Process entered 00000001:00000010:3.0:1788880172.754162:0:203405:0:(hash.c:1065:cfs_hash_create()) alloc '(hs)': 280 at 00000000a2ead94f. 00000001:00000010:3.0:1788880172.754167:0:203405:0:(hash.c:933:cfs_hash_buckets_realloc()) alloc '(new_bkts)': 256 at 00000000d4182dc9. 00000001:00000010:3.0:1788880172.754205:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000009bd452a3. 00000001:00000010:3.0:1788880172.754283:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000032086649. 00000001:00000010:3.0:1788880172.754344:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000fafacf29. 00000001:00000010:3.0:1788880172.754399:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000004f91f3c. 00000001:00000010:3.0:1788880172.754451:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000001c328829. 00000001:00000010:3.0:1788880172.754504:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000053e913c6. 00000001:00000010:3.0:1788880172.754554:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000f39a0990. 00000001:00000010:3.0:1788880172.754608:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000e426935d. 00000001:00000010:3.0:1788880172.754658:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000004f3ce913. 00000001:00000010:3.0:1788880172.754704:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000d3e2a73d. 00000001:00000010:3.0:1788880172.754742:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000002823106a. 00000001:00000010:3.0:1788880172.754784:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000093086e5. 00000001:00000010:3.0:1788880172.754833:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000246411cf. 00000001:00000010:3.0:1788880172.754898:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000007bf9e947. 00000001:00000010:3.0:1788880172.754955:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000005257acd1. 00000001:00000010:3.0:1788880172.755005:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000002069127f. 00000001:00000010:3.0:1788880172.755092:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000004dd30a41. 00000001:00000010:3.0:1788880172.755165:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000020051610. 00000001:00000010:3.0:1788880172.755284:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000002ed4ba3c. 00000001:00000010:3.0:1788880172.755361:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000f240bc74. 00000001:00000010:3.0:1788880172.755435:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000006ff851ae. 00000001:00000010:3.0:1788880172.755506:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000003cac50f8. 00000001:00000010:3.0:1788880172.755580:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000496cb640. 00000001:00000010:3.0:1788880172.755651:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000009b12b20a. 00000001:00000010:3.0:1788880172.755723:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000008b45fad2. 00000001:00000010:3.0:1788880172.755790:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000f86b6a10. 00000001:00000010:3.0:1788880172.755838:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000098ef1eb8. 00000001:00000010:3.0:1788880172.755895:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000009ca14d12. 00000001:00000010:3.0:1788880172.755947:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000ecacff71. 00000001:00000010:3.0:1788880172.755998:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 0000000023611cad. 00000001:00000010:3.0:1788880172.756060:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 000000001f77b874. 00000001:00000010:3.0:1788880172.756117:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 32944 at 00000000a092e435. 00000020:00000010:3.0:1788880172.756158:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 0000000081d16086. 00000020:00000010:3.0:1788880172.756161:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 96 at 000000007dbbefd6. 00010000:00000001:3.0:1788880172.756324:0:203405:0:(ldlm_pool.c:891:ldlm_pool_init()) Process entered 00010000:00000001:3.0:1788880172.756366:0:203405:0:(ldlm_pool.c:810:ldlm_pool_debugfs_init()) Process entered 00000020:00000010:3.0:1788880172.756396:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 00000000d6551634. 00000020:00000010:3.0:1788880172.756400:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 352 at 00000000253a0a56. 00010000:00000001:3.0:1788880172.756425:0:203405:0:(ldlm_pool.c:865:ldlm_pool_debugfs_init()) Process leaving 00010000:00010000:3.0:1788880172.756427:0:203405:0:(ldlm_pool.c:925:ldlm_pool_init()) Lock pool ldlm-pool-lustre-MDT0001-mdc-ffff8ca787527000-2 is initialized 00010000:00000001:3.0:1788880172.756430:0:203405:0:(ldlm_pool.c:927:ldlm_pool_init()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880172.756433:0:203405:0:(ldlm_resource.c:1084:ldlm_namespace_new()) Process leaving (rc=18446617249896434688 : -126823813116928 : ffff8ca787ab8c00) 00010000:00000001:3.0:1788880172.756435:0:203405:0:(ldlm_lib.c:603:client_obd_setup()) Process leaving (rc=0 : 0 : 0) 00000008:00000020:3.0:1788880172.756443:0:203405:0:(osc_request.c:745:osc_update_next_shrink()) next time 12336 to shrink grant 00000008:00000001:3.0:1788880172.756445:0:203405:0:(osc_request.c:4083:osc_setup_common()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.756643:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 00000000e3254fcf. 00000020:00000010:3.0:1788880172.756647:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 512 at 0000000073c28f31. 00000020:00000010:3.0:1788880172.756673:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 00000000042bbdf6. 00000020:00000010:3.0:1788880172.756676:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 3680 at 0000000073b4460c. 00000002:00000001:3.0:1788880172.756697:0:203405:0:(mdc_request.c:3022:mdc_llog_init()) Process entered 00000040:00000001:3.0:1788880172.756699:0:203405:0:(llog_obd.c:135:llog_setup()) Process entered 00000040:00000010:3.0:1788880172.756703:0:203405:0:(llog_obd.c:26:llog_new_ctxt()) kmalloced '(ctxt)': 168 at 00000000709015c2. 00000020:00000040:3.0:1788880172.756705:0:203405:0:(genops.c:889:class_export_get()) GET export 000000004a9f1dd1 refcount=2 00000040:01000000:3.0:1788880172.756708:0:203405:0:(llog_obd.c:197:llog_setup()) obd lustre-MDT0001-mdc-ffff8ca787527000 ctxt 13 is initialized 00000040:00000001:3.0:1788880172.756709:0:203405:0:(llog_obd.c:201:llog_setup()) Process leaving (rc=0 : 0 : 0) 00000002:00000040:3.0:1788880172.756712:0:203405:0:(lustre_log.h:387:llog_ctxt_get()) GETting ctxt 00000000709015c2 : new refcount 2 00000040:00000001:3.0:1788880172.756714:0:203405:0:(llog_net.c:31:llog_initiator_connect()) Process entered 00000020:00000040:3.0:1788880172.756715:0:203405:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=2 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000040:00000001:3.0:1788880172.756717:0:203405:0:(llog_net.c:44:llog_initiator_connect()) Process leaving (rc=0 : 0 : 0) 00000002:00000040:3.0:1788880172.756718:0:203405:0:(lustre_log.h:402:llog_ctxt_put()) PUTting ctxt 00000000709015c2 : new refcount 1 00000002:00000001:3.0:1788880172.756725:0:203405:0:(mdc_request.c:3032:mdc_llog_init()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880172.756726:0:203405:0:(mdc_changelog.c:929:mdc_changelog_cdev_init()) Process entered 00000002:00000010:3.0:1788880172.756728:0:203405:0:(mdc_changelog.c:931:mdc_changelog_cdev_init()) kmalloced '(entry)': 1304 at 00000000b4b4c355. 00000002:00000001:3.0:1788880172.756899:0:203405:0:(mdc_changelog.c:974:mdc_changelog_cdev_init()) Process leaving via out_unlock (rc=0 : 0 : 0x0) 00000002:00000001:3.0:1788880172.756903:0:203405:0:(mdc_changelog.c:989:mdc_changelog_cdev_init()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880172.756906:0:203405:0:(mdc_request.c:3081:mdc_setup()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880172.756909:0:203405:0:(mdc_dev.c:1752:mdc_device_alloc()) Process leaving (rc=18446617250661747840 : -126823047803776 : ffff8ca7b5494c80) 00000002:00000001:3.0:1788880172.756913:0:203405:0:(mdc_dev.c:1758:mdc_device_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.756917:0:203405:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000fe6d4325. 00000020:00000010:3.0:1788880172.756926:0:203405:0:(cl_object.c:1137:cl_key_fini()) kfreed 'info': 72 at 00000000f6896c3f. 00000008:00000010:3.0:1788880172.756936:0:203405:0:(osc_dev.c:100:osc_key_fini()) slab-freed '(info)': 2864 at 00000000aeeba8e2. 00020000:00000010:3.0:1788880172.756945:0:203405:0:(lov_dev.c:95:lov_key_fini()) slab-freed '(info)': 192 at 00000000b8c7c243. 00008000:00000010:3.0:1788880172.756952:0:203405:0:(echo_client.c:424:echo_thread_key_fini()) slab-freed '(info)': 904 at 00000000d0b9f970. 00000080:00000010:3.0:1788880172.756959:0:203405:0:(vvp_dev.c:80:ll_thread_key_fini()) slab-freed '(lti)': 168 at 0000000020ae732f. 00000080:00000010:3.0:1788880172.756965:0:203405:0:(vvp_dev.c:130:vvp_thread_key_fini()) slab-freed '(vti)': 592 at 000000007ec0a7c6. 00000020:00000010:3.0:1788880172.756972:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000003422132. 00000020:00000010:3.0:1788880172.756978:0:203405:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 0000000014f65f10. 00000020:00000010:3.0:1788880172.756983:0:203405:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 000000000bcc11c1. 00000008:00000010:3.0:1788880172.756989:0:203405:0:(osc_dev.c:125:osc_session_fini()) slab-freed '(info)': 352 at 00000000ae551350. 00020000:00000010:3.0:1788880172.756994:0:203405:0:(lov_dev.c:136:lov_session_key_fini()) slab-freed '(info)': 584 at 000000009b9eb526. 00008000:00000010:3.0:1788880172.756998:0:203405:0:(echo_client.c:449:echo_session_key_fini()) slab-freed '(session)': 8 at 00000000db05d82b. 00000080:00000010:3.0:1788880172.757002:0:203405:0:(vvp_dev.c:105:vvp_session_key_fini()) slab-freed '(session)': 240 at 00000000e9f8c309. 00000020:00000010:3.0:1788880172.757007:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 000000005fb6640b. 00000020:00000001:3.0:1788880172.757013:0:203405:0:(obd_class.h:601:obd_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880172.757017:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-MDT0001-mdc-ffff8ca787527000 (000000001b5d8280) now 4 - setup 00000020:00000080:3.0:1788880172.757020:0:203405:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-MDT0001-mdc-ffff8ca787527000 (uuid ea27006f-52a8-4007-b69a-dcac8279445a) 00000020:00000001:3.0:1788880172.757023:0:203405:0:(obd_config.c:828:class_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.757025:0:203405:0:(obd_config.c:1618:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.757041:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 136 at 0000000047bb7169. 00000020:00000010:3.0:1788880172.757049:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 39 at 00000000e7da380a. 00000020:00000001:3.0:1788880172.757055:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.757060:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3018 type=0x10620000 idx=25 00000040:00001000:3.0:1788880172.757072:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 25 lrh_len: 160 (4072 remains) 00000040:00001000:3.0:1788880172.757075:0:203405:0:(llog.c:805:llog_process_thread()) index: 25, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.757079:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:3.0:1788880172.757084:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 34 at 000000009e7d9176. 00000020:01000000:3.0:1788880172.757089:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf014, instance name: lustre-clilmv-ffff8ca787527000 00000020:00000010:3.0:1788880172.757092:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 152 at 0000000047bb7169. 00000020:00000080:3.0:1788880172.757096:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf014 00000020:00000001:3.0:1788880172.757102:0:203405:0:(obd_class.h:689:obd_process_config()) Process entered 00000020:00000010:3.0:1788880172.757106:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000046d2f41b. 00000020:00000010:3.0:1788880172.757112:0:203405:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 0000000016c67947. 00000040:00000010:3.0:1788880172.757117:0:203405:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000d1766dda. 00000020:00000010:3.0:1788880172.757122:0:203405:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 0000000003cce9d8. 00000020:00000010:3.0:1788880172.757129:0:203405:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 000000005fb6640b. 00040000:00000010:3.0:1788880172.757136:0:203405:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 00000000571e0604. 00040000:00000010:3.0:1788880172.757143:0:203405:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 0000000003422132. 00800000:00000001:3.0:1788880172.757150:0:203405:0:(lmv_obd.c:1377:lmv_process_config()) Process entered 00800000:00000001:3.0:1788880172.757156:0:203405:0:(lmv_obd.c:437:lmv_add_target()) Process entered 00800000:01000000:3.0:1788880172.757159:0:203405:0:(lmv_obd.c:439:lmv_add_target()) Target uuid: lustre-MDT0001_UUID. index 1 00800000:00000010:3.0:1788880172.757169:0:203405:0:(lmv_obd.c:448:lmv_add_target()) kmalloced '(tgt)': 368 at 00000000fe6d4325. 00020000:00000001:3.0:1788880172.757173:0:203405:0:(lu_tgt_descs.c:352:ltd_add_tgt()) Process entered 00020000:00000001:3.0:1788880172.757177:0:203405:0:(lu_tgt_descs.c:383:ltd_add_tgt()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:3.0:1788880172.757179:0:203405:0:(lmv_obd.c:466:lmv_add_target()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:3.0:1788880172.757181:0:203405:0:(lmv_obd.c:1394:lmv_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00800000:00000001:3.0:1788880172.757185:0:203405:0:(lmv_obd.c:1400:lmv_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.757188:0:203405:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 0000000016c67947. 00000040:00000010:3.0:1788880172.757195:0:203405:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000d1766dda. 00000020:00000010:3.0:1788880172.757202:0:203405:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 0000000003cce9d8. 00000020:00000010:3.0:1788880172.757210:0:203405:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 000000005fb6640b. 00040000:00000010:3.0:1788880172.757218:0:203405:0:(lquota_lib.c:35:lquota_key_fini()) kfreed 'info': 248 at 00000000571e0604. 00040000:00000010:3.0:1788880172.757223:0:203405:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 0000000003422132. 00000020:00000010:3.0:1788880172.757229:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000046d2f41b. 00000020:00000001:3.0:1788880172.757283:0:203405:0:(obd_class.h:704:obd_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.757286:0:203405:0:(obd_class.h:689:obd_process_config()) Process entered 00000020:00000010:3.0:1788880172.757291:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000003422132. 00000020:00000010:3.0:1788880172.757298:0:203405:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000005fb6640b. 00000020:00000010:3.0:1788880172.757311:0:203405:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000e83229b9. 00000008:00000010:3.0:1788880172.757318:0:203405:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000aeeba8e2. 00020000:00000010:3.0:1788880172.757325:0:203405:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000b8c7c243. 00008000:00000010:3.0:1788880172.757330:0:203405:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000d0b9f970. 00000080:00000010:3.0:1788880172.757338:0:203405:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 0000000020ae732f. 00000080:00000010:3.0:1788880172.757344:0:203405:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 000000007ec0a7c6. 00020000:00000001:3.0:1788880172.757350:0:203405:0:(lov_obd.c:814:lov_process_config_base()) Process entered 00020000:00000001:3.0:1788880172.757352:0:203405:0:(lov_obd.c:868:lov_process_config_base()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:3.0:1788880172.757358:0:203405:0:(lov_dev.c:450:lov_add_mdc_target()) Process entered 00020000:00000001:3.0:1788880172.757360:0:203405:0:(lov_dev.c:507:lov_add_mdc_target()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:3.0:1788880172.757364:0:203405:0:(lov_dev.c:573:lov_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.757367:0:203405:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 000000005fb6640b. 00000020:00000010:3.0:1788880172.757376:0:203405:0:(cl_object.c:1137:cl_key_fini()) kfreed 'info': 72 at 00000000e83229b9. 00000008:00000010:3.0:1788880172.757383:0:203405:0:(osc_dev.c:100:osc_key_fini()) slab-freed '(info)': 2864 at 00000000aeeba8e2. 00020000:00000010:3.0:1788880172.757390:0:203405:0:(lov_dev.c:95:lov_key_fini()) slab-freed '(info)': 192 at 00000000b8c7c243. 00008000:00000010:3.0:1788880172.757395:0:203405:0:(echo_client.c:424:echo_thread_key_fini()) slab-freed '(info)': 904 at 00000000d0b9f970. 00000080:00000010:3.0:1788880172.757398:0:203405:0:(vvp_dev.c:80:ll_thread_key_fini()) slab-freed '(lti)': 168 at 0000000020ae732f. 00000080:00000010:3.0:1788880172.757407:0:203405:0:(vvp_dev.c:130:vvp_thread_key_fini()) slab-freed '(vti)': 592 at 000000007ec0a7c6. 00000020:00000010:3.0:1788880172.757416:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000003422132. 00000020:00000001:3.0:1788880172.757421:0:203405:0:(obd_class.h:704:obd_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.757424:0:203405:0:(obd_config.c:1689:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.757427:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 152 at 0000000047bb7169. 00000020:00000010:3.0:1788880172.757431:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 34 at 000000009e7d9176. 00000020:00000001:3.0:1788880172.757440:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.757443:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf30b8 type=0x10620000 idx=26 00000040:00001000:3.0:1788880172.757446:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 26 lrh_len: 224 (3912 remains) 00000040:00001000:3.0:1788880172.757449:0:203405:0:(llog.c:805:llog_process_thread()) index: 26, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.757453:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:3.0:1788880172.757456:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:3.0:1788880172.757459:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:3.0:1788880172.757462:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000010:3.0:1788880172.757468:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 00000000571e0604. 00000020:00000080:3.0:1788880172.757471:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1788880172.757473:0:203405:0:(obd_config.c:1561:class_process_config()) marker 16 (0x2) lustre-MDT0001 add mdc 00000020:00000001:3.0:1788880172.757483:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.757486:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 00000000571e0604. 00000020:00000001:3.0:1788880172.757489:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.757492:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3198 type=0x10620000 idx=27 00000040:00001000:3.0:1788880172.757494:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 27 lrh_len: 224 (3688 remains) 00000040:00001000:3.0:1788880172.757496:0:203405:0:(llog.c:805:llog_process_thread()) index: 27, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.757498:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:3.0:1788880172.757501:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:3.0:1788880172.757503:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:3.0:1788880172.757506:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x5 00000020:01000000:3.0:1788880172.757509:0:203405:0:(obd_config.c:1899:class_config_llog_handler()) SKIP #18 00000020:01000000:3.0:1788880172.757512:0:203405:0:(obd_config.c:1925:class_config_llog_handler()) skipping 0x6 00000020:00000001:3.0:1788880172.757514:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.757517:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3278 type=0x10620000 idx=28 00000040:00001000:3.0:1788880172.757519:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 28 lrh_len: 80 (3464 remains) 00000040:00001000:3.0:1788880172.757523:0:203405:0:(llog.c:805:llog_process_thread()) index: 28, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.757526:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:01000000:3.0:1788880172.757528:0:203405:0:(obd_config.c:1925:class_config_llog_handler()) skipping 0x6 00000020:00000001:3.0:1788880172.757531:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.757534:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf32c8 type=0x10620000 idx=29 00000040:00001000:3.0:1788880172.757538:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 29 lrh_len: 224 (3384 remains) 00000040:00001000:3.0:1788880172.757540:0:203405:0:(llog.c:805:llog_process_thread()) index: 29, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.757543:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:3.0:1788880172.757545:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:3.0:1788880172.757547:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:3.0:1788880172.757550:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x6 mark_flg=0x6 00000020:00000010:3.0:1788880172.757556:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 00000000571e0604. 00000020:00000080:3.0:1788880172.757560:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1788880172.757563:0:203405:0:(obd_config.c:1561:class_process_config()) marker 18 (0x6) lustre sys.timeout 00000020:00000001:3.0:1788880172.757566:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.757569:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 00000000571e0604. 00000020:00000001:3.0:1788880172.757575:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.757579:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf33a8 type=0x10620000 idx=30 00000040:00001000:3.0:1788880172.757583:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 30 lrh_len: 224 (3160 remains) 00000040:00001000:3.0:1788880172.757593:0:203405:0:(llog.c:805:llog_process_thread()) index: 30, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.757596:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:3.0:1788880172.757599:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:3.0:1788880172.757601:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:3.0:1788880172.757603:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000001:3.0:1788880172.757609:0:203405:0:(obd_mount.c:953:lustre_check_exclusion()) Process entered 00000020:01000004:3.0:1788880172.757611:0:203405:0:(obd_mount.c:961:lustre_check_exclusion()) Check exclusion lustre-OST0000 (0) in 0 of 192.168.204.111@tcp:/lustre 00000020:00000001:3.0:1788880172.757616:0:203405:0:(obd_mount.c:969:lustre_check_exclusion()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.757621:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 00000000571e0604. 00000020:00000080:3.0:1788880172.757623:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1788880172.757625:0:203405:0:(obd_config.c:1561:class_process_config()) marker 24 (0x1) lustre-OST0000 add osc 00000020:00000001:3.0:1788880172.757627:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.757631:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 00000000571e0604. 00000020:00000001:3.0:1788880172.757639:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.757641:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3488 type=0x10620000 idx=31 00000040:00001000:3.0:1788880172.757644:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 31 lrh_len: 88 (2936 remains) 00000040:00001000:3.0:1788880172.757647:0:203405:0:(llog.c:805:llog_process_thread()) index: 31, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.757649:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:3.0:1788880172.757655:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 64 at 000000009e7d9176. 00000020:00000080:3.0:1788880172.757658:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf005 00000020:00000080:3.0:1788880172.757667:0:203405:0:(obd_config.c:1485:class_process_config()) adding mapping from uuid 192.168.204.111@tcp to nid 0x20000c0a8cc6f (192.168.204.111@tcp) 00000100:00000010:3.0:1788880172.757675:0:203405:0:(lustre_peer.c:90:class_add_uuid()) kmalloced '(data)': 704 at 00000000d04c3961. 00000100:00000040:3.0:1788880172.757683:0:203405:0:(lustre_peer.c:130:class_add_uuid()) found uuid 192.168.204.111@tcp 192.168.204.111@tcp cnt=1 00000400:00000200:3.0:1788880172.757688:0:203405:0:(peer.c:1363:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:3.0:1788880172.757692:0:203405:0:(api-ni.c:3274:LNetNIInit()) refs 1 00000400:00000200:3.0:1788880172.757697:0:203405:0:(peer.c:1377:LNetAddPeer()) Adding NID[0]: 192.168.204.111@tcp 00000400:00000200:3.0:1788880172.757707:0:203405:0:(peer.c:1822:lnet_peer_add()) peer 192.168.204.111@tcp NID flags 0x100001: -17 00000400:00000200:3.0:1788880172.757727:0:203405:0:(peer.c:2890:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:3.0:1788880172.757733:0:203405:0:(peer.c:2899:lnet_discover_peer_locked()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp: 0. pending discovery 00000400:00000200:3.0:1788880172.757737:0:203405:0:(peer.c:1431:LNetAddPeer()) A peer exists for NID 192.168.204.111@tcp 00000100:00000010:3.0:1788880172.757740:0:203405:0:(lustre_peer.c:132:class_add_uuid()) kfreed 'data': 704 at 00000000d04c3961. 00000020:00000001:3.0:1788880172.757746:0:203405:0:(obd_config.c:1500:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.757750:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 64 at 000000009e7d9176. 00000020:00000001:3.0:1788880172.757757:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.757767:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf34e0 type=0x10620000 idx=32 00000040:00001000:3.0:1788880172.757771:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 32 lrh_len: 128 (2848 remains) 00000040:00001000:3.0:1788880172.757773:0:203405:0:(llog.c:805:llog_process_thread()) index: 32, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.757775:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:3.0:1788880172.757781:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 39 at 000000009e7d9176. 00000020:01000000:3.0:1788880172.757784:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf001, instance name: lustre-OST0000-osc-ffff8ca787527000 00000020:00000010:3.0:1788880172.757789:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 136 at 0000000047bb7169. 00000020:00000080:3.0:1788880172.757793:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf001 00000020:00000001:3.0:1788880172.757795:0:203405:0:(obd_config.c:683:class_attach_name()) Process entered 00000020:00000001:3.0:1788880172.757797:0:203405:0:(genops.c:350:class_newdev()) Process entered 00000020:00000010:3.0:1788880172.757807:0:203405:0:(genops.c:51:obd_device_alloc()) slab-alloced 'obd': 6592 at 00000000bd6a9017. 00000020:00000080:3.0:1788880172.757812:0:203405:0:(genops.c:413:class_newdev()) Allocate new device lustre-OST0000-osc-ffff8ca787527000 (00000000bd6a9017) 00000020:00000001:3.0:1788880172.757816:0:203405:0:(genops.c:952:__class_new_export()) Process entered 00000020:00000010:3.0:1788880172.757821:0:203405:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 0000000003cce9d8. 00000020:00000001:3.0:1788880172.757825:0:203405:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:3.0:1788880172.757830:0:203405:0:(lustre_handles.c:93:class_handle_hash()) added object 0000000003cce9d8 with handle 0x805d548a36a88682 to hash 00000020:00000001:3.0:1788880172.757833:0:203405:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:3.0:1788880172.757840:0:203405:0:(obd_class.h:1016:obd_init_export()) Process entered 00000020:00000001:3.0:1788880172.757842:0:203405:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.757846:0:203405:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617249958660096 : -126823750891520 : ffff8ca78b610800) 00000020:00000040:3.0:1788880172.757850:0:203405:0:(genops.c:900:class_export_put()) PUTting export 0000000003cce9d8 : new refcount 1 00000020:00000040:3.0:1788880172.757859:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-OST0000-osc-ffff8ca787527000 (00000000bd6a9017) now 2 - obd_device_list 00000020:00000001:3.0:1788880172.757863:0:203405:0:(genops.c:555:class_register_device()) Process leaving (rc=0 : 0 : 0) 00000020:00000080:3.0:1788880172.757865:0:203405:0:(obd_config.c:720:class_attach_name()) OBD: dev 5 attached type osc with refcount 2 00000020:00000001:3.0:1788880172.757868:0:203405:0:(obd_config.c:722:class_attach_name()) Process leaving (rc=18446617249929206208 : -126823780345408 : ffff8ca7899f99c0) 00000020:00000001:3.0:1788880172.757873:0:203405:0:(obd_config.c:1479:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.757876:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 136 at 0000000047bb7169. 00000020:00000010:3.0:1788880172.757881:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 39 at 000000009e7d9176. 00000020:00000001:3.0:1788880172.757888:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.757892:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3560 type=0x10620000 idx=33 00000040:00001000:3.0:1788880172.757895:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 33 lrh_len: 144 (2720 remains) 00000040:00001000:3.0:1788880172.757897:0:203405:0:(llog.c:805:llog_process_thread()) index: 33, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.757899:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:3.0:1788880172.757912:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 39 at 000000009e7d9176. 00000020:01000000:3.0:1788880172.757917:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf003, instance name: lustre-OST0000-osc-ffff8ca787527000 00000020:00000010:3.0:1788880172.757922:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 136 at 0000000047bb7169. 00000020:00000080:3.0:1788880172.757927:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf003 00000020:00000001:3.0:1788880172.757932:0:203405:0:(obd_config.c:742:class_setup()) Process entered 00000001:00000001:3.0:1788880172.757971:0:203405:0:(hash.c:1038:cfs_hash_create()) Process entered 00000001:00000010:3.0:1788880172.757978:0:203405:0:(hash.c:1065:cfs_hash_create()) alloc '(hs)': 232 at 00000000571e0604. 00000001:00000010:3.0:1788880172.757990:0:203405:0:(hash.c:933:cfs_hash_buckets_realloc()) alloc '(new_bkts)': 32 at 0000000049939d37. 00000001:00000010:3.0:1788880172.757995:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000003422132. 00000001:00000010:3.0:1788880172.757998:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 000000005fb6640b. 00000001:00000010:3.0:1788880172.758003:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 0000000046d2f41b. 00000001:00000010:3.0:1788880172.758006:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 344 at 00000000d1766dda. 00000020:00000001:3.0:1788880172.758010:0:203405:0:(obd_class.h:565:obd_setup()) Process entered 00000020:00000010:3.0:1788880172.758013:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000016c67947. 00000020:00000010:3.0:1788880172.758018:0:203405:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 0000000014f65f10. 00000020:00000010:3.0:1788880172.758023:0:203405:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 000000000bcc11c1. 00000008:00000010:3.0:1788880172.758040:0:203405:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000ae551350. 00020000:00000010:3.0:1788880172.758044:0:203405:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 000000009b9eb526. 00008000:00000010:3.0:1788880172.758049:0:203405:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 00000000db05d82b. 00000080:00000010:3.0:1788880172.758054:0:203405:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 00000000e9f8c309. 00000020:00000010:3.0:1788880172.758062:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000937f3d66. 00000020:00000010:3.0:1788880172.758065:0:203405:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000d0916d04. 00000020:00000010:3.0:1788880172.758069:0:203405:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000e83229b9. 00000008:00000010:3.0:1788880172.758072:0:203405:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000aeeba8e2. 00020000:00000010:3.0:1788880172.758075:0:203405:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000b8c7c243. 00008000:00000010:3.0:1788880172.758078:0:203405:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000d0b9f970. 00000080:00000010:3.0:1788880172.758081:0:203405:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 0000000020ae732f. 00000080:00000010:3.0:1788880172.758084:0:203405:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 000000007ec0a7c6. 00000008:00000010:3.0:1788880172.758087:0:203405:0:(osc_dev.c:193:osc_device_alloc()) kmalloced '(osc)': 104 at 00000000fea7d0da. 00000008:00000001:3.0:1788880172.758092:0:203405:0:(osc_request.c:4103:osc_setup()) Process entered 00000008:00000001:3.0:1788880172.758095:0:203405:0:(osc_request.c:4061:osc_setup_common()) Process entered 00000100:00000001:3.0:1788880172.758098:0:203405:0:(ptlrpcd.c:908:ptlrpcd_addref()) Process entered 00000100:00000001:3.0:1788880172.758100:0:203405:0:(ptlrpcd.c:917:ptlrpcd_addref()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880172.758103:0:203405:0:(ldlm_lib.c:297:client_obd_setup()) Process entered 00010000:00000010:3.0:1788880172.758118:0:203405:0:(ldlm_lib.c:420:client_obd_setup()) kmalloced '(cli->cl_read_io_latency_by_size)': 3920 at 00000000cb2ca7c9. 00010000:00000010:3.0:1788880172.758124:0:203405:0:(ldlm_lib.c:433:client_obd_setup()) kmalloced '(cli->cl_write_io_latency_by_size)': 3920 at 000000007b9bc715. 00010000:00000001:3.0:1788880172.758137:0:203405:0:(ldlm_lockd.c:3036:ldlm_get_ref()) Process entered 00010000:00000001:3.0:1788880172.758140:0:203405:0:(ldlm_lockd.c:3045:ldlm_get_ref()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.758145:0:203405:0:(genops.c:1285:class_new_import()) kmalloced '(imp)': 1720 at 00000000beb50720. 00000020:00000040:3.0:1788880172.758148:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-OST0000-osc-ffff8ca787527000 (00000000bd6a9017) now 3 - import 00000020:00000001:3.0:1788880172.758171:0:203405:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880172.758174:0:203405:0:(genops.c:1244:class_import_put()) import 00000000beb50720 refcount=1 obd=lustre-OST0000-osc-ffff8ca787527000 00000020:00000001:3.0:1788880172.758177:0:203405:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:3.0:1788880172.758180:0:203405:0:(ldlm_lib.c:49:import_set_conn()) Process entered 00000100:00000200:3.0:1788880172.758194:0:203405:0:(events.c:559:ptlrpc_uuid_to_peer()) 192.168.204.111@tcp->12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880172.758198:0:203405:0:(connection.c:104:ptlrpc_connection_get()) Process entered 00000400:00000200:3.0:1788880172.758208:0:203405:0:(peer.c:1550:LNetPrimaryNID()) NID 192.168.204.111@tcp primary NID 192.168.204.111@tcp rc 0 00000100:00000001:3.0:1788880172.758212:0:203405:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788880172.758218:0:203405:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 6 to 192.168.204.111@tcp 00000100:00000001:3.0:1788880172.758221:0:203405:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000001:3.0:1788880172.758224:0:203405:0:(connection.c:110:ptlrpc_connection_get()) Process leaving via out (rc=18446617250405033856 : -126823304517760 : 0xffff8ca7a5fc2780) 00000100:00000040:3.0:1788880172.758230:0:203405:0:(connection.c:149:ptlrpc_connection_get()) conn=000000008a5be7d5 refcount 6 to 192.168.204.111@tcp 00000100:00000040:3.0:1788880172.758287:0:203405:0:(client.c:137:ptlrpc_uuid_to_connection()) 192.168.204.111@tcp -> 000000008a5be7d5 00010000:00000010:3.0:1788880172.758293:0:203405:0:(ldlm_lib.c:70:import_set_conn()) kmalloced '(imp_conn)': 88 at 0000000050701c82. 00010000:00080000:3.0:1788880172.758297:0:203405:0:(ldlm_lib.c:102:import_set_conn()) imp 00000000beb50720@lustre-OST0000-osc-ffff8ca787527000: add connection 192.168.204.111@tcp at head 00010000:00000001:3.0:1788880172.758301:0:203405:0:(ldlm_lib.c:109:import_set_conn()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880172.758306:0:203405:0:(ldlm_resource.c:976:ldlm_namespace_new()) Process entered 00010000:00000001:3.0:1788880172.758308:0:203405:0:(ldlm_lockd.c:3036:ldlm_get_ref()) Process entered 00010000:00000001:3.0:1788880172.758311:0:203405:0:(ldlm_lockd.c:3045:ldlm_get_ref()) Process leaving (rc=0 : 0 : 0) 00010000:00000010:3.0:1788880172.758316:0:203405:0:(ldlm_resource.c:993:ldlm_namespace_new()) kmalloced '(ns)': 848 at 000000001539e75c. 00000001:00000001:3.0:1788880172.758320:0:203405:0:(hash.c:1038:cfs_hash_create()) Process entered 00000001:00000010:3.0:1788880172.758324:0:203405:0:(hash.c:1065:cfs_hash_create()) alloc '(hs)': 280 at 00000000192406cc. 00000001:00000010:3.0:1788880172.758334:0:203405:0:(hash.c:933:cfs_hash_buckets_realloc()) alloc '(new_bkts)': 128 at 000000006566d7d7. 00000001:00000010:3.0:1788880172.758340:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 000000009bb7a2fc. 00000001:00000010:3.0:1788880172.758348:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 00000000daef27d6. 00000001:00000010:3.0:1788880172.758357:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 000000000faf13ed. 00000001:00000010:3.0:1788880172.758364:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 00000000b69c728b. 00000001:00000010:3.0:1788880172.758386:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 00000000482661c4. 00000001:00000010:3.0:1788880172.758392:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 00000000a8683e5c. 00000001:00000010:3.0:1788880172.758399:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 0000000002065ace. 00000001:00000010:3.0:1788880172.758406:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 00000000cc7b207f. 00000001:00000010:3.0:1788880172.758413:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 0000000001b29d19. 00000001:00000010:3.0:1788880172.758421:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 00000000cffe43f7. 00000001:00000010:3.0:1788880172.758427:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 00000000e859184c. 00000001:00000010:3.0:1788880172.758434:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 000000007383eca8. 00000001:00000010:3.0:1788880172.758440:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 0000000075cda7fb. 00000001:00000010:3.0:1788880172.758447:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 000000005360b6e7. 00000001:00000010:3.0:1788880172.758453:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 00000000fab281f9. 00000001:00000010:3.0:1788880172.758472:0:203405:0:(hash.c:946:cfs_hash_buckets_realloc()) alloc '(new_bkts[i])': 4272 at 000000005cc9430a. 00000020:00000010:3.0:1788880172.758497:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 000000006806ae46. 00000020:00000010:3.0:1788880172.758504:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 96 at 00000000043c7ff4. 00010000:00000001:3.0:1788880172.758689:0:203405:0:(ldlm_pool.c:891:ldlm_pool_init()) Process entered 00010000:00000001:3.0:1788880172.758736:0:203405:0:(ldlm_pool.c:810:ldlm_pool_debugfs_init()) Process entered 00000020:00000010:3.0:1788880172.758793:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 00000000b50720f2. 00000020:00000010:3.0:1788880172.758799:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 352 at 0000000050075317. 00010000:00000001:3.0:1788880172.758824:0:203405:0:(ldlm_pool.c:865:ldlm_pool_debugfs_init()) Process leaving 00010000:00010000:3.0:1788880172.758827:0:203405:0:(ldlm_pool.c:925:ldlm_pool_init()) Lock pool ldlm-pool-lustre-OST0000-osc-ffff8ca787527000-3 is initialized 00010000:00000001:3.0:1788880172.758830:0:203405:0:(ldlm_pool.c:927:ldlm_pool_init()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880172.758833:0:203405:0:(ldlm_resource.c:1084:ldlm_namespace_new()) Process leaving (rc=18446617249924412416 : -126823785139200 : ffff8ca789567400) 00010000:00000001:3.0:1788880172.758838:0:203405:0:(ldlm_lib.c:603:client_obd_setup()) Process leaving (rc=0 : 0 : 0) 00000008:00000020:3.0:1788880172.758850:0:203405:0:(osc_request.c:745:osc_update_next_shrink()) next time 12336 to shrink grant 00000008:00000001:3.0:1788880172.758854:0:203405:0:(osc_request.c:4083:osc_setup_common()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.759310:0:203405:0:(lprocfs_status.c:1341:lprocfs_obd_setup()) Process leaving via already_registered (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.759352:0:203405:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 00000000f87a5e62. 00000020:00000010:3.0:1788880172.759356:0:203405:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 3680 at 00000000c1305ece. 00000100:00000010:3.0:1788880172.759397:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000004908e872. 00000100:00000010:3.0:1788880172.759552:0:203405:0:(client.c:658:ptlrpc_add_rqs_to_pool()) kmalloced '(msg)': 524288 at 000000008607f1ee. 00000100:00000010:3.0:1788880172.759559:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d83c275f. 00000100:00000010:3.0:1788880172.759725:0:203405:0:(client.c:658:ptlrpc_add_rqs_to_pool()) kmalloced '(msg)': 524288 at 000000007803c60d. 00000100:00000010:3.0:1788880172.759739:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000af5b70a6. 00000100:00000010:3.0:1788880172.759941:0:203405:0:(client.c:658:ptlrpc_add_rqs_to_pool()) kmalloced '(msg)': 524288 at 00000000afb45d35. 00000100:00000010:3.0:1788880172.759948:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000a9e6e3b4. 00000100:00000010:3.0:1788880172.760113:0:203405:0:(client.c:658:ptlrpc_add_rqs_to_pool()) kmalloced '(msg)': 524288 at 00000000002f4460. 00000100:00000010:3.0:1788880172.760120:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000030150b8f. 00000100:00000010:3.0:1788880172.760314:0:203405:0:(client.c:658:ptlrpc_add_rqs_to_pool()) kmalloced '(msg)': 524288 at 00000000a145e79f. 00000100:00000010:3.0:1788880172.760320:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000065ea33e9. 00000100:00000010:3.0:1788880172.760470:0:203405:0:(client.c:658:ptlrpc_add_rqs_to_pool()) kmalloced '(msg)': 524288 at 00000000d026c8fb. 00000100:00000010:3.0:1788880172.760475:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000a7bc402e. 00000100:00000010:3.0:1788880172.760630:0:203405:0:(client.c:658:ptlrpc_add_rqs_to_pool()) kmalloced '(msg)': 524288 at 000000000349631a. 00000100:00000010:3.0:1788880172.760637:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000253535e1. 00000100:00000010:3.0:1788880172.760793:0:203405:0:(client.c:658:ptlrpc_add_rqs_to_pool()) kmalloced '(msg)': 524288 at 0000000095323955. 00000100:00000010:3.0:1788880172.760798:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000782d610a. 00000100:00000010:3.0:1788880172.760965:0:203405:0:(client.c:658:ptlrpc_add_rqs_to_pool()) kmalloced '(msg)': 524288 at 0000000039c77fc1. 00000100:00000010:3.0:1788880172.760970:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000004045c868. 00000100:00000010:3.0:1788880172.761140:0:203405:0:(client.c:658:ptlrpc_add_rqs_to_pool()) kmalloced '(msg)': 524288 at 0000000018f70865. 00000008:00000001:3.0:1788880172.761146:0:203405:0:(osc_request.c:4137:osc_setup()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:3.0:1788880172.761150:0:203405:0:(osc_dev.c:213:osc_device_alloc()) Process leaving (rc=18446617250661744640 : -126823047806976 : ffff8ca7b5494000) 00000008:00000001:3.0:1788880172.761154:0:203405:0:(osc_dev.c:155:osc_device_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.761158:0:203405:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000d0916d04. 00000020:00000010:3.0:1788880172.761165:0:203405:0:(cl_object.c:1137:cl_key_fini()) kfreed 'info': 72 at 00000000e83229b9. 00000008:00000010:3.0:1788880172.761169:0:203405:0:(osc_dev.c:100:osc_key_fini()) slab-freed '(info)': 2864 at 00000000aeeba8e2. 00020000:00000010:3.0:1788880172.761174:0:203405:0:(lov_dev.c:95:lov_key_fini()) slab-freed '(info)': 192 at 00000000b8c7c243. 00008000:00000010:3.0:1788880172.761179:0:203405:0:(echo_client.c:424:echo_thread_key_fini()) slab-freed '(info)': 904 at 00000000d0b9f970. 00000080:00000010:3.0:1788880172.761185:0:203405:0:(vvp_dev.c:80:ll_thread_key_fini()) slab-freed '(lti)': 168 at 0000000020ae732f. 00000080:00000010:3.0:1788880172.761193:0:203405:0:(vvp_dev.c:130:vvp_thread_key_fini()) slab-freed '(vti)': 592 at 000000007ec0a7c6. 00000020:00000010:3.0:1788880172.761199:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000937f3d66. 00000020:00000010:3.0:1788880172.761206:0:203405:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 0000000014f65f10. 00000020:00000010:3.0:1788880172.761212:0:203405:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 000000000bcc11c1. 00000008:00000010:3.0:1788880172.761218:0:203405:0:(osc_dev.c:125:osc_session_fini()) slab-freed '(info)': 352 at 00000000ae551350. 00020000:00000010:3.0:1788880172.761221:0:203405:0:(lov_dev.c:136:lov_session_key_fini()) slab-freed '(info)': 584 at 000000009b9eb526. 00008000:00000010:3.0:1788880172.761228:0:203405:0:(echo_client.c:449:echo_session_key_fini()) slab-freed '(session)': 8 at 00000000db05d82b. 00000080:00000010:3.0:1788880172.761273:0:203405:0:(vvp_dev.c:105:vvp_session_key_fini()) slab-freed '(session)': 240 at 00000000e9f8c309. 00000020:00000010:3.0:1788880172.761278:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000016c67947. 00000020:00000001:3.0:1788880172.761284:0:203405:0:(obd_class.h:601:obd_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880172.761288:0:203405:0:(obd_config.c:994:class_incref()) incref lustre-OST0000-osc-ffff8ca787527000 (00000000bd6a9017) now 4 - setup 00000020:00000080:3.0:1788880172.761293:0:203405:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-OST0000-osc-ffff8ca787527000 (uuid ea27006f-52a8-4007-b69a-dcac8279445a) 00000020:00000001:3.0:1788880172.761295:0:203405:0:(obd_config.c:828:class_setup()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.761297:0:203405:0:(obd_config.c:1618:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.761302:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 136 at 0000000047bb7169. 00000020:00000010:3.0:1788880172.761307:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 39 at 000000009e7d9176. 00000020:00000001:3.0:1788880172.761315:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.761321:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf35f0 type=0x10620000 idx=34 00000040:00001000:3.0:1788880172.761326:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 34 lrh_len: 128 (2576 remains) 00000040:00001000:3.0:1788880172.761329:0:203405:0:(llog.c:805:llog_process_thread()) index: 34, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.761332:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:3.0:1788880172.761336:0:203405:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 34 at 000000009e7d9176. 00000020:01000000:3.0:1788880172.761339:0:203405:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf00d, instance name: lustre-clilov-ffff8ca787527000 00000020:00000010:3.0:1788880172.761342:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 120 at 0000000014f65f10. 00000020:00000080:3.0:1788880172.761344:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf00d 00000020:00000001:3.0:1788880172.761350:0:203405:0:(obd_class.h:689:obd_process_config()) Process entered 00000020:00000010:3.0:1788880172.761356:0:203405:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000d0916d04. 00000020:00000010:3.0:1788880172.761360:0:203405:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 0000000004f7ba43. 00000020:00000010:3.0:1788880172.761364:0:203405:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000e83229b9. 00000008:00000010:3.0:1788880172.761369:0:203405:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000aeeba8e2. 00020000:00000010:3.0:1788880172.761372:0:203405:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000b8c7c243. 00008000:00000010:3.0:1788880172.761374:0:203405:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000d0b9f970. 00000080:00000010:3.0:1788880172.761380:0:203405:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 0000000020ae732f. 00000080:00000010:3.0:1788880172.761384:0:203405:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 000000007ec0a7c6. 00020000:00000001:3.0:1788880172.761388:0:203405:0:(lov_obd.c:814:lov_process_config_base()) Process entered 00020000:00000001:3.0:1788880172.761394:0:203405:0:(lov_obd.c:477:lov_add_target()) Process entered 00020000:01000000:3.0:1788880172.761396:0:203405:0:(lov_obd.c:479:lov_add_target()) uuid:lustre-OST0000_UUID idx:0 gen:1 active:1 00020000:00000010:3.0:1788880172.761406:0:203405:0:(lov_obd.c:502:lov_add_target()) kmalloced '(tgt)': 368 at 000000007b4712cb. 00020000:01000000:3.0:1788880172.761408:0:203405:0:(lov_obd.c:508:lov_add_target()) lustre-clilov-ffff8ca787527000: Adding target lustre-OST0000_UUID 00020000:00000001:3.0:1788880172.761410:0:203405:0:(lu_tgt_descs.c:352:ltd_add_tgt()) Process entered 00020000:00000010:3.0:1788880172.761424:0:203405:0:(lu_tgt_descs.c:371:ltd_add_tgt()) kmalloced '(ltd->ltd_tgt_idx[index / (((1UL) << 12) / sizeof(void *))])': 4096 at 00000000c70670ce. 00020000:00000001:3.0:1788880172.761427:0:203405:0:(lu_tgt_descs.c:383:ltd_add_tgt()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.761430:0:203405:0:(lu_tgt_pool.c:120:lu_tgt_pool_add_lock()) Process entered 00000020:00000001:3.0:1788880172.761433:0:203405:0:(lu_tgt_pool.c:138:lu_tgt_pool_add_lock()) Process leaving 00020000:01000000:3.0:1788880172.761436:0:203405:0:(lov_obd.c:530:lov_add_target()) idx=0 ltd_gen=1 ld_tgt_count=1 00020000:00000001:3.0:1788880172.761438:0:203405:0:(lov_obd.c:537:lov_add_target()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:3.0:1788880172.761440:0:203405:0:(lov_obd.c:846:lov_process_config_base()) Process leaving via out (rc=0 : 0 : 0x0) 00020000:00000001:3.0:1788880172.761443:0:203405:0:(lov_obd.c:868:lov_process_config_base()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:3.0:1788880172.761445:0:203405:0:(lov_dev.c:395:lov_cl_add_target()) Process entered 00020000:00000001:3.0:1788880172.761447:0:203405:0:(lov_dev.c:350:lov_expand_targets()) Process entered 00020000:00000010:3.0:1788880172.761454:0:203405:0:(lov_dev.c:358:lov_expand_targets()) kmalloced '(newd)': 512 at 000000000d032c7e. 00020000:00000001:3.0:1788880172.761457:0:203405:0:(lov_dev.c:372:lov_expand_targets()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:3.0:1788880172.761459:0:203405:0:(lov_dev.c:424:lov_cl_add_target()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:3.0:1788880172.761462:0:203405:0:(lov_dev.c:573:lov_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.761465:0:203405:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 0000000004f7ba43. 00000020:00000010:3.0:1788880172.761469:0:203405:0:(cl_object.c:1137:cl_key_fini()) kfreed 'info': 72 at 00000000e83229b9. 00000008:00000010:3.0:1788880172.761472:0:203405:0:(osc_dev.c:100:osc_key_fini()) slab-freed '(info)': 2864 at 00000000aeeba8e2. 00020000:00000010:3.0:1788880172.761476:0:203405:0:(lov_dev.c:95:lov_key_fini()) slab-freed '(info)': 192 at 00000000b8c7c243. 00008000:00000010:3.0:1788880172.761479:0:203405:0:(echo_client.c:424:echo_thread_key_fini()) slab-freed '(info)': 904 at 00000000d0b9f970. 00000080:00000010:3.0:1788880172.761486:0:203405:0:(vvp_dev.c:80:ll_thread_key_fini()) slab-freed '(lti)': 168 at 0000000020ae732f. 00000080:00000010:3.0:1788880172.761490:0:203405:0:(vvp_dev.c:130:vvp_thread_key_fini()) slab-freed '(vti)': 592 at 000000007ec0a7c6. 00000020:00000010:3.0:1788880172.761492:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000d0916d04. 00000020:00000001:3.0:1788880172.761495:0:203405:0:(obd_class.h:704:obd_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880172.761498:0:203405:0:(obd_config.c:1693:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.761500:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 120 at 0000000014f65f10. 00000020:00000010:3.0:1788880172.761503:0:203405:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 34 at 000000009e7d9176. 00000020:00000001:3.0:1788880172.761506:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.761507:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3670 type=0x10620000 idx=35 00000040:00001000:3.0:1788880172.761510:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 35 lrh_len: 224 (2448 remains) 00000040:00001000:3.0:1788880172.761511:0:203405:0:(llog.c:805:llog_process_thread()) index: 35, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.761513:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:3.0:1788880172.761515:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:3.0:1788880172.761517:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:3.0:1788880172.761518:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000010:3.0:1788880172.761525:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 000000008c752d14. 00000020:00000080:3.0:1788880172.761540:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1788880172.761542:0:203405:0:(obd_config.c:1561:class_process_config()) marker 24 (0x2) lustre-OST0000 add osc 00000020:00000001:3.0:1788880172.761545:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.761549:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 000000008c752d14. 00000020:00000001:3.0:1788880172.761552:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.761554:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3750 type=0x10620000 idx=36 00000040:00001000:3.0:1788880172.761556:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 36 lrh_len: 224 (2224 remains) 00000040:00001000:3.0:1788880172.761558:0:203405:0:(llog.c:805:llog_process_thread()) index: 36, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.761559:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:3.0:1788880172.761560:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:3.0:1788880172.761561:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:3.0:1788880172.761562:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000001:3.0:1788880172.761564:0:203405:0:(obd_mount.c:953:lustre_check_exclusion()) Process entered 00000020:00000001:3.0:1788880172.761566:0:203405:0:(obd_mount.c:958:lustre_check_exclusion()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:3.0:1788880172.761569:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 000000008c752d14. 00000020:00000080:3.0:1788880172.761570:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1788880172.761571:0:203405:0:(obd_config.c:1561:class_process_config()) marker 25 (0x1) lustre sys.timeout 00000020:00000001:3.0:1788880172.761573:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.761575:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 000000008c752d14. 00000020:00000001:3.0:1788880172.761576:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.761578:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3830 type=0x10620000 idx=37 00000040:00001000:3.0:1788880172.761578:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 37 lrh_len: 80 (2000 remains) 00000040:00001000:3.0:1788880172.761579:0:203405:0:(llog.c:805:llog_process_thread()) index: 37, lh_last_idx: 38 synced_idx: 0 lgh_last_idx: 38 00000020:00000001:3.0:1788880172.761580:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:3.0:1788880172.761585:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 56 at 000000009e7d9176. 00000020:00000080:3.0:1788880172.761586:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf009 00000020:00000080:3.0:1788880172.761587:0:203405:0:(obd_config.c:1535:class_process_config()) changing lustre timeout from 100 to 20 00000020:00000001:3.0:1788880172.761588:0:203405:0:(obd_config.c:1539:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880172.761590:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 56 at 000000009e7d9176. 00000020:00000001:3.0:1788880172.761593:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:3.0:1788880172.761594:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3880 type=0x10620000 idx=38 00000040:00001000:3.0:1788880172.761596:0:203405:0:(llog.c:728:llog_process_thread()) synced_idx: 38 00000040:00000001:3.0:1788880172.761600:0:203405:0:(lustre_log.h:475:llog_next_block()) Process entered 00000040:00000001:3.0:1788880172.761602:0:203405:0:(llog_client.c:140:llog_client_next_block()) Process entered 00000020:00000040:3.0:1788880172.761616:0:203405:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:3.0:1788880172.761622:0:203405:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000020:00000040:3.0:1788880172.761625:0:203405:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880172.761628:0:203405:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880172.761632:0:203405:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880172.761638:0:203405:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880172.761644:0:203405:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000d0916d04. 00000100:00000001:3.0:1788880172.761652:0:203405:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.761658:0:203405:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:3.0:1788880172.761660:0:203405:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:3.0:1788880172.761667:0:203405:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000047bb7169. 00000100:00000001:3.0:1788880172.761670:0:203405:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063673984 : -126823645877632 : ffff8ca791a36a80) 00000100:00000001:3.0:1788880172.761672:0:203405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880172.761674:0:203405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000001:3.0:1788880172.761678:0:203405:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880172.761681:0:203405:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.761684:0:203405:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:3.0:1788880172.761686:0:203405:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880172.761695:0:203405:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780633088/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.761704:0:203405:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880172.761706:0:203405:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880172.761711:0:203405:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880172.761714:0:203405:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880172.761717:0:203405:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880172.761723:0:203405:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job llog_process_th:fa4856d5-51a8-499e-b3f8-b477743a3dec:203405:1875776780633088:192.168.204.111@tcp:502: 00000100:00000001:3.0:1788880172.761728:0:203405:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880172.761735:0:203405:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880172.761737:0:203405:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880172.761740:0:203405:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880172.761755:0:203405:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 16384 at 0000000008985801. 02000000:00000001:3.0:1788880172.761760:0:203405:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880172.761768:0:203405:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000067d070dd. 00000400:00000010:3.0:1788880172.761776:0:203405:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 000000009a18536c. 00000100:00000200:3.0:1788880172.761791:0:203405:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 16384 bytes, xid 1875776780633088, portal 25 00000100:00000001:3.0:1788880172.761793:0:203405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880172.761795:0:203405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000040:3.0:1788880172.761802:0:203405:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780633088/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.761810:0:203405:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880172.761814:0:203405:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880172.761819:0:203405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eab8d071. 00000100:00000200:3.0:1788880172.761824:0:203405:0:(niobuf.c:94:ptl_send_buf()) Sending 272 bytes to portal 26, xid 1875776780633088, offset 0 00000400:00000200:3.0:1788880172.761837:0:203405:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880172.761854:0:203405:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880172.761860:0:203405:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880172.761864:0:203405:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880172.761867:0:203405:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880172.761871:0:203405:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880172.761878:0:203405:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [11:11:256:4294967295] 192.168.204.111@tcp LPNI seq info [11:11:8:4294967295] 00000400:00000200:3.0:1788880172.761895:0:203405:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880172.761906:0:203405:0:(socklnd_cb.c:931:ksocknal_send()) sending 272 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880172.761911:0:203405:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000800:00000200:3.0:1788880172.761919:0:203405:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880172.761928:0:203405:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880172.761933:0:203405:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 368 niov 1 nkiov 1 00000100:00000001:3.0:1788880172.761954:0:203405:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.761960:0:203405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305803870 00000100:00000010:3.0:1788880172.761967:0:203405:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e83229b9. 00000100:00000001:3.0:1788880172.761976:0:203405:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880172.761987:0:203405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305803870 00000100:00000010:3.0:1788880172.761992:0:203405:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a9d0cfcc. 00000100:00000001:3.0:1788880172.761997:0:203405:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880172.762004:0:203405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305803870 00000100:00000010:3.0:1788880172.762019:0:203405:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005869c8be. 00000100:00000001:3.0:1788880172.762026:0:203405:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880172.762041:0:203405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305803870 00000100:00000010:3.0:1788880172.762048:0:203405:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000594debb1. 00000100:00000001:3.0:1788880172.762053:0:203405:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880172.762057:0:203405:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880172.762061:0:203405:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.762063:0:203405:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880172.762066:0:203405:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:3.0:1788880172.762068:0:203405:0:(client.c:2702:ptlrpc_set_wait()) set 0000000047bb7169 going to sleep for 11 seconds 00000100:00000001:3.0:1788880172.762073:0:203405:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880172.762077:0:203405:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880172.762121:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880172.762127:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:0.0:1788880172.762133:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.762146:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880172.762154:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eab8d071 00000400:00000010:0.0:1788880172.762156:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eab8d071. 00000100:00000001:0.0:1788880172.762162:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880172.762170:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780633088/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.762181:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.762183:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000d7dcc0ad x1875776780633088/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.762188:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.762189:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880172.763916:0:203405:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880172.763925:0:203405:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880172.763932:0:203405:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880172.763936:0:203405:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000067d070dd. 00000400:00000200:0.0:1788880172.763940:0:203405:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009a18536c 00000400:00000010:0.0:1788880172.763942:0:203405:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 000000009a18536c. 00000100:00000001:0.0:1788880172.763946:0:203405:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.763952:0:203405:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780633088/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880172.763962:0:203405:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780633088/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.763969:0:203405:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880172.763971:0:203405:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.763974:0:203405:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880172.763978:0:203405:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880172.763981:0:203405:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880172.763984:0:203405:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.763988:0:203405:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880172.763997:0:203405:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=0 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880172.764002:0:203405:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=0 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880172.764005:0:203405:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880172.764007:0:203405:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.764010:0:203405:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880172.764013:0:203405:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.764023:0:203405:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880172.764040:0:203405:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780633088/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.764047:0:203405:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880172.764051:0:203405:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880172.764053:0:203405:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880172.764057:0:203405:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780633088/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880172.764064:0:203405:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job llog_process_th:fa4856d5-51a8-499e-b3f8-b477743a3dec:203405:1875776780633088:192.168.204.111@tcp:502: 00000100:00000001:0.0:1788880172.764071:0:203405:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.764073:0:203405:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.764077:0:203405:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:0.0:1788880172.764079:0:203405:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.764082:0:203405:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780633088/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.764089:0:203405:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880172.764092:0:203405:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000047bb7169. 00000100:00000001:0.0:1788880172.764095:0:203405:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:0.0:1788880172.764097:0:203405:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:0.0:1788880172.764107:0:203405:0:(llog_client.c:198:llog_client_next_block()) Process leaving 00000100:00000001:0.0:1788880172.764108:0:203405:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.764111:0:203405:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780633088/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880188 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.764118:0:203405:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880172.764121:0:203405:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880172.764124:0:203405:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 16384 at 0000000008985801. 02000000:00000001:0.0:1788880172.764137:0:203405:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880172.764141:0:203405:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880172.764143:0:203405:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880172.764146:0:203405:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880172.764149:0:203405:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000d0916d04. 02000000:00000001:0.0:1788880172.764154:0:203405:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880172.764156:0:203405:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880172.764166:0:203405:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:0.0:1788880172.764170:0:203405:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880172.764172:0:203405:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:0.0:1788880172.764175:0:203405:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880172.764177:0:203405:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880172.764179:0:203405:0:(genops.c:1251:class_import_put()) Process leaving 00000040:00000001:0.0:1788880172.764182:0:203405:0:(lustre_log.h:485:llog_next_block()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:0.0:1788880172.764185:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3880 type=0x10620000 idx=38 00000040:00001000:0.0:1788880172.764190:0:203405:0:(llog.c:784:llog_process_thread()) lrh_index: 38 lrh_len: 224 (1920 remains) 00000040:00001000:0.0:1788880172.764192:0:203405:0:(llog.c:805:llog_process_thread()) index: 38, lh_last_idx: 38 synced_idx: 38 lgh_last_idx: 38 00000020:00000001:0.0:1788880172.764196:0:203405:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:0.0:1788880172.764199:0:203405:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:0.0:1788880172.764201:0:203405:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:0.0:1788880172.764203:0:203405:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000010:0.0:1788880172.764210:0:203405:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 00000000f92e72a2. 00000020:00000080:0.0:1788880172.764213:0:203405:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:0.0:1788880172.764216:0:203405:0:(obd_config.c:1561:class_process_config()) marker 25 (0x2) lustre sys.timeout 00000020:00000001:0.0:1788880172.764219:0:203405:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:0.0:1788880172.764222:0:203405:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 00000000f92e72a2. 00000020:00000001:0.0:1788880172.764227:0:203405:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:0.0:1788880172.764229:0:203405:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3960 type=0x0 idx=0 00000040:00000001:0.0:1788880172.764250:0:203405:0:(llog.c:684:llog_process_thread()) Process leaving via out (rc=0 : 0 : 0x0) 00000040:00080000:0.0:1788880172.764254:0:203405:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 39 count 39 00000040:00000010:0.0:1788880172.764257:0:203405:0:(llog.c:899:llog_process_thread()) kvfree 'buf': 8192 at 00000000c092b841. 00000020:00000010:0.0:1788880172.764268:0:203405:0:(lu_object.c:2058:lu_env_remove()) kfreed_rcu 'lei': 40 at 0000000095cce73e. 00000020:00000010:0.0:1788880172.764281:0:203405:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 0000000027d4cece. 00000040:00000010:0.0:1788880172.764287:0:203405:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000c7975147. 00000020:00000010:0.0:1788880172.764292:0:203405:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 000000001e393693. 00000020:00000010:0.0:1788880172.764299:0:203405:0:(tgt_main.c:741:tgt_key_fini()) slab-freed '(info)': 1288 at 000000004a155362. 00000020:00000010:0.0:1788880172.764306:0:203405:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 000000009a0ddd94. 80000000:00000010:0.0:1788880172.764311:0:203405:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 00000000bdb31c0d. 40000000:00000010:0.0:1788880172.764316:0:203405:0:(fid_handler.c:479:seq_key_fini()) kfreed 'info': 48 at 000000009978e132. 00100000:00000010:0.0:1788880172.764322:0:203405:0:(lfsck_lib.c:43:lfsck_key_fini()) kfreed 'info': 1800 at 00000000d610634e. 00040000:00000010:0.0:1788880172.764327:0:203405:0:(lquota_lib.c:35:lquota_key_fini()) kfreed 'info': 248 at 000000009cf93093. 00040000:00000010:0.0:1788880172.764333:0:203405:0:(qmt_dev.c:418:qmt_key_fini()) kfreed 'info': 1256 at 0000000004b8209f. 00040000:00000010:0.0:1788880172.764345:0:203405:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 000000003720a685. 00000004:00000010:0.0:1788880172.764349:0:203405:0:(mdd_device.c:2420:mdd_key_fini()) kfreed 'info': 1872 at 00000000c85a7d49. 00000004:00000010:0.0:1788880172.764356:0:203405:0:(mdt_handler.c:8495:mdt_key_fini()) kfreed 'info': 2568 at 000000002af8b51d. 00000004:00000010:0.0:1788880172.764363:0:203405:0:(lod_dev.c:2378:lod_key_fini()) kfreed 'info': 1224 at 000000001a0c73ca. 00002000:00000010:0.0:1788880172.764369:0:203405:0:(ofd_dev.c:3002:ofd_key_fini()) kfreed 'info': 816 at 00000000b34868d0. 00000004:00000010:0.0:1788880172.764373:0:203405:0:(osp_dev.c:1888:osp_key_fini()) kfreed 'info': 616 at 00000000813c171c. 00000020:00000010:0.0:1788880172.764378:0:203405:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000b1120e74. 00000040:00000010:3.0:1788880172.764402:0:203392:0:(llog.c:994:llog_process_or_fork()) kfreed 'lpi': 96 at 000000002b2fe58c. 00000040:00000001:3.0:1788880172.764409:0:203392:0:(llog.c:995:llog_process_or_fork()) Process leaving (rc=0 : 0 : 0) 00000020:01000000:3.0:1788880172.764412:0:203392:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 1-38 (rc=0) 00000040:00000010:3.0:1788880172.764416:0:203392:0:(llog.c:70:llog_free_handle()) kvfree 'loghandle->lgh_hdr': 8192 at 000000002063d487. 00000040:00000010:3.0:1788880172.764420:0:203392:0:(llog.c:72:llog_free_handle()) kfreed 'loghandle': 352 at 00000000dc9842d3. 00000020:00000001:3.0:1788880172.764422:0:203392:0:(obd_config.c:2151:class_config_parse_llog()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:3.0:1788880172.764425:0:203392:0:(mgc_request.c:1724:mgc_process_cfg_log()) Process leaving 00000020:00000010:3.0:1788880172.764428:0:203392:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000356bd8da. 00000040:00000010:3.0:1788880172.764430:0:203392:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000f002f5b3. 00000020:00000010:3.0:1788880172.764432:0:203392:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 00000000df738f23. 00000020:00000010:3.0:1788880172.764435:0:203392:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 00000000abce1aea. 80000000:00000010:3.0:1788880172.764438:0:203392:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 000000003c9d1628. 00040000:00000010:3.0:1788880172.764441:0:203392:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 00000000cf77ae66. 20000000:00000010:3.0:1788880172.764445:0:203392:0:(mgs_handler.c:1787:mgs_key_fini()) kfreed 'info': 392 at 0000000030240e3c. 00000020:00000010:3.0:1788880172.764448:0:203392:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000ba022444. 10000000:00000010:3.0:1788880172.764450:0:203392:0:(mgc_request.c:1730:mgc_process_cfg_log()) kfreed 'env': 56 at 000000007bbc051d. 10000000:01000000:3.0:1788880172.764454:0:203392:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'lustre-client' succeeded (0). 10000000:00000001:3.0:1788880172.764468:0:203392:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:3.0:1788880172.764471:0:203392:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:3.0:1788880172.764474:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880172.764478:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000000e3a19ac refcount=3 00000020:00000001:3.0:1788880172.764480:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:3.0:1788880172.764493:0:203392:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:3.0:1788880172.764496:0:203392:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:3.0:1788880172.764501:0:203392:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a8865f lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7909 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:3.0:1788880172.764512:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:3.0:1788880172.764514:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:3.0:1788880172.764516:0:203392:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a8865f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7909 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:3.0:1788880172.764523:0:203392:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:3.0:1788880172.764525:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:3.0:1788880172.764526:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:3.0:1788880172.764528:0:203392:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:3.0:1788880172.764530:0:203392:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:3.0:1788880172.764533:0:203392:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:3.0:1788880172.764535:0:203392:0:(mgc_request.c:1801:mgc_process_log()) Process log lustre-cliir-ffff8ca787527000 from 1 10000000:00000001:3.0:1788880172.764538:0:203392:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:3.0:1788880172.764540:0:203392:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00000100:00000010:3.0:1788880172.764546:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000020:00000040:3.0:1788880172.764549:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880172.764554:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880172.764562:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880172.764567:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880172.764575:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000fdddab51. 00000100:00000001:3.0:1788880172.764582:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880172.764587:0:203392:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:3.0:1788880172.764590:0:203392:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:3.0:1788880172.764598:0:203392:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at 00000000aee22452. 00010000:00000001:3.0:1788880172.764603:0:203392:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:3.0:1788880172.764607:0:203392:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 000000001ffdd456. 00000020:00000010:3.0:1788880172.764614:0:203392:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 128 at 00000000436e48be. 00000020:00000001:3.0:1788880172.764618:0:203392:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:3.0:1788880172.764620:0:203392:0:(lustre_handles.c:93:class_handle_hash()) added object 000000001ffdd456 with handle 0x805d548a36a88689 to hash 00000020:00000001:3.0:1788880172.764622:0:203392:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:3.0:1788880172.764631:0:203392:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000001:3.0:1788880172.764634:0:203392:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000001:3.0:1788880172.764640:0:203392:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:3.0:1788880172.764643:0:203392:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880172.764646:0:203392:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788880172.764653:0:203392:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788880172.764674:0:203392:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:3.0:1788880172.764681:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:3.0:1788880172.764683:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:3.0:1788880172.764690:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000c9db1781. 00000100:00000001:3.0:1788880172.764693:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063672256 : -126823645879360 : ffff8ca791a363c0) 00000100:00000001:3.0:1788880172.764695:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880172.764696:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000100:00000001:3.0:1788880172.764700:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880172.764702:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.764704:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:3.0:1788880172.764705:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880172.764708:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.764714:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880172.764716:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880172.764720:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880172.764722:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880172.764723:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880172.764725:0:203392:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780633216:192.168.204.111@tcp:101: 00000100:00000001:3.0:1788880172.764729:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880172.764740:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880172.764742:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880172.764744:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880172.764748:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000352996e3. 02000000:00000001:3.0:1788880172.764750:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880172.764754:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ce91c67e. 00000400:00000010:3.0:1788880172.764759:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cd783db0. 00000100:00000200:3.0:1788880172.764764:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780633216, portal 25 00000100:00000001:3.0:1788880172.764765:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880172.764766:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000100:00000040:3.0:1788880172.764769:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.764776:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880172.764780:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880172.764784:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000200:3.0:1788880172.764787:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780633216, offset 0 00000400:00000200:3.0:1788880172.764794:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880172.764805:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880172.764812:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880172.764817:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880172.764821:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880172.764825:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880172.764829:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [12:12:256:4294967295] 192.168.204.111@tcp LPNI seq info [12:12:8:4294967295] 00000400:00000200:3.0:1788880172.764841:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880172.764847:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880172.764852:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000055d7d92. 00000800:00000200:3.0:1788880172.764860:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880172.764868:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880172.764872:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000055d7d92 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:3.0:1788880172.764894:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.764900:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305803873 00000100:00000001:3.0:1788880172.764909:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305803870 00000100:00000001:3.0:1788880172.764920:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305803873 00000100:00000001:3.0:1788880172.764924:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305803870 00000100:00000001:3.0:1788880172.764928:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305803873 00000100:00000001:3.0:1788880172.764931:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305803870 00000100:00000001:3.0:1788880172.764935:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305803873 00000100:00000001:3.0:1788880172.764939:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305803870 00000100:00000001:3.0:1788880172.764942:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880172.764945:0:203392:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.764948:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880172.764950:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:3.0:1788880172.764952:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 00000000c9db1781 going to sleep for 11 seconds 00000100:00000001:3.0:1788880172.764955:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880172.764957:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880172.764991:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880172.764996:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000055d7d92. 00000400:00000200:0.0:1788880172.765000:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.765005:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880172.765009:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:0.0:1788880172.765011:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:0.0:1788880172.765015:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880172.765019:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.765038:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.765041:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.765048:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.765050:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880172.766266:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.766276:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.766279:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.766296:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.766308:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880172.766323:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b00480 00000400:00000200:0.0:1788880172.766330:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0x159 [1] + 224 00000800:00000001:0.0:1788880172.766335:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.766344:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.766347:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.766352:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880172.766357:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.766369:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880172.766381:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.766399:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880172.766410:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.766414:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.766427:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880172.766433:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880172.766436:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880172.766438:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ce91c67e. 00000400:00000200:0.0:1788880172.766442:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cd783db0 00000400:00000010:0.0:1788880172.766444:0:203392:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cd783db0. 00000100:00000001:0.0:1788880172.766447:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.766450:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880172.766459:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.766466:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880172.766468:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.766471:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880172.766474:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880172.766476:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880172.766479:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.766482:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880172.766488:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=0 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880172.766493:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=0 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880172.766496:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880172.766498:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.766501:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880172.766503:0:203392:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.766506:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880172.766509:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.766527:0:203392:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880172.766530:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880172.766532:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880172.766536:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880172.766543:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780633216:192.168.204.111@tcp:101: 00000100:00000001:0.0:1788880172.766548:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.766551:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.766554:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:0.0:1788880172.766556:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.766559:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.766565:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880172.766568:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000c9db1781. 00000100:00000001:0.0:1788880172.766571:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:0.0:1788880172.766573:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.766576:0:203392:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:0.0:1788880172.766579:0:203392:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880172.766581:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880172.766585:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000001ffdd456 refcount=4 00000020:00000001:0.0:1788880172.766588:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000001:0.0:1788880172.766591:0:203392:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000040:0.0:1788880172.766597:0:203392:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 000000001ffdd456, remote cookie: 0x92f11317f60a7910, flags: 0x0 00010000:00000001:0.0:1788880172.766600:0:203392:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:0.0:1788880172.766605:0:203392:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:0.0:1788880172.766607:0:203392:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:0.0:1788880172.766609:0:203392:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:0.0:1788880172.766611:0:203392:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:0.0:1788880172.766613:0:203392:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x2:0x0].0x0 (00000000aee22452) refcount = 1 00010000:00010000:0.0:1788880172.766617:0:203392:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880172.766635:0:203392:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:0.0:1788880172.766638:0:203392:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:0.0:1788880172.766639:0:203392:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:0.0:1788880172.766642:0:203392:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:0.0:1788880172.766646:0:203392:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880172.766649:0:203392:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880172.766655:0:203392:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:0.0:1788880172.766657:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880172.766659:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880172.766661:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880172.766662:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880172.766664:0:203392:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.766667:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.766671:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780633216/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.766678:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880172.766681:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880172.766684:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000352996e3. 02000000:00000001:0.0:1788880172.766688:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880172.766690:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880172.766692:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880172.766695:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880172.766697:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000fdddab51. 02000000:00000001:0.0:1788880172.766701:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880172.766703:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880172.766705:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000001:0.0:1788880172.766710:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880172.766711:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:0.0:1788880172.766714:0:203392:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788880172.766717:0:203392:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:0.0:1788880172.766719:0:203392:0:(mgc_request.c:108:config_log_get()) log lustre-cliir (0000000008e74578) refs 2 10000000:00000001:0.0:1788880172.766722:0:203392:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.766728:0:203392:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880172.766730:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880172.766732:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000001ffdd456 refcount=3 00000020:00000001:0.0:1788880172.766733:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000001:0.0:1788880172.766736:0:203392:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000001:0.0:1788880172.766738:0:203392:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:0.0:1788880172.766739:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880172.766740:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880172.766742:0:203392:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788880172.766745:0:203392:0:(mgc_request.c:1519:mgc_process_recover_log()) Process entered 10000000:00000010:0.0:1788880172.766750:0:203392:0:(mgc_request.c:1531:mgc_process_recover_log()) kmalloced '(folios)': 2048 at 000000006aef07b8. 00000100:00000010:0.0:1788880172.767222:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ff0be673. 00000020:00000040:0.0:1788880172.767227:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880172.767230:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880172.767632:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880172.767636:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880172.767644:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000d653e4f6. 00000100:00000001:0.0:1788880172.767650:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.767655:0:203392:0:(client.c:227:ptlrpc_prep_bulk_imp()) Process entered 00000100:00000010:0.0:1788880172.767659:0:203392:0:(client.c:169:ptlrpc_new_bulk()) kmalloced '(desc)': 984 at 000000006ece4223. 00000100:00000010:0.0:1788880172.767664:0:203392:0:(client.c:174:ptlrpc_new_bulk()) kmalloced '(desc->bd_vec)': 4096 at 000000004244f03d. 00000020:00000040:0.0:1788880172.767666:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880172.767680:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:0.0:1788880172.767682:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:0.0:1788880172.767688:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000a75a789f. 00000100:00000001:0.0:1788880172.767690:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249982991552 : -126823726560064 : ffff8ca78cd44cc0) 00000100:00000001:0.0:1788880172.767693:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880172.767694:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000100:00000001:0.0:1788880172.767699:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880172.767702:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.767704:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:0.0:1788880172.767705:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880172.767711:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.767718:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880172.767725:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.767729:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880172.767731:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880172.767734:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880172.767737:0:203392:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780633344:192.168.204.111@tcp:256: 00000100:00000001:0.0:1788880172.767742:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880172.767748:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880172.767750:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.767753:0:203392:0:(niobuf.c:346:ptlrpc_register_bulk()) Process entered 00000400:00000010:0.0:1788880172.767759:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000e3619b0a. 00000400:00000010:0.0:1788880172.767768:0:203392:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 4216 at 00000000a6467e0e. 00000100:00000200:0.0:1788880172.767774:0:203392:0:(niobuf.c:453:ptlrpc_register_bulk()) Setup 1 bulk put-sink buffers: 256 pages 1048576 bytes, mbits x0x6aa0250b00500-0x6aa0250b00500, portal 33 00000100:00000001:0.0:1788880172.767777:0:203392:0:(niobuf.c:455:ptlrpc_register_bulk()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.767780:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880172.767784:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000063883087. 02000000:00000001:0.0:1788880172.767786:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880172.767789:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000000396877. 00000400:00000010:0.0:1788880172.767793:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000200:0.0:1788880172.767796:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780633344, portal 25 00000100:00000001:0.0:1788880172.767798:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880172.767799:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000100:00000040:0.0:1788880172.767804:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.767811:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880172.767814:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880172.767817:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005c796512. 00000100:00000200:0.0:1788880172.767820:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 304 bytes to portal 26, xid 1875776780633344, offset 0 00000400:00000200:0.0:1788880172.767827:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880172.767839:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880172.767846:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880172.767851:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880172.767855:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880172.767859:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880172.767870:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [13:13:256:4294967295] 192.168.204.111@tcp LPNI seq info [13:13:8:4294967295] 00000400:00000200:0.0:1788880172.767881:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880172.767887:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 304 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880172.767893:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000800:00000200:0.0:1788880172.767900:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880172.767907:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880172.767912:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 400 niov 1 nkiov 1 00000100:00000001:0.0:1788880172.767925:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.767928:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305803876 00000100:00000001:0.0:1788880172.767935:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305803873 00000100:00000001:0.0:1788880172.767940:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305803876 00000100:00000001:0.0:1788880172.767943:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305803873 00000100:00000001:0.0:1788880172.767946:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305803876 00000100:00000001:0.0:1788880172.767949:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305803873 00000100:00000001:0.0:1788880172.767952:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305803876 00000100:00000001:0.0:1788880172.767955:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305803873 00000100:00000001:0.0:1788880172.767958:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880172.767961:0:203392:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.767964:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880172.767966:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:0.0:1788880172.767968:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 00000000a75a789f going to sleep for 11 seconds 00000100:00000001:0.0:1788880172.767972:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880172.767974:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880172.768070:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880172.768075:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f92e72a2. 00000400:00000200:0.0:1788880172.768081:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.768086:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880172.768090:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005c796512 00000400:00000010:0.0:1788880172.768093:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005c796512. 00000100:00000001:0.0:1788880172.768096:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880172.768105:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.768113:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.768117:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.768123:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.768125:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880172.769110:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.769128:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.769132:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.769135:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.769142:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880172.769156:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 132 into portal 33 MB=0x6aa0250b00500 00000400:00000200:0.0:1788880172.769163:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 21 from 12345-192.168.204.111@tcp of length 132/132 into md 0x161 [256] + 0 00000400:00000010:0.0:1788880172.769168:0:201279:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000e3619b0a. 00000400:00000200:0.0:1788880172.769172:0:201279:0:(lib-md.c:42:lnet_md_unlink()) Queueing unlink of md 00000000a6467e0e 00000800:00000001:0.0:1788880172.769177:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.769185:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.769187:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.769191:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880172.769195:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a6467e0e 00000400:00000010:0.0:1788880172.769197:0:201279:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 4216 at 00000000a6467e0e. 00000100:00000001:0.0:1788880172.769201:0:201279:0:(events.c:174:client_bulk_callback()) Process entered 00000100:00000200:0.0:1788880172.769203:0:201279:0:(events.c:192:client_bulk_callback()) event type 2, status 0, req 00000000ff0be673 desc 000000006ece4223 mbits 1875776780633344 00000100:00000001:0.0:1788880172.769213:0:201279:0:(events.c:217:client_bulk_callback()) Process leaving 00000400:00000200:0.0:1788880172.769218:0:201279:0:(lib-md.c:96:lnet_cpt_of_md()) Couldn't resolve first page of md 0000000000000000 with offset 0 00000400:00000200:0.0:1788880172.769224:0:201279:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880172.769229:0:201279:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [14:14:256:4294967295] 192.168.204.111@tcp LPNI seq info [14:14:8:4294967295] 00000400:00000200:0.0:1788880172.769265:0:201279:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) 192.168.204.111@tcp : ACK try# 0 00000800:00000200:0.0:1788880172.769273:0:201279:0:(socklnd_cb.c:931:ksocknal_send()) sending 0 bytes in 256 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880172.769278:0:201279:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 4328 at 00000000a6467e0e. 00000800:00000200:0.0:1788880172.769291:0:201279:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880172.769297:0:201279:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880172.769301:0:201279:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000a6467e0e type 0, nob 96 niov 1 nkiov 0 00000800:00000200:0.0:1788880172.769359:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880172.769362:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 4328 at 00000000a6467e0e. 00000400:00000200:0.0:1788880172.769366:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.769371:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: ACK: OK 00000800:00000001:0.0:1788880172.769379:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.769383:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.769400:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880172.769404:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880172.769991:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880172.769997:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880172.769999:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880172.770003:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880172.770010:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880172.770019:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 240 into portal 25 MB=0x6aa0250b00500 00000400:00000200:3.0:1788880172.770025:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 240/240 into md 0x165 [1] + 224 00000800:00000001:3.0:1788880172.770043:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880172.770050:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880172.770052:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880172.770057:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880172.770061:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880172.770067:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880172.770078:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=240 offset=224 replen=240 req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.770095:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880172.770102:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880172.770107:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880172.770110:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788880172.770112:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880172.770115:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880172.770118:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000000396877. 00000400:00000200:0.0:1788880172.770123:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:0.0:1788880172.770125:0:203392:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:0.0:1788880172.770129:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.770135:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880172.770145:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.770152:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880172.770155:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.770157:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880172.770160:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880172.770163:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880172.770165:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.770168:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880172.770174:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=0 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880172.770179:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=0 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880172.770182:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880172.770184:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.770187:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880172.770190:0:203392:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.770199:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880172.770204:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Bulk req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880172.770214:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Bulk to Interpret req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 2 fl Bulk:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.770221:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880172.770224:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880172.770227:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880172.770258:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780633344:192.168.204.111@tcp:256: 00000100:00000001:0.0:1788880172.770265:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.770269:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.770271:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:0.0:1788880172.770274:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.770278:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.770284:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880172.770287:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000a75a789f. 00000100:00000001:0.0:1788880172.770290:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:0.0:1788880172.770292:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:0.0:1788880172.770297:0:203392:0:(mgc_request.c:1599:mgc_process_recover_log()) Latest version 11, more 0. 10000000:00000001:0.0:1788880172.770301:0:203392:0:(mgc_request.c:1308:mgc_apply_recover_logs()) Process entered 10000000:00000010:0.0:1788880172.770308:0:203392:0:(mgc_request.c:1331:mgc_apply_recover_logs()) kmalloced '(nidlist)': 640 at 0000000018c57bb0. 10000000:00000010:0.0:1788880172.770312:0:203392:0:(mgc_request.c:1343:mgc_apply_recover_logs()) kmalloced '(obdname)': 128 at 000000001c376fa8. 10000000:00000200:0.0:1788880172.770318:0:203392:0:(mgc_request.c:1241:mgc_build_nidlist()) use 1 from 1 NIDs 10000000:00000040:0.0:1788880172.770321:0:203392:0:(mgc_request.c:1440:mgc_apply_recover_logs()) Looking for OBD lustre-MDT0000-mdc-ffff8ca787527000 00010000:00000001:0.0:1788880172.770329:0:203392:0:(ldlm_lib.c:159:client_import_add_nids_to_conn()) Process entered 00000100:00000001:0.0:1788880172.770333:0:203392:0:(lustre_peer.c:229:class_check_uuid()) Process entered 00000100:00000040:0.0:1788880172.770336:0:203392:0:(lustre_peer.c:232:class_check_uuid()) check if uuid 192.168.204.111@tcp has 192.168.204.111@tcp. 00000100:00000001:0.0:1788880172.770339:0:203392:0:(lustre_peer.c:251:class_check_uuid()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.770341:0:203392:0:(lustre_peer.c:189:class_add_nids_to_uuid()) Process entered 00000100:00000200:0.0:1788880172.770349:0:203392:0:(lustre_peer.c:194:class_add_nids_to_uuid()) Comparing 192.168.204.111@tcp with 192.168.204.111@tcp 00000100:00000200:0.0:1788880172.770351:0:203392:0:(lustre_peer.c:201:class_add_nids_to_uuid()) Updating UUID '192.168.204.111@tcp' 00000400:00000200:0.0:1788880172.770355:0:203392:0:(peer.c:1363:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:0.0:1788880172.770358:0:203392:0:(api-ni.c:3274:LNetNIInit()) refs 1 00000400:00000200:0.0:1788880172.770362:0:203392:0:(peer.c:1377:LNetAddPeer()) Adding NID[0]: 192.168.204.111@tcp 00000400:00000200:0.0:1788880172.770368:0:203392:0:(peer.c:1822:lnet_peer_add()) peer 192.168.204.111@tcp NID flags 0x100001: -17 00000400:00000200:0.0:1788880172.770378:0:203392:0:(peer.c:2890:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:0.0:1788880172.770382:0:203392:0:(peer.c:2899:lnet_discover_peer_locked()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp: 0. pending discovery 00000400:00000200:0.0:1788880172.770385:0:203392:0:(peer.c:1431:LNetAddPeer()) A peer exists for NID 192.168.204.111@tcp 00000100:00000001:0.0:1788880172.770388:0:203392:0:(lustre_peer.c:219:class_add_nids_to_uuid()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.770390:0:203392:0:(ldlm_lib.c:174:client_import_add_nids_to_conn()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:0.0:1788880172.770393:0:203392:0:(mgc_request.c:1486:mgc_apply_recover_logs()) ir apply logs 9/11 for lustre-MDT0000-mdc-ffff8ca787527000 10000000:00000200:0.0:1788880172.770396:0:203392:0:(mgc_request.c:1241:mgc_build_nidlist()) use 1 from 1 NIDs 10000000:00000040:0.0:1788880172.770398:0:203392:0:(mgc_request.c:1440:mgc_apply_recover_logs()) Looking for OBD lustre-MDT0001-mdc-ffff8ca787527000 00010000:00000001:0.0:1788880172.770402:0:203392:0:(ldlm_lib.c:159:client_import_add_nids_to_conn()) Process entered 00000100:00000001:0.0:1788880172.770404:0:203392:0:(lustre_peer.c:229:class_check_uuid()) Process entered 00000100:00000040:0.0:1788880172.770407:0:203392:0:(lustre_peer.c:232:class_check_uuid()) check if uuid 192.168.204.111@tcp has 192.168.204.111@tcp. 00000100:00000001:0.0:1788880172.770409:0:203392:0:(lustre_peer.c:251:class_check_uuid()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.770411:0:203392:0:(lustre_peer.c:189:class_add_nids_to_uuid()) Process entered 00000100:00000200:0.0:1788880172.770413:0:203392:0:(lustre_peer.c:194:class_add_nids_to_uuid()) Comparing 192.168.204.111@tcp with 192.168.204.111@tcp 00000100:00000200:0.0:1788880172.770415:0:203392:0:(lustre_peer.c:201:class_add_nids_to_uuid()) Updating UUID '192.168.204.111@tcp' 00000400:00000200:0.0:1788880172.770417:0:203392:0:(peer.c:1363:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:0.0:1788880172.770419:0:203392:0:(api-ni.c:3274:LNetNIInit()) refs 1 00000400:00000200:0.0:1788880172.770421:0:203392:0:(peer.c:1377:LNetAddPeer()) Adding NID[0]: 192.168.204.111@tcp 00000400:00000200:0.0:1788880172.770425:0:203392:0:(peer.c:1822:lnet_peer_add()) peer 192.168.204.111@tcp NID flags 0x100001: -17 00000400:00000200:0.0:1788880172.770430:0:203392:0:(peer.c:2890:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:0.0:1788880172.770434:0:203392:0:(peer.c:2899:lnet_discover_peer_locked()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp: 0. pending discovery 00000400:00000200:0.0:1788880172.770436:0:203392:0:(peer.c:1431:LNetAddPeer()) A peer exists for NID 192.168.204.111@tcp 00000100:00000001:0.0:1788880172.770438:0:203392:0:(lustre_peer.c:219:class_add_nids_to_uuid()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.770440:0:203392:0:(ldlm_lib.c:174:client_import_add_nids_to_conn()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:0.0:1788880172.770443:0:203392:0:(mgc_request.c:1486:mgc_apply_recover_logs()) ir apply logs 10/11 for lustre-MDT0001-mdc-ffff8ca787527000 10000000:00000200:0.0:1788880172.770445:0:203392:0:(mgc_request.c:1241:mgc_build_nidlist()) use 1 from 1 NIDs 10000000:00000040:0.0:1788880172.770448:0:203392:0:(mgc_request.c:1440:mgc_apply_recover_logs()) Looking for OBD lustre-OST0000-osc-ffff8ca787527000 00010000:00000001:0.0:1788880172.770452:0:203392:0:(ldlm_lib.c:159:client_import_add_nids_to_conn()) Process entered 00000100:00000001:0.0:1788880172.770454:0:203392:0:(lustre_peer.c:229:class_check_uuid()) Process entered 00000100:00000040:0.0:1788880172.770463:0:203392:0:(lustre_peer.c:232:class_check_uuid()) check if uuid 192.168.204.111@tcp has 192.168.204.111@tcp. 00000100:00000001:0.0:1788880172.770466:0:203392:0:(lustre_peer.c:251:class_check_uuid()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.770468:0:203392:0:(lustre_peer.c:189:class_add_nids_to_uuid()) Process entered 00000100:00000200:0.0:1788880172.770470:0:203392:0:(lustre_peer.c:194:class_add_nids_to_uuid()) Comparing 192.168.204.111@tcp with 192.168.204.111@tcp 00000100:00000200:0.0:1788880172.770472:0:203392:0:(lustre_peer.c:201:class_add_nids_to_uuid()) Updating UUID '192.168.204.111@tcp' 00000400:00000200:0.0:1788880172.770474:0:203392:0:(peer.c:1363:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:0.0:1788880172.770476:0:203392:0:(api-ni.c:3274:LNetNIInit()) refs 1 00000400:00000200:0.0:1788880172.770479:0:203392:0:(peer.c:1377:LNetAddPeer()) Adding NID[0]: 192.168.204.111@tcp 00000400:00000200:0.0:1788880172.770483:0:203392:0:(peer.c:1822:lnet_peer_add()) peer 192.168.204.111@tcp NID flags 0x100001: -17 00000400:00000200:0.0:1788880172.770489:0:203392:0:(peer.c:2890:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:0.0:1788880172.770493:0:203392:0:(peer.c:2899:lnet_discover_peer_locked()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp: 0. pending discovery 00000400:00000200:0.0:1788880172.770496:0:203392:0:(peer.c:1431:LNetAddPeer()) A peer exists for NID 192.168.204.111@tcp 00000100:00000001:0.0:1788880172.770498:0:203392:0:(lustre_peer.c:219:class_add_nids_to_uuid()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.770500:0:203392:0:(ldlm_lib.c:174:client_import_add_nids_to_conn()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:0.0:1788880172.770503:0:203392:0:(mgc_request.c:1486:mgc_apply_recover_logs()) ir apply logs 11/11 for lustre-OST0000-osc-ffff8ca787527000 10000000:00000010:0.0:1788880172.770506:0:203392:0:(mgc_request.c:1491:mgc_apply_recover_logs()) kfreed 'obdname': 128 at 000000001c376fa8. 10000000:00000010:0.0:1788880172.770511:0:203392:0:(mgc_request.c:1493:mgc_apply_recover_logs()) kfreed 'nidlist': 640 at 0000000018c57bb0. 10000000:00000001:0.0:1788880172.770514:0:203392:0:(mgc_request.c:1495:mgc_apply_recover_logs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.770517:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.770521:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ff0be673 x1875776780633344/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880188 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.770528:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880172.770531:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880172.770533:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000063883087. 02000000:00000001:0.0:1788880172.770538:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880172.770540:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880172.770542:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880172.770546:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880172.770547:0:203392:0:(client.c:312:ptlrpc_free_bulk()) Process entered 00000020:00000001:0.0:1788880172.770550:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880172.770551:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880172.770553:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000010:0.0:1788880172.770574:0:203392:0:(client.c:333:ptlrpc_free_bulk()) kvfree 'desc->bd_vec': 4096 at 000000004244f03d. 00000100:00000010:0.0:1788880172.770580:0:203392:0:(client.c:334:ptlrpc_free_bulk()) kfreed 'desc': 984 at 000000006ece4223. 00000100:00000001:0.0:1788880172.770583:0:203392:0:(client.c:335:ptlrpc_free_bulk()) Process leaving 02000000:00000010:0.0:1788880172.770592:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000d653e4f6. 02000000:00000001:0.0:1788880172.770596:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880172.770599:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880172.770601:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ff0be673. 00000100:00000001:0.0:1788880172.770606:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880172.770607:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880172.771325:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305803876, at 4305803877 00000100:00000010:3.0:1788880172.771336:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000594debb1. 00000100:00000040:2.0:1788880172.771494:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305803876, at 4305803877 10000000:00000010:0.0:1788880172.771495:0:203392:0:(mgc_request.c:1651:mgc_process_recover_log()) kvfree 'folios': 2048 at 000000006aef07b8. 00000100:00000010:2.0:1788880172.771501:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005869c8be. 10000000:01000000:0.0:1788880172.771503:0:203392:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'lustre-cliir' succeeded (0). 10000000:00000001:0.0:1788880172.771517:0:203392:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:0.0:1788880172.771520:0:203392:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880172.771522:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880172.771526:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000001ffdd456 refcount=3 00000020:00000001:0.0:1788880172.771529:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000001:0.0:1788880172.771532:0:203392:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000001:0.0:1788880172.771535:0:203392:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:0.0:1788880172.771540:0:203392:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880172.771550:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880172.771552:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:0.0:1788880172.771554:0:203392:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880172.771561:0:203392:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:0.0:1788880172.771563:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880172.771564:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:0.0:1788880172.771566:0:203392:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788880172.771569:0:203392:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788880172.771572:0:203392:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:0.0:1788880172.771575:0:203392:0:(mgc_request.c:1801:mgc_process_log()) Process log params-ffff8ca787527000 from 1 10000000:00000001:0.0:1788880172.771578:0:203392:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:0.0:1788880172.771580:0:203392:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00000010:0.0:1788880172.771585:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ff0be673. 00000020:00000040:0.0:1788880172.771589:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880172.771594:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880172.771603:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880172.771607:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880172.771613:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000d653e4f6. 00000100:00000001:0.0:1788880172.771620:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.771625:0:203392:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:0.0:1788880172.771628:0:203392:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:0.0:1788880172.771636:0:203392:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at 000000002cbe30a6. 00010000:00000001:0.0:1788880172.771641:0:203392:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:0.0:1788880172.771645:0:203392:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 00000000f14805ea. 00000020:00000010:0.0:1788880172.771650:0:203392:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 128 at 000000001c376fa8. 00000020:00000001:0.0:1788880172.771654:0:203392:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:0.0:1788880172.771657:0:203392:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000f14805ea with handle 0x805d548a36a88690 to hash 00000020:00000001:0.0:1788880172.771659:0:203392:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:0.0:1788880172.771660:0:203392:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617250084075008 : -126823625476608 : ffff8ca792dab600) 00010000:00000001:0.0:1788880172.771664:0:203392:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617250084075008 : -126823625476608 : ffff8ca792dab600) 00010000:00000001:0.0:1788880172.771668:0:203392:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:0.0:1788880172.771672:0:203392:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880172.771675:0:203392:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 00000000f14805ea/0x805d548a36a88690 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788880172.771682:0:203392:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 00000000f14805ea/0x805d548a36a88690 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788880172.771692:0:203392:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 00000000f14805ea/0x805d548a36a88690 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:0.0:1788880172.771699:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:0.0:1788880172.771702:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:0.0:1788880172.771708:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000a75a789f. 00000100:00000001:0.0:1788880172.771710:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249982991552 : -126823726560064 : ffff8ca78cd44cc0) 00000100:00000001:0.0:1788880172.771713:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880172.771715:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000100:00000001:0.0:1788880172.771721:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880172.771724:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.771733:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:0.0:1788880172.771735:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880172.771741:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.771750:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880172.771752:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.771755:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880172.771758:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880172.771761:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880172.771765:0:203392:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780633472:192.168.204.111@tcp:101: 00000100:00000001:0.0:1788880172.771770:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880172.771776:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880172.771779:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.771781:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880172.771785:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000006ece4223. 02000000:00000001:0.0:1788880172.771787:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880172.771792:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000000396877. 00000400:00000010:0.0:1788880172.771797:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000200:0.0:1788880172.771803:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780633472, portal 25 00000100:00000001:0.0:1788880172.771806:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880172.771808:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000100:00000040:0.0:1788880172.771813:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.771821:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880172.771826:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880172.771829:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005c796512. 00000100:00000200:0.0:1788880172.771832:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780633472, offset 0 00000400:00000200:0.0:1788880172.771840:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880172.771850:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880172.771856:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880172.771861:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880172.771865:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880172.771868:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880172.771877:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [15:15:256:4294967295] 192.168.204.111@tcp LPNI seq info [15:15:8:4294967295] 00000400:00000200:0.0:1788880172.771889:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880172.771895:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880172.771900:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000800:00000200:0.0:1788880172.771907:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880172.771915:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880172.771919:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:0.0:1788880172.771930:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.771934:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305803880 00000100:00000001:0.0:1788880172.771939:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305803876 00000100:00000001:0.0:1788880172.771943:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305803880 00000100:00000001:0.0:1788880172.771947:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305803876 00000100:00000001:0.0:1788880172.771950:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305803880 00000100:00000010:0.0:1788880172.771953:0:203392:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008921ad2e. 00000100:00000001:0.0:1788880172.771958:0:203392:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880172.771962:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305803880 00000100:00000010:0.0:1788880172.771965:0:203392:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001a3d5793. 00000100:00000001:0.0:1788880172.771968:0:203392:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880172.771971:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880172.771975:0:203392:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.771978:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880172.771980:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:0.0:1788880172.771983:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 00000000a75a789f going to sleep for 11 seconds 00000100:00000001:0.0:1788880172.771986:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880172.771988:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880172.772063:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880172.772066:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f92e72a2. 00000400:00000200:0.0:1788880172.772071:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.772075:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880172.772079:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005c796512 00000400:00000010:0.0:1788880172.772087:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005c796512. 00000100:00000001:0.0:1788880172.772091:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880172.772095:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.772104:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.772107:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.772114:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.772115:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880172.773251:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880172.773263:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880172.773267:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880172.773270:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880172.773278:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880172.773291:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b00580 00000400:00000200:2.0:1788880172.773298:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0x16d [1] + 224 00000800:00000001:2.0:1788880172.773303:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880172.773310:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880172.773313:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880172.773316:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880172.773320:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880172.773326:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880172.773336:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.773352:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880172.773359:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880172.773362:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.773378:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880172.773393:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880172.773395:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880172.773397:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000000396877. 00000400:00000200:0.0:1788880172.773401:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:0.0:1788880172.773403:0:203392:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:0.0:1788880172.773405:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.773409:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880172.773417:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.773424:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880172.773425:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.773428:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880172.773430:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880172.773432:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880172.773434:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.773437:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880172.773443:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=0 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880172.773447:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=0 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880172.773449:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880172.773451:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.773453:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880172.773455:0:203392:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.773458:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880172.773462:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.773477:0:203392:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880172.773489:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880172.773491:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880172.773495:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880172.773502:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780633472:192.168.204.111@tcp:101: 00000100:00000001:0.0:1788880172.773507:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880172.773509:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.773511:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:0.0:1788880172.773513:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.773515:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.773521:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880172.773524:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000a75a789f. 00000100:00000001:0.0:1788880172.773527:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:0.0:1788880172.773528:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.773531:0:203392:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:0.0:1788880172.773533:0:203392:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880172.773535:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880172.773538:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000f14805ea refcount=4 00000020:00000001:0.0:1788880172.773540:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250084075008 : -126823625476608 : ffff8ca792dab600) 00010000:00000001:0.0:1788880172.773543:0:203392:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250084075008 : -126823625476608 : ffff8ca792dab600) 00010000:00000040:0.0:1788880172.773548:0:203392:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 00000000f14805ea, remote cookie: 0x92f11317f60a7917, flags: 0x0 00010000:00000001:0.0:1788880172.773552:0:203392:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:0.0:1788880172.773556:0:203392:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:0.0:1788880172.773559:0:203392:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:0.0:1788880172.773560:0:203392:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:0.0:1788880172.773562:0:203392:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:0.0:1788880172.773564:0:203392:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x736d61726170:0x3:0x0].0x0 (000000002cbe30a6) refcount = 1 00010000:00010000:0.0:1788880172.773568:0:203392:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 00000000f14805ea/0x805d548a36a88690 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7917 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880172.773585:0:203392:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:0.0:1788880172.773587:0:203392:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:0.0:1788880172.773588:0:203392:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:0.0:1788880172.773591:0:203392:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:0.0:1788880172.773593:0:203392:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880172.773595:0:203392:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 00000000f14805ea/0x805d548a36a88690 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7917 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880172.773601:0:203392:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:0.0:1788880172.773602:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880172.773604:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880172.773605:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880172.773606:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880172.773607:0:203392:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.773618:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.773622:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ff0be673 x1875776780633472/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880188 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.773628:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880172.773629:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880172.773631:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000006ece4223. 02000000:00000001:0.0:1788880172.773635:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880172.773636:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880172.773638:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880172.773640:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880172.773642:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000d653e4f6. 02000000:00000001:0.0:1788880172.773644:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880172.773645:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880172.773647:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ff0be673. 00000100:00000001:0.0:1788880172.773649:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880172.773650:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:0.0:1788880172.773652:0:203392:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788880172.773655:0:203392:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:0.0:1788880172.773656:0:203392:0:(mgc_request.c:108:config_log_get()) log params (00000000dbabba98) refs 2 10000000:00000001:0.0:1788880172.773659:0:203392:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880172.773661:0:203392:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880172.773666:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880172.773668:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000f14805ea refcount=3 00000020:00000001:0.0:1788880172.773669:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250084075008 : -126823625476608 : ffff8ca792dab600) 00010000:00000001:0.0:1788880172.773671:0:203392:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250084075008 : -126823625476608 : ffff8ca792dab600) 00010000:00000001:0.0:1788880172.773673:0:203392:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:0.0:1788880172.773675:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880172.773676:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880172.773677:0:203392:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788880172.773679:0:203392:0:(mgc_request.c:1668:mgc_process_cfg_log()) Process entered 10000000:00000010:0.0:1788880172.773683:0:203392:0:(mgc_request.c:1682:mgc_process_cfg_log()) kmalloced '(env)': 56 at 00000000a9f9e43c. 00000020:00000010:0.0:1788880172.773686:0:203392:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000d653e4f6. 00000020:00000010:0.0:1788880172.773692:0:203392:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 0000000063883087. 00000040:00000010:0.0:1788880172.773695:0:203392:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000605d7309. 00000020:00000010:0.0:1788880172.773700:0:203392:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 000000006aef07b8. 00000020:00000010:0.0:1788880172.773705:0:203392:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 0000000081a10824. 80000000:00000010:0.0:1788880172.773709:0:203392:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 000000006050354c. 00040000:00000010:0.0:1788880172.773714:0:203392:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000358be29b. 20000000:00000010:0.0:1788880172.773719:0:203392:0:(mgs_handler.c:1787:mgs_key_init()) kmalloced '(value)': 392 at 000000008f22742c. 10000000:00000040:0.0:1788880172.773724:0:203392:0:(lustre_log.h:387:llog_ctxt_get()) GETting ctxt 000000006d5d0799 : new refcount 2 00000020:00000001:0.0:1788880172.773727:0:203392:0:(obd_config.c:2120:class_config_parse_llog()) Process entered 00000020:00000040:0.0:1788880172.773728:0:203392:0:(obd_config.c:2122:class_config_parse_llog()) looking up llog params 00000040:00000001:0.0:1788880172.773731:0:203392:0:(llog.c:1538:llog_open()) Process entered 00000040:00000010:0.0:1788880172.773735:0:203392:0:(llog.c:42:llog_alloc_handle()) kmalloced '(loghandle)': 352 at 00000000b405d9c4. 00000040:00000001:0.0:1788880172.773739:0:203392:0:(llog_client.c:70:llog_client_open()) Process entered 00000020:00000040:0.0:1788880172.773741:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:0.0:1788880172.773745:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ff0be673. 00000020:00000040:0.0:1788880172.773747:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880172.773750:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880172.773753:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880172.773756:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880172.773760:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000b52810a8. 00000100:00000001:0.0:1788880172.773763:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.773768:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:0.0:1788880172.773770:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:0.0:1788880172.773773:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000a75a789f. 00000100:00000001:0.0:1788880172.773780:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249982991552 : -126823726560064 : ffff8ca78cd44cc0) 00000100:00000001:0.0:1788880172.773782:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880172.773784:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000100:00000001:0.0:1788880172.773787:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880172.773789:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.773790:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:0.0:1788880172.773791:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880172.773795:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.773802:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880172.773803:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.773806:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880172.773807:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880172.773810:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880172.773812:0:203392:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780633600:192.168.204.111@tcp:501: 00000100:00000001:0.0:1788880172.773816:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880172.773820:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880172.773822:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880172.773824:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880172.773828:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007c59c7b9. 02000000:00000001:0.0:1788880172.773829:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880172.773832:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000000396877. 00000400:00000010:0.0:1788880172.773836:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000200:0.0:1788880172.773841:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780633600, portal 25 00000100:00000001:0.0:1788880172.773842:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880172.773843:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000100:00000040:0.0:1788880172.773847:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.773853:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880172.773856:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880172.773857:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005c796512. 00000100:00000200:0.0:1788880172.773859:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 504 bytes to portal 26, xid 1875776780633600, offset 0 00000400:00000200:0.0:1788880172.773864:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880172.773875:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880172.773880:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880172.773884:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880172.773886:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880172.773888:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880172.773892:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [16:16:256:4294967295] 192.168.204.111@tcp LPNI seq info [16:16:8:4294967295] 00000400:00000200:0.0:1788880172.773900:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880172.773906:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 504 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880172.773909:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000800:00000200:0.0:1788880172.773914:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880172.773920:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880172.773922:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 600 niov 1 nkiov 1 00000100:00000001:0.0:1788880172.773935:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.773939:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305803882 00000100:00000001:0.0:1788880172.773943:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305803880 00000100:00000001:0.0:1788880172.773947:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305803882 00000100:00000001:0.0:1788880172.773950:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305803880 00000100:00000001:0.0:1788880172.773952:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305803882 00000100:00000001:0.0:1788880172.773955:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305803880 00000100:00000001:0.0:1788880172.773958:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305803882 00000100:00000001:0.0:1788880172.773961:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305803880 00000100:00000001:0.0:1788880172.773963:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880172.773966:0:203392:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.773969:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880172.773971:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:0.0:1788880172.773973:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 00000000a75a789f going to sleep for 11 seconds 00000100:00000001:0.0:1788880172.773976:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880172.773978:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880172.773996:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880172.774000:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f92e72a2. 00000400:00000200:3.0:1788880172.774018:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880172.774022:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880172.774025:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005c796512 00000400:00000010:3.0:1788880172.774039:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005c796512. 00000100:00000001:3.0:1788880172.774043:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880172.774049:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.774057:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880172.774059:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 1788880188 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.774065:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.774066:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880172.774990:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.775004:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.775006:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.775008:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.775012:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880172.775019:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 272 into portal 25 MB=0x6aa0250b00600 00000400:00000200:0.0:1788880172.775023:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 272/272 into md 0x175 [1] + 224 00000800:00000001:0.0:1788880172.775026:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.775042:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.775043:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.775046:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880172.775048:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.775051:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880172.775058:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=272 offset=224 replen=272 req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.775070:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880172.775074:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.775076:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.775094:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880172.775099:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880172.775101:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880172.775103:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000000396877. 00000400:00000200:2.0:1788880172.775106:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:2.0:1788880172.775108:0:203392:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:2.0:1788880172.775111:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880172.775115:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880172.775135:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.775140:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880172.775141:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.775143:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880172.775144:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880172.775146:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880172.775147:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880172.775149:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880172.775151:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=0 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880172.775153:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=0 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880172.775155:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880172.775156:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880172.775157:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880172.775158:0:203392:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.775160:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880172.775161:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.775165:0:203392:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880172.775167:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880172.775167:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880172.775169:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 1788880188 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880172.775173:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780633600:192.168.204.111@tcp:501: 00000100:00000001:2.0:1788880172.775176:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.775177:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.775178:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:2.0:1788880172.775179:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880172.775181:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 1788880188 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.775185:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880172.775189:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000a75a789f. 00000100:00000001:2.0:1788880172.775191:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:2.0:1788880172.775192:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:2.0:1788880172.775196:0:203392:0:(llog_client.c:122:llog_client_open()) Process leaving 00000020:00000001:2.0:1788880172.775198:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880172.775199:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880172.775201:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880172.775202:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880172.775204:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ff0be673 x1875776780633600/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 504/272 e 0 to 0 dl 1788880188 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.775210:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880172.775212:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880172.775214:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007c59c7b9. 02000000:00000001:2.0:1788880172.775216:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880172.775218:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880172.775219:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880172.775220:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880172.775222:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000b52810a8. 02000000:00000001:2.0:1788880172.775225:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880172.775226:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880172.775227:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ff0be673. 00000100:00000001:2.0:1788880172.775231:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880172.775261:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000040:00000001:2.0:1788880172.775264:0:203392:0:(llog.c:1563:llog_open()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:2.0:1788880172.775267:0:203392:0:(llog.c:415:llog_init_handle()) Process entered 00000040:00000010:2.0:1788880172.775274:0:203392:0:(llog.c:420:llog_init_handle()) kmalloced '(llh)': 8192 at 00000000c68794ed. 00000040:00000001:2.0:1788880172.775284:0:203392:0:(llog.c:363:llog_read_header()) Process entered 00000040:00000001:2.0:1788880172.775286:0:203392:0:(llog_client.c:269:llog_client_read_header()) Process entered 00000020:00000040:2.0:1788880172.775291:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:2.0:1788880172.775296:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000020:00000040:2.0:1788880172.775299:0:203392:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880172.775301:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880172.775303:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880172.775305:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880172.775308:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000008f0935d3. 00000100:00000001:2.0:1788880172.775311:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.775313:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:2.0:1788880172.775320:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:2.0:1788880172.775323:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000000211877. 00000100:00000001:2.0:1788880172.775326:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249931840320 : -126823777711296 : ffff8ca789c7cb40) 00000100:00000001:2.0:1788880172.775328:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880172.775330:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000100:00000001:2.0:1788880172.775333:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880172.775335:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.775337:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:2.0:1788880172.775338:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880172.775342:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.775349:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880172.775350:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880172.775353:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880172.775355:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880172.775357:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880172.775360:0:203392:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780633728:192.168.204.111@tcp:503: 00000100:00000001:2.0:1788880172.775363:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880172.775366:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880172.775368:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880172.775370:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880172.775385:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 16384 at 000000005f7a266f. 02000000:00000001:2.0:1788880172.775387:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880172.775390:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000400:00000010:2.0:1788880172.775393:0:203392:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 0000000055a084bf. 00000100:00000200:2.0:1788880172.775396:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 16384 bytes, xid 1875776780633728, portal 25 00000100:00000001:2.0:1788880172.775397:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880172.775398:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000100:00000040:2.0:1788880172.775402:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.775409:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880172.775412:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880172.775415:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f779aa21. 00000100:00000200:2.0:1788880172.775418:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 272 bytes to portal 26, xid 1875776780633728, offset 0 00000400:00000200:2.0:1788880172.775429:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880172.775436:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880172.775441:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880172.775444:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880172.775447:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880172.775450:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880172.775454:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [17:17:256:4294967295] 192.168.204.111@tcp LPNI seq info [17:17:8:4294967295] 00000400:00000200:2.0:1788880172.775471:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880172.775474:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 272 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880172.775477:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000036dc6dd9. 00000800:00000200:2.0:1788880172.775480:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880172.775484:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880172.775485:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000036dc6dd9 type 1, nob 368 niov 1 nkiov 1 00000100:00000001:2.0:1788880172.775495:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.775498:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305803884 00000100:00000001:2.0:1788880172.775502:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305803882 00000100:00000001:2.0:1788880172.775505:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305803884 00000100:00000001:2.0:1788880172.775508:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305803882 00000100:00000001:2.0:1788880172.775510:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305803884 00000100:00000001:2.0:1788880172.775512:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305803882 00000100:00000001:2.0:1788880172.775514:0:203392:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305803884 00000100:00000001:2.0:1788880172.775517:0:203392:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305803882 00000100:00000001:2.0:1788880172.775519:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880172.775521:0:203392:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.775523:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880172.775524:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:2.0:1788880172.775526:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 0000000000211877 going to sleep for 11 seconds 00000100:00000001:2.0:1788880172.775529:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880172.775530:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880172.775563:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880172.775566:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000036dc6dd9. 00000400:00000200:0.0:1788880172.775570:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.775572:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880172.775574:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f779aa21 00000400:00000010:0.0:1788880172.775576:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f779aa21. 00000100:00000001:0.0:1788880172.775578:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880172.775580:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.775586:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.775587:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.775591:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.775592:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880172.776523:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880172.776537:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880172.776544:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880172.776546:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880172.776552:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880172.776605:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 8416 into portal 25 MB=0x6aa0250b00680 00000400:00000200:3.0:1788880172.776609:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 8416/8416 into md 0x17d [4] + 224 00000800:00000001:3.0:1788880172.776613:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880172.776634:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880172.776636:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880172.776638:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880172.776641:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880172.776645:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880172.776652:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=8416 offset=224 replen=8416 req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.776670:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880172.776677:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880172.776680:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.776691:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880172.776697:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880172.776699:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880172.776700:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:2.0:1788880172.776704:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000055a084bf 00000400:00000010:2.0:1788880172.776706:0:203392:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 0000000055a084bf. 00000100:00000001:2.0:1788880172.776709:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880172.776714:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880172.776722:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.776728:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880172.776730:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.776732:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880172.776733:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880172.776735:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880172.776737:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880172.776738:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880172.776741:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=0 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880172.776744:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=0 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880172.776745:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880172.776746:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880172.776747:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880172.776749:0:203392:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.776750:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880172.776752:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.776755:0:203392:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880172.776757:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880172.776758:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880172.776760:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880172.776765:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job mount.lustre:fa4856d5-51a8-499e-b3f8-b477743a3dec:203392:1875776780633728:192.168.204.111@tcp:503: 00000100:00000001:2.0:1788880172.776774:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.776775:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.776777:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:2.0:1788880172.776778:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880172.776780:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.776784:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880172.776785:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000000211877. 00000100:00000001:2.0:1788880172.776786:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:2.0:1788880172.776787:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:2.0:1788880172.776791:0:203392:0:(llog_client.c:319:llog_client_read_header()) Process leaving 00000100:00000001:2.0:1788880172.776791:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880172.776793:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000fd6b7ee1 x1875776780633728/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880188 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.776797:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880172.776798:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880172.776799:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 16384 at 000000005f7a266f. 02000000:00000001:2.0:1788880172.776804:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880172.776806:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880172.776807:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880172.776808:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880172.776810:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000008f0935d3. 02000000:00000001:2.0:1788880172.776812:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880172.776813:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880172.776815:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000fd6b7ee1. 00000100:00000001:2.0:1788880172.776818:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880172.776819:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788880172.776821:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880172.776822:0:203392:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880172.776823:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 00000040:00000001:2.0:1788880172.776825:0:203392:0:(llog.c:403:llog_read_header()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:2.0:1788880172.776827:0:203392:0:(llog.c:481:llog_init_handle()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:2.0:1788880172.776831:0:203392:0:(llog.c:951:llog_process_or_fork()) Process entered 00000040:00000010:2.0:1788880172.776833:0:203392:0:(llog.c:953:llog_process_or_fork()) kmalloced '(lpi)': 96 at 000000000171f5bf. 00000040:00001000:2.0:1788880172.776835:0:203392:0:(llog.c:968:llog_process_or_fork()) Processing [0xa:0x2:0x0] flags 0x004 startcat -1 startidx -1 first_idx 0 last_idx 0 read_mode 0 00000040:00000001:0.0:1788880172.777057:0:203410:0:(llog.c:552:llog_process_thread()) Process entered 00000020:00000010:0.0:1788880172.777064:0:203410:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000b5c40084. 00000020:00000010:0.0:1788880172.777069:0:203410:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000e0fe5497. 00000040:00000010:0.0:1788880172.777073:0:203410:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000a4a3549f. 00000020:00000010:0.0:1788880172.777077:0:203410:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000964e9d35. 00000020:00000010:0.0:1788880172.777083:0:203410:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 00000000cb5db106. 00000020:00000010:0.0:1788880172.777089:0:203410:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000fdf771c7. 80000000:00000010:0.0:1788880172.777093:0:203410:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 000000005b8bebe2. 40000000:00000010:0.0:1788880172.777097:0:203410:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 00000000e94ffc2c. 00100000:00000010:0.0:1788880172.777104:0:203410:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 00000000ffa4505d. 00040000:00000010:0.0:1788880172.777109:0:203410:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 00000000b899fe8d. 00040000:00000010:0.0:1788880172.777114:0:203410:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 00000000842c011f. 00040000:00000010:0.0:1788880172.777119:0:203410:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 0000000030a8fe2c. 00000004:00000010:0.0:1788880172.777124:0:203410:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 00000000e90dfa14. 00000004:00000010:0.0:1788880172.777129:0:203410:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 000000004244f03d. 00000004:00000010:0.0:1788880172.777134:0:203410:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 000000006330c641. 00002000:00000010:0.0:1788880172.777139:0:203410:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 0000000025f082c3. 00000004:00000010:0.0:1788880172.777143:0:203410:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 0000000018c57bb0. 00000020:00000010:0.0:1788880172.777148:0:203410:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 00000000e0784e30. 00000040:00000010:0.0:1788880172.777155:0:203410:0:(llog.c:578:llog_process_thread()) kmalloced '(buf)': 8192 at 00000000a6467e0e. 00000040:00080000:0.0:1788880172.777549:0:203410:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 1 00000040:00000010:0.0:1788880172.777552:0:203410:0:(llog.c:899:llog_process_thread()) kvfree 'buf': 8192 at 00000000a6467e0e. 00000020:00000010:0.0:1788880172.777558:0:203410:0:(lu_object.c:2058:lu_env_remove()) kfreed_rcu 'lei': 40 at 00000000e0784e30. 00000020:00000010:0.0:1788880172.777565:0:203410:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000e0fe5497. 00000040:00000010:0.0:1788880172.777568:0:203410:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000a4a3549f. 00000020:00000010:0.0:1788880172.777571:0:203410:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 00000000964e9d35. 00000020:00000010:0.0:1788880172.777574:0:203410:0:(tgt_main.c:741:tgt_key_fini()) slab-freed '(info)': 1288 at 00000000cb5db106. 00000020:00000010:0.0:1788880172.777577:0:203410:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 00000000fdf771c7. 80000000:00000010:0.0:1788880172.777579:0:203410:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 000000005b8bebe2. 40000000:00000010:0.0:1788880172.777582:0:203410:0:(fid_handler.c:479:seq_key_fini()) kfreed 'info': 48 at 00000000e94ffc2c. 00100000:00000010:0.0:1788880172.777584:0:203410:0:(lfsck_lib.c:43:lfsck_key_fini()) kfreed 'info': 1800 at 00000000ffa4505d. 00040000:00000010:0.0:1788880172.777586:0:203410:0:(lquota_lib.c:35:lquota_key_fini()) kfreed 'info': 248 at 00000000b899fe8d. 00040000:00000010:0.0:1788880172.777589:0:203410:0:(qmt_dev.c:418:qmt_key_fini()) kfreed 'info': 1256 at 00000000842c011f. 00040000:00000010:0.0:1788880172.777599:0:203410:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 0000000030a8fe2c. 00000004:00000010:0.0:1788880172.777609:0:203410:0:(mdd_device.c:2420:mdd_key_fini()) kfreed 'info': 1872 at 00000000e90dfa14. 00000004:00000010:0.0:1788880172.777613:0:203410:0:(mdt_handler.c:8495:mdt_key_fini()) kfreed 'info': 2568 at 000000004244f03d. 00000004:00000010:0.0:1788880172.777618:0:203410:0:(lod_dev.c:2378:lod_key_fini()) kfreed 'info': 1224 at 000000006330c641. 00002000:00000010:0.0:1788880172.777621:0:203410:0:(ofd_dev.c:3002:ofd_key_fini()) kfreed 'info': 816 at 0000000025f082c3. 00000004:00000010:0.0:1788880172.777624:0:203410:0:(osp_dev.c:1888:osp_key_fini()) kfreed 'info': 616 at 0000000018c57bb0. 00000020:00000010:0.0:1788880172.777631:0:203410:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000b5c40084. 00000040:00000010:2.0:1788880172.777669:0:203392:0:(llog.c:994:llog_process_or_fork()) kfreed 'lpi': 96 at 000000000171f5bf. 00000040:00000001:2.0:1788880172.777675:0:203392:0:(llog.c:995:llog_process_or_fork()) Process leaving (rc=0 : 0 : 0) 00000020:01000000:2.0:1788880172.777677:0:203392:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 1-0 (rc=0) 00000040:00000010:2.0:1788880172.777682:0:203392:0:(llog.c:70:llog_free_handle()) kvfree 'loghandle->lgh_hdr': 8192 at 00000000c68794ed. 00000040:00000010:2.0:1788880172.777686:0:203392:0:(llog.c:72:llog_free_handle()) kfreed 'loghandle': 352 at 00000000b405d9c4. 00000020:00000001:2.0:1788880172.777688:0:203392:0:(obd_config.c:2151:class_config_parse_llog()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880172.777692:0:203392:0:(mgc_request.c:1724:mgc_process_cfg_log()) Process leaving 00000020:00000010:2.0:1788880172.777695:0:203392:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 0000000063883087. 00000040:00000010:2.0:1788880172.777699:0:203392:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000605d7309. 00000020:00000010:2.0:1788880172.777702:0:203392:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 000000006aef07b8. 00000020:00000010:2.0:1788880172.777705:0:203392:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 0000000081a10824. 80000000:00000010:2.0:1788880172.777709:0:203392:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 000000006050354c. 00040000:00000010:2.0:1788880172.777712:0:203392:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 00000000358be29b. 20000000:00000010:2.0:1788880172.777715:0:203392:0:(mgs_handler.c:1787:mgs_key_fini()) kfreed 'info': 392 at 000000008f22742c. 00000020:00000010:2.0:1788880172.777718:0:203392:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000d653e4f6. 10000000:00000010:2.0:1788880172.777720:0:203392:0:(mgc_request.c:1730:mgc_process_cfg_log()) kfreed 'env': 56 at 00000000a9f9e43c. 10000000:01000000:2.0:1788880172.777722:0:203392:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'params' succeeded (0). 10000000:00000001:2.0:1788880172.777732:0:203392:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:2.0:1788880172.777735:0:203392:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:2.0:1788880172.777737:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880172.777740:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000f14805ea refcount=3 00000020:00000001:2.0:1788880172.777742:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250084075008 : -126823625476608 : ffff8ca792dab600) 00010000:00000001:2.0:1788880172.777746:0:203392:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250084075008 : -126823625476608 : ffff8ca792dab600) 00010000:00000001:2.0:1788880172.777748:0:203392:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:2.0:1788880172.777752:0:203392:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 00000000f14805ea/0x805d548a36a88690 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7917 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880172.777761:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880172.777762:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:2.0:1788880172.777764:0:203392:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 00000000f14805ea/0x805d548a36a88690 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7917 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880172.777770:0:203392:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:2.0:1788880172.777776:0:203392:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880172.777778:0:203392:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:2.0:1788880172.777779:0:203392:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880172.777781:0:203392:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880172.777784:0:203392:0:(mgc_request.c:2016:mgc_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880172.777787:0:203392:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 0000000026532f47. 00000040:00000010:2.0:1788880172.777791:0:203392:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000fa0f7569. 00000020:00000010:2.0:1788880172.777793:0:203392:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 0000000068effccb. 00000020:00000010:2.0:1788880172.777799:0:203392:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 00000000c0495ef7. 00040000:00000010:2.0:1788880172.777802:0:203392:0:(lquota_lib.c:35:lquota_key_fini()) kfreed 'info': 248 at 00000000d0fdfcdc. 00040000:00000010:2.0:1788880172.777806:0:203392:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 0000000078217d0d. 00000020:00000010:2.0:1788880172.777810:0:203392:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000058b4ee38. 00000020:00000001:2.0:1788880172.777813:0:203392:0:(obd_class.h:704:obd_process_config()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880172.777815:0:203392:0:(obd_mount.c:81:lustre_process_log()) kfreed 'lcfg': 184 at 00000000617a5f88. 00000020:00000010:2.0:1788880172.777818:0:203392:0:(obd_mount.c:83:lustre_process_log()) kfreed 'bufs': 104 at 000000001ca1494e. 00000020:00000001:2.0:1788880172.777822:0:203392:0:(obd_mount.c:92:lustre_process_log()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.777825:0:203392:0:(obd_config.c:1127:class_get_profile()) Process entered 00000020:00000001:2.0:1788880172.777827:0:203392:0:(obd_config.c:1113:class_get_profile_nolock()) Process entered 00000020:00000001:2.0:1788880172.777829:0:203392:0:(obd_config.c:1117:class_get_profile_nolock()) Process leaving (rc=18446617250282352256 : -126823427199360 : ffff8ca79eac2e80) 00000020:00000001:2.0:1788880172.777831:0:203392:0:(obd_config.c:1131:class_get_profile()) Process leaving (rc=18446617250282352256 : -126823427199360 : ffff8ca79eac2e80) 00000080:01000000:2.0:1788880172.777834:0:203392:0:(llite_lib.c:1623:ll_fill_super()) Found profile lustre-client: mdc=lustre-clilmv osc=lustre-clilov 00000080:00000010:2.0:1788880172.777838:0:203392:0:(llite_lib.c:1626:ll_fill_super()) kmalloced '(dt)': 33 at 00000000f0a428ff. 00000080:00000010:2.0:1788880172.777841:0:203392:0:(llite_lib.c:1632:ll_fill_super()) kmalloced '(md)': 33 at 0000000057834ed5. 00000080:00000001:2.0:1788880172.777844:0:203392:0:(llite_lib.c:437:client_common_fill_super()) Process entered 00000080:00000010:2.0:1788880172.777851:0:203392:0:(llite_lib.c:445:client_common_fill_super()) kmalloced '(data)': 192 at 0000000000211877. 00000080:00000010:2.0:1788880172.777853:0:203392:0:(llite_lib.c:449:client_common_fill_super()) kmalloced '(osfs)': 144 at 0000000055a084bf. 00000080:00000040:2.0:1788880172.777856:0:203392:0:(obd_cksum.h:77:obd_cksum_types_supported_client()) Crypto hash speed: crc 1651, crc32c 9963, adler 1371 00000080:00000001:2.0:1788880172.777860:0:203392:0:(obd_class.h:860:obd_connect()) Process entered 00800000:00000001:2.0:1788880172.777863:0:203392:0:(lmv_obd.c:236:lmv_connect()) Process entered 00000020:00000001:2.0:1788880172.777866:0:203392:0:(genops.c:1396:class_connect()) Process entered 00000020:00000001:2.0:1788880172.777868:0:203392:0:(genops.c:952:__class_new_export()) Process entered 00000020:00000010:2.0:1788880172.777870:0:203392:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 00000000c18d0b7e. 00000020:00000001:2.0:1788880172.777873:0:203392:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880172.777875:0:203392:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000c18d0b7e with handle 0x805d548a36a88697 to hash 00000020:00000001:2.0:1788880172.777877:0:203392:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:2.0:1788880172.777889:0:203392:0:(obd_class.h:1016:obd_init_export()) Process entered 00000020:00000001:2.0:1788880172.777896:0:203392:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880172.777899:0:203392:0:(obd_config.c:994:class_incref()) incref lustre-clilmv-ffff8ca787527000 (00000000aeea1d9b) now 5 - export 00000020:00000001:2.0:1788880172.777902:0:203392:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617249948762112 : -126823760789504 : ffff8ca78aca0000) 00000020:00000040:2.0:1788880172.777904:0:203392:0:(genops.c:900:class_export_put()) PUTting export 00000000c18d0b7e : new refcount 1 00000020:00000080:2.0:1788880172.777907:0:203392:0:(genops.c:1406:class_connect()) connect: client ea27006f-52a8-4007-b69a-dcac8279445a, cookie 0x805d548a36a88697 00000020:00000001:2.0:1788880172.777909:0:203392:0:(genops.c:1407:class_connect()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.777911:0:203392:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880172.777912:0:203392:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88697 00000020:00000001:2.0:1788880172.777913:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880172.777915:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000c18d0b7e refcount=2 00000020:00000001:2.0:1788880172.777917:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948762112 : -126823760789504 : ffff8ca78aca0000) 00000020:00000001:2.0:1788880172.777919:0:203392:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948762112 : -126823760789504 : ffff8ca78aca0000) 00800000:00000001:2.0:1788880172.777942:0:203392:0:(lmv_obd.c:490:lmv_check_connect()) Process entered 00800000:01000000:2.0:1788880172.777944:0:203392:0:(lmv_obd.c:512:lmv_check_connect()) Time to connect ea27006f-52a8-4007-b69a-dcac8279445a to lustre-clilmv-ffff8ca787527000 00800000:00000001:2.0:1788880172.777947:0:203392:0:(lmv_obd.c:324:lmv_connect_mdc()) Process entered 00800000:01000000:2.0:1788880172.777953:0:203392:0:(lmv_obd.c:335:lmv_connect_mdc()) connect to lustre-MDT0000-mdc-ffff8ca787527000(ea27006f-52a8-4007-b69a-dcac8279445a) - lustre-MDT0000_UUID, ea27006f-52a8-4007-b69a-dcac8279445a 00800000:00000001:2.0:1788880172.777956:0:203392:0:(obd_class.h:1356:obd_register_observer()) Process entered 00800000:00000001:2.0:1788880172.777958:0:203392:0:(obd_class.h:1369:obd_register_observer()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880172.777960:0:203392:0:(obd_class.h:860:obd_connect()) Process entered 00010000:00000001:2.0:1788880172.777962:0:203392:0:(ldlm_lib.c:668:client_connect_import()) Process entered 00000020:00000001:2.0:1788880172.777964:0:203392:0:(genops.c:1396:class_connect()) Process entered 00000020:00000001:2.0:1788880172.777965:0:203392:0:(genops.c:952:__class_new_export()) Process entered 00000020:00000010:2.0:1788880172.777968:0:203392:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 0000000017f88dbf. 00000020:00000001:2.0:1788880172.777969:0:203392:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880172.777971:0:203392:0:(lustre_handles.c:93:class_handle_hash()) added object 0000000017f88dbf with handle 0x805d548a36a8869e to hash 00000020:00000001:2.0:1788880172.777973:0:203392:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:2.0:1788880172.777977:0:203392:0:(obd_class.h:1016:obd_init_export()) Process entered 00000020:00000001:2.0:1788880172.777978:0:203392:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880172.777980:0:203392:0:(obd_config.c:994:class_incref()) incref lustre-MDT0000-mdc-ffff8ca787527000 (000000009cb5873b) now 5 - export 00000020:00000001:2.0:1788880172.777981:0:203392:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617249948786688 : -126823760764928 : ffff8ca78aca6000) 00000020:00000040:2.0:1788880172.777983:0:203392:0:(genops.c:900:class_export_put()) PUTting export 0000000017f88dbf : new refcount 1 00000020:00000080:2.0:1788880172.777985:0:203392:0:(genops.c:1406:class_connect()) connect: client ea27006f-52a8-4007-b69a-dcac8279445a, cookie 0x805d548a36a8869e 00000020:00000001:2.0:1788880172.777990:0:203392:0:(genops.c:1407:class_connect()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.777992:0:203392:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880172.777993:0:203392:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a8869e 00000020:00000001:2.0:1788880172.777994:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880172.777995:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET export 0000000017f88dbf refcount=2 00000020:00000001:2.0:1788880172.777997:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948786688 : -126823760764928 : ffff8ca78aca6000) 00000020:00000001:2.0:1788880172.777999:0:203392:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948786688 : -126823760764928 : ffff8ca78aca6000) 00000100:00000001:2.0:1788880172.778004:0:203392:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00000001:2.0:1788880172.778006:0:203392:0:(import.c:485:import_select_connection()) Process entered 00000100:00080000:2.0:1788880172.778010:0:203392:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 0 00000400:00000200:2.0:1788880172.778016:0:203392:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000100:00000001:2.0:1788880172.778020:0:203392:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880172.778023:0:203392:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 7 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880172.778025:0:203392:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000001:2.0:1788880172.778038:0:203392:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880172.778039:0:203392:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a8869e 00000020:00000001:2.0:1788880172.778040:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880172.778042:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET export 0000000017f88dbf refcount=3 00000020:00000001:2.0:1788880172.778044:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948786688 : -126823760764928 : ffff8ca78aca6000) 00000020:00000001:2.0:1788880172.778045:0:203392:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948786688 : -126823760764928 : ffff8ca78aca6000) 00000100:00000001:2.0:1788880172.778047:0:203392:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880172.778049:0:203392:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 8 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880172.778051:0:203392:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000040:2.0:1788880172.778053:0:203392:0:(genops.c:900:class_export_put()) PUTting export 0000000017f88dbf : new refcount 2 00000100:00080000:2.0:1788880172.778055:0:203392:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8ca787527000: import 0000000095473c20 using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000100:00000001:2.0:1788880172.778058:0:203392:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880172.778060:0:203392:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:2.0:1788880172.778066:0:203392:0:(sec.c:1453:sptlrpc_sec_create()) Process entered 02000000:08000000:2.0:1788880172.778075:0:203392:0:(sec.c:1471:sptlrpc_sec_create()) mdc lustre-MDT0000-mdc-ffff8ca787527000: select security flavor null 02000000:00000001:2.0:1788880172.778079:0:203392:0:(sec.c:1492:sptlrpc_sec_create()) Process leaving (rc=18446744072650609120 : -1058942496 : ffffffffc0e1d1e0) 02000000:00000001:2.0:1788880172.778082:0:203392:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.778088:0:203392:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:2.0:1788880172.778091:0:203392:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 134217728, lost: 0. 00000008:00000001:2.0:1788880172.778095:0:203392:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.778096:0:203392:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880172.778100:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000020:00000040:2.0:1788880172.778104:0:203392:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:2.0:1788880172.778108:0:203392:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 02000000:00000001:2.0:1788880172.778109:0:203392:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.778112:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880172.778115:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880172.778118:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880172.778123:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 00000000df2b698c. 00000100:00000001:2.0:1788880172.778129:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880172.778135:0:203392:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fd6b7ee1 x1875776780633856/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880172.778144:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880172.778146:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880172.778150:0:203392:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fd6b7ee1] to pc [ptlrpcd_rcv+-1] req@00000000fd6b7ee1 x1875776780633856/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.778169:0:203392:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.778172:0:203392:0:(pinger.c:367:ptlrpc_pinger_add_import()) Process entered 00000100:00080000:2.0:1788880172.778174:0:203392:0:(pinger.c:373:ptlrpc_pinger_add_import()) adding pingable import ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID 00000020:00000040:2.0:1788880172.778176:0:203392:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880172.778195:0:203392:0:(pinger.c:384:ptlrpc_pinger_add_import()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880172.778199:0:203392:0:(ldlm_lib.c:725:client_connect_import()) Process leaving 00800000:00000001:2.0:1788880172.778204:0:203392:0:(obd_class.h:876:obd_connect()) Process leaving (rc=0 : 0 : 0) 40000000:00000001:2.0:1788880172.778208:0:203392:0:(fid_request.c:435:client_fid_init()) Process entered 40000000:00000010:2.0:1788880172.778215:0:203392:0:(fid_request.c:438:client_fid_init()) kmalloced '(cli->cl_seq)': 224 at 00000000d0fdfcdc. 40000000:00000010:2.0:1788880172.778221:0:203392:0:(fid_request.c:442:client_fid_init()) kmalloced '(prefix)': 133 at 00000000126d8278. 40000000:00000001:2.0:1788880172.778228:0:203392:0:(fid_request.c:401:seq_client_init()) Process entered 00000100:00000040:0.0:1788880172.778269:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11141) 00000800:00000200:0.0:1788880172.778529:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880172.778533:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000020:00000040:2.0:1788880172.778535:0:203392:0:(genops.c:889:class_export_get()) GET export 0000000017f88dbf refcount=3 00000400:00000200:0.0:1788880172.778540:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.778546:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880172.778551:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000994ad42e 00000400:00000010:0.0:1788880172.778553:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000994ad42e. 00000100:00000001:0.0:1788880172.778559:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880172.778563:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000fd6b7ee1 x1875776780633856/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880182 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.778573:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880172.778577:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000fd6b7ee1 x1875776780633856/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880182 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.778584:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.778586:0:201279:0:(events.c:68:request_out_callback()) Process leaving 40000000:00000010:2.0:1788880172.778627:0:203392:0:(fid_request.c:450:client_fid_init()) kfreed 'prefix': 133 at 00000000126d8278. 40000000:00000001:2.0:1788880172.778631:0:203392:0:(fid_request.c:459:client_fid_init()) Process leaving (rc=0 : 0 : 0) 80000000:00000001:2.0:1788880172.778637:0:203392:0:(fld_request.c:140:fld_client_add_target()) Process entered 80000000:00000040:2.0:1788880172.778640:0:203392:0:(fld_request.c:148:fld_client_add_target()) cli-lustre-clilmv-ffff8ca787527000: Adding target lustre-MDT0000-mdc-ffff8ca787527000 (idx 0) 80000000:00000010:2.0:1788880172.778643:0:203392:0:(fld_request.c:150:fld_client_add_target()) kmalloced '(target)': 40 at 00000000a61e1d3c. 00000020:00000040:2.0:1788880172.778646:0:203392:0:(genops.c:889:class_export_get()) GET export 0000000017f88dbf refcount=4 80000000:00000001:2.0:1788880172.778648:0:203392:0:(fld_request.c:176:fld_client_add_target()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880172.778651:0:203392:0:(mdc_request.c:3107:mdc_init_ea_size()) Process entered 00000002:00000001:2.0:1788880172.778653:0:203392:0:(mdc_request.c:3114:mdc_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880172.778655:0:203392:0:(lu_tgt_descs.c:82:lu_qos_add_tgt()) Process entered 00020000:00000010:2.0:1788880172.778658:0:203392:0:(lu_tgt_descs.c:102:lu_qos_add_tgt()) kmalloced '(svr)': 104 at 000000001ca1494e. 00020000:00001000:2.0:1788880172.778662:0:203392:0:(lu_tgt_descs.c:121:lu_qos_add_tgt()) add tgt lustre-MDT0000_UUID to server 192.168.204.111@tcp (1 targets) 00020000:00000001:2.0:1788880172.778665:0:203392:0:(lu_tgt_descs.c:143:lu_qos_add_tgt()) Process leaving (rc=0 : 0 : 0) 00800000:01000000:2.0:1788880172.778666:0:203392:0:(lmv_obd.c:389:lmv_connect_mdc()) Connected to lustre-MDT0000-mdc-ffff8ca787527000(ea27006f-52a8-4007-b69a-dcac8279445a) successfully (5) 00800000:00000001:2.0:1788880172.778670:0:203392:0:(obd_class.h:1044:obd_statfs_async()) Process entered 00800000:00000004:2.0:1788880172.778672:0:203392:0:(obd_class.h:1057:obd_statfs_async()) lustre-MDT0000-mdc-ffff8ca787527000: age 10136, max_age 0 00000100:00000010:2.0:1788880172.778676:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 00000020:00000040:2.0:1788880172.778679:0:203392:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=5 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880172.778681:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880172.778683:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880172.778685:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880172.778689:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000036dc6dd9. 00000100:00000001:2.0:1788880172.778692:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.778695:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880172.778696:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880172.778701:0:203392:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_01+1] req@00000000175072aa x1875776780633984/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00800000:00000001:2.0:1788880172.778724:0:203392:0:(obd_class.h:1061:obd_statfs_async()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880172.778733:0:203392:0:(lmv_obd.c:403:lmv_connect_mdc()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880172.778737:0:203392:0:(lmv_obd.c:324:lmv_connect_mdc()) Process entered 00000100:00000001:3.0:1788880172.778737:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00800000:01000000:2.0:1788880172.778741:0:203392:0:(lmv_obd.c:335:lmv_connect_mdc()) connect to lustre-MDT0001-mdc-ffff8ca787527000(ea27006f-52a8-4007-b69a-dcac8279445a) - lustre-MDT0001_UUID, ea27006f-52a8-4007-b69a-dcac8279445a 00800000:00000001:2.0:1788880172.778745:0:203392:0:(obd_class.h:1356:obd_register_observer()) Process entered 00000020:00000010:3.0:1788880172.778746:0:201286:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 00000000039a16c7. 00800000:00000001:2.0:1788880172.778748:0:203392:0:(obd_class.h:1369:obd_register_observer()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880172.778752:0:203392:0:(obd_class.h:860:obd_connect()) Process entered 00000020:00000010:3.0:1788880172.778753:0:201286:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000d0916d04. 00010000:00000001:2.0:1788880172.778754:0:203392:0:(ldlm_lib.c:668:client_connect_import()) Process entered 80000000:00000010:3.0:1788880172.778760:0:201286:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000594debb1. 00000020:00000001:2.0:1788880172.778762:0:203392:0:(genops.c:1396:class_connect()) Process entered 00000020:00000001:2.0:1788880172.778765:0:203392:0:(genops.c:952:__class_new_export()) Process entered 40000000:00000010:3.0:1788880172.778765:0:201286:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 000000009e7d9176. 00000020:00000010:2.0:1788880172.778769:0:203392:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 00000000673f59bf. 00000020:00000001:2.0:1788880172.778773:0:203392:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000008:00000010:3.0:1788880172.778774:0:201286:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000aeeba8e2. 00000020:00000040:2.0:1788880172.778775:0:203392:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000673f59bf with handle 0x805d548a36a886a5 to hash 00000020:00000001:2.0:1788880172.778780:0:203392:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:2.0:1788880172.778782:0:203392:0:(obd_class.h:1016:obd_init_export()) Process entered 00020000:00000010:3.0:1788880172.778783:0:201286:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000b8c7c243. 00000020:00000001:2.0:1788880172.778786:0:203392:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880172.778791:0:203392:0:(obd_config.c:994:class_incref()) incref lustre-MDT0001-mdc-ffff8ca787527000 (000000001b5d8280) now 5 - export 00008000:00000010:3.0:1788880172.778791:0:201286:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000d0b9f970. 00000020:00000001:2.0:1788880172.778798:0:203392:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00100000:00000010:3.0:1788880172.778801:0:201286:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 000000004fc6b5e3. 00000020:00000040:2.0:1788880172.778802:0:203392:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 1 00000020:00000080:2.0:1788880172.778807:0:203392:0:(genops.c:1406:class_connect()) connect: client ea27006f-52a8-4007-b69a-dcac8279445a, cookie 0x805d548a36a886a5 00040000:00000010:3.0:1788880172.778810:0:201286:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 0000000022ee3f90. 00000020:00000001:2.0:1788880172.778811:0:203392:0:(genops.c:1407:class_connect()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880172.778815:0:203392:0:(genops.c:813:class_conn2export()) Process entered 00040000:00000010:3.0:1788880172.778818:0:201286:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 000000002294f5ed. 00000020:00000040:2.0:1788880172.778819:0:203392:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:2.0:1788880172.778824:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00040000:00000010:3.0:1788880172.778825:0:201286:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 0000000004f7ba43. 00000020:00000040:2.0:1788880172.778827:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=2 00000020:00000001:2.0:1788880172.778831:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000004:00000010:3.0:1788880172.778833:0:201286:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 0000000075265214. 00000020:00000001:2.0:1788880172.778838:0:203392:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000004:00000010:3.0:1788880172.778842:0:201286:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 000000000dec04da. 00000100:00000001:2.0:1788880172.778843:0:203392:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00000001:2.0:1788880172.778848:0:203392:0:(import.c:485:import_select_connection()) Process entered 00000004:00000010:3.0:1788880172.778850:0:201286:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 000000006304aab5. 00000100:00080000:2.0:1788880172.778853:0:203392:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 0 00002000:00000010:3.0:1788880172.778857:0:201286:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 000000008dad43aa. 00000400:00000200:2.0:1788880172.778862:0:203392:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000004:00000010:3.0:1788880172.778863:0:201286:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 00000000a86c3f1c. 00000100:00000001:2.0:1788880172.778868:0:203392:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000080:00000010:3.0:1788880172.778871:0:201286:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 0000000020ae732f. 00000100:00000040:2.0:1788880172.778873:0:203392:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 9 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880172.778877:0:203392:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000001:2.0:1788880172.778881:0:203392:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880172.778884:0:203392:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:2.0:1788880172.778887:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880172.778891:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=3 00000020:00000001:2.0:1788880172.778894:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000020:00000001:2.0:1788880172.778899:0:203392:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000100:00000001:2.0:1788880172.778901:0:203392:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880172.778912:0:203392:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 10 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880172.778916:0:203392:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000040:2.0:1788880172.778920:0:203392:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 2 00000100:00080000:2.0:1788880172.778926:0:203392:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: import 00000000b98b8a4e using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000100:00000001:2.0:1788880172.778930:0:203392:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880172.778934:0:203392:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:2.0:1788880172.778938:0:203392:0:(sec.c:1453:sptlrpc_sec_create()) Process entered 02000000:08000000:2.0:1788880172.778942:0:203392:0:(sec.c:1471:sptlrpc_sec_create()) mdc lustre-MDT0001-mdc-ffff8ca787527000: select security flavor null 02000000:00000001:2.0:1788880172.778944:0:203392:0:(sec.c:1492:sptlrpc_sec_create()) Process leaving (rc=18446744072650609120 : -1058942496 : ffffffffc0e1d1e0) 02000000:00000001:2.0:1788880172.778947:0:203392:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.778949:0:203392:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:2.0:1788880172.778951:0:203392:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 134217728, lost: 0. 00000008:00000001:2.0:1788880172.778954:0:203392:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.778955:0:203392:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880172.778959:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000080:00000010:3.0:1788880172.778961:0:201286:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 000000007ec0a7c6. 00000020:00000040:2.0:1788880172.778963:0:203392:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:2.0:1788880172.778967:0:203392:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 02000000:00000001:2.0:1788880172.778968:0:203392:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000008:00000010:3.0:1788880172.778969:0:201286:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000ae551350. 00000100:00000001:2.0:1788880172.778971:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880172.778974:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00020000:00000010:3.0:1788880172.778974:0:201286:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 000000009b9eb526. 02000000:00000001:2.0:1788880172.778976:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00008000:00000010:3.0:1788880172.778979:0:201286:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 00000000db05d82b. 02000000:00000010:2.0:1788880172.778982:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 00000000150c5893. 00000080:00000010:3.0:1788880172.778985:0:201286:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 00000000e9f8c309. 00000100:00000001:2.0:1788880172.778987:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.778989:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880172.778993:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00100000:2.0:1788880172.778994:0:203392:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d838cab4 x1875776780634112/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:3.0:1788880172.779000:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780633984/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.779010:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880172.779013:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.779016:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880172.779019:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880172.779020:0:203392:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d838cab4] to pc [ptlrpcd_rcv+-1] req@00000000d838cab4 x1875776780634112/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788880172.779048:0:201286:0:(client.c:1832:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@00000000175072aa x1875776780633984/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.779052:0:203392:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.779057:0:203392:0:(pinger.c:367:ptlrpc_pinger_add_import()) Process entered 00000100:00080000:2.0:1788880172.779061:0:203392:0:(pinger.c:373:ptlrpc_pinger_add_import()) adding pingable import ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID 00000100:00000001:3.0:1788880172.779061:0:201286:0:(client.c:1837:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.779066:0:201286:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.779071:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000020:00000040:2.0:1788880172.779072:0:203392:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880172.779074:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.779078:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.779081:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.779086:0:203392:0:(pinger.c:384:ptlrpc_pinger_add_import()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.779088:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:2.0:1788880172.779090:0:203392:0:(ldlm_lib.c:725:client_connect_import()) Process leaving 00000100:00000001:3.0:1788880172.779091:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880172.779095:0:203392:0:(obd_class.h:876:obd_connect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.779096:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880172.779099:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 40000000:00000001:2.0:1788880172.779100:0:203392:0:(fid_request.c:435:client_fid_init()) Process entered 00000100:00000001:3.0:1788880172.779104:0:201286:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 40000000:00000010:2.0:1788880172.779105:0:203392:0:(fid_request.c:438:client_fid_init()) kmalloced '(cli->cl_seq)': 224 at 000000009cf93093. 00000100:00000001:3.0:1788880172.779109:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 40000000:00000010:2.0:1788880172.779111:0:203392:0:(fid_request.c:442:client_fid_init()) kmalloced '(prefix)': 133 at 00000000126d8278. 00000100:00000001:3.0:1788880172.779113:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 40000000:00000001:2.0:1788880172.779119:0:203392:0:(fid_request.c:401:seq_client_init()) Process entered 00000100:00000001:3.0:1788880172.779119:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880172.779121:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11141) 00000020:00000040:2.0:1788880172.779123:0:203392:0:(genops.c:889:class_export_get()) GET export 00000000673f59bf refcount=3 00000100:00000001:3.0:1788880172.779125:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 40000000:00000010:2.0:1788880172.779175:0:203392:0:(fid_request.c:450:client_fid_init()) kfreed 'prefix': 133 at 00000000126d8278. 40000000:00000001:2.0:1788880172.779180:0:203392:0:(fid_request.c:459:client_fid_init()) Process leaving (rc=0 : 0 : 0) 80000000:00000001:2.0:1788880172.779185:0:203392:0:(fld_request.c:140:fld_client_add_target()) Process entered 80000000:00000040:2.0:1788880172.779188:0:203392:0:(fld_request.c:148:fld_client_add_target()) cli-lustre-clilmv-ffff8ca787527000: Adding target lustre-MDT0001-mdc-ffff8ca787527000 (idx 1) 80000000:00000010:2.0:1788880172.779191:0:203392:0:(fld_request.c:150:fld_client_add_target()) kmalloced '(target)': 40 at 000000004909a0e0. 00000020:00000040:2.0:1788880172.779194:0:203392:0:(genops.c:889:class_export_get()) GET export 00000000673f59bf refcount=4 80000000:00000001:2.0:1788880172.779198:0:203392:0:(fld_request.c:176:fld_client_add_target()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880172.779201:0:203392:0:(mdc_request.c:3107:mdc_init_ea_size()) Process entered 00000002:00000001:2.0:1788880172.779204:0:203392:0:(mdc_request.c:3114:mdc_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880172.779209:0:203392:0:(lu_tgt_descs.c:82:lu_qos_add_tgt()) Process entered 00020000:00001000:2.0:1788880172.779215:0:203392:0:(lu_tgt_descs.c:121:lu_qos_add_tgt()) add tgt lustre-MDT0001_UUID to server 192.168.204.111@tcp (2 targets) 00020000:00000001:2.0:1788880172.779220:0:203392:0:(lu_tgt_descs.c:143:lu_qos_add_tgt()) Process leaving (rc=0 : 0 : 0) 00800000:01000000:2.0:1788880172.779225:0:203392:0:(lmv_obd.c:389:lmv_connect_mdc()) Connected to lustre-MDT0001-mdc-ffff8ca787527000(ea27006f-52a8-4007-b69a-dcac8279445a) successfully (5) 00800000:00000001:2.0:1788880172.779230:0:203392:0:(obd_class.h:1044:obd_statfs_async()) Process entered 00800000:00000004:2.0:1788880172.779258:0:203392:0:(obd_class.h:1057:obd_statfs_async()) lustre-MDT0001-mdc-ffff8ca787527000: age 10136, max_age 0 00000100:00000010:2.0:1788880172.779265:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880172.779271:0:203392:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=5 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880172.779276:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880172.779279:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880172.779284:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880172.779291:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000ea04942e. 00000100:00000001:2.0:1788880172.779297:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.779303:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880172.779306:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880172.779316:0:203392:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_02+2] req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00800000:00000001:2.0:1788880172.779352:0:203392:0:(obd_class.h:1061:obd_statfs_async()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880172.779361:0:203392:0:(lmv_obd.c:403:lmv_connect_mdc()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.779361:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00800000:00000001:2.0:1788880172.779368:0:203392:0:(lmv_obd.c:283:lmv_init_ea_size()) Process entered 00000020:00000010:3.0:1788880172.779368:0:201287:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 00000000903a59a7. 00000002:00000001:2.0:1788880172.779372:0:203392:0:(mdc_request.c:3107:mdc_init_ea_size()) Process entered 00000020:00000010:3.0:1788880172.779373:0:201287:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000bb819a61. 00000002:00000001:2.0:1788880172.779375:0:203392:0:(mdc_request.c:3114:mdc_init_ea_size()) Process leaving (rc=0 : 0 : 0) 80000000:00000010:3.0:1788880172.779377:0:201287:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 000000006478bb14. 00000002:00000001:2.0:1788880172.779379:0:203392:0:(mdc_request.c:3107:mdc_init_ea_size()) Process entered 00000002:00000001:2.0:1788880172.779383:0:203392:0:(mdc_request.c:3114:mdc_init_ea_size()) Process leaving (rc=0 : 0 : 0) 40000000:00000010:3.0:1788880172.779384:0:201287:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 000000004c5cfc03. 00800000:00000001:2.0:1788880172.779386:0:203392:0:(lmv_obd.c:311:lmv_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000008:00000010:3.0:1788880172.779390:0:201287:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 000000003ed943f4. 00800000:00000001:2.0:1788880172.779391:0:203392:0:(lmv_obd.c:523:lmv_check_connect()) Process leaving 00020000:00000010:3.0:1788880172.779394:0:201287:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000d1b42643. 00800000:00000001:2.0:1788880172.779395:0:203392:0:(lmv_obd.c:263:lmv_connect()) Process leaving (rc=0 : 0 : 0) 00008000:00000010:3.0:1788880172.779399:0:201287:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 0000000043cb18d2. 00000080:00000001:2.0:1788880172.779401:0:203392:0:(obd_class.h:876:obd_connect()) Process leaving (rc=0 : 0 : 0) 00100000:00000010:3.0:1788880172.779402:0:201287:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 000000004943a065. 00000080:00000001:2.0:1788880172.779407:0:203392:0:(obd_class.h:1075:obd_statfs()) Process entered 00040000:00000010:3.0:1788880172.779408:0:201287:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 000000007bde436a. 00000080:00000004:2.0:1788880172.779411:0:203392:0:(obd_class.h:1091:obd_statfs()) lustre-clilmv-ffff8ca787527000: age 10136, max_age 11135 00040000:00000010:3.0:1788880172.779414:0:201287:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 000000006e0432fd. 00800000:00000001:2.0:1788880172.779418:0:203392:0:(lmv_obd.c:1447:lmv_statfs()) Process entered 00000800:00000200:0.0:1788880172.779421:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00040000:00000010:3.0:1788880172.779421:0:201287:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000fdddab51. 00800000:00000010:2.0:1788880172.779423:0:203392:0:(lmv_obd.c:1451:lmv_statfs()) kmalloced '(temp)': 144 at 00000000126d8278. 00000800:00000010:0.0:1788880172.779427:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000004:00000010:3.0:1788880172.779428:0:201287:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 00000000d484f229. 00800000:00000001:2.0:1788880172.779429:0:203392:0:(obd_class.h:1075:obd_statfs()) Process entered 00800000:00000004:2.0:1788880172.779433:0:203392:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff8ca787527000: age 10136, max_age 11135 00000400:00000200:0.0:1788880172.779435:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000002:00000001:2.0:1788880172.779440:0:203392:0:(mdc_request.c:1667:mdc_statfs()) Process entered 00000020:00000040:2.0:1788880172.779444:0:203392:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=6 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000200:0.0:1788880172.779445:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880172.779452:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a131edae 00000100:00000010:2.0:1788880172.779453:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000400:00000010:0.0:1788880172.779456:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a131edae. 00000020:00000040:2.0:1788880172.779458:0:203392:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=7 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880172.779463:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880172.779463:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880172.779467:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000200:0.0:1788880172.779470:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780634112/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880182 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880172.779471:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880172.779477:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000002c5b8fef. 00000100:00000001:2.0:1788880172.779483:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.779484:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880172.779488:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000040:0.0:1788880172.779489:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780634112/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880182 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.779492:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:2.0:1788880172.779499:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 000000009e67569f. 00000100:00000001:2.0:1788880172.779503:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249931841280 : -126823777710336 : ffff8ca789c7cf00) 00000100:00000001:2.0:1788880172.779509:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880172.779512:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000100:00000001:2.0:1788880172.779520:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880172.779523:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.779523:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880172.779527:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880172.779534:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:2.0:1788880172.779537:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880172.779543:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.779558:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880172.779562:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00080000:2.0:1788880172.779570:0:203392:0:(client.c:1832:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880172.779584:0:203392:0:(client.c:1837:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880172.779587:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880172.779588:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880172.779594:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000009e67569f going to sleep for 0 seconds 00000100:00000001:2.0:1788880172.779598:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880172.779602:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.779607:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880172.779608:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880172.779609:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880172.779674:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000004:00000010:3.0:1788880172.779677:0:201287:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 0000000074539a6a. 00000004:00000010:3.0:1788880172.779684:0:201287:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 00000000581a32f5. 00000800:00000001:0.0:1788880172.779686:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00002000:00000010:3.0:1788880172.779689:0:201287:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 00000000ce55c909. 00000800:00000001:0.0:1788880172.779692:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000004:00000010:3.0:1788880172.779696:0:201287:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 00000000af8e29f2. 00000800:00000001:0.0:1788880172.779698:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000080:00000010:3.0:1788880172.779703:0:201287:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 0000000074c156cc. 00000400:00000200:0.0:1788880172.779708:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880172.779723:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 416 into portal 10 MB=0x6aa0250b00700 00000080:00000010:3.0:1788880172.779728:0:201287:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 00000000975245d5. 00000400:00000200:0.0:1788880172.779734:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 416/416 into md 0x185 [1] + 0 00000008:00000010:3.0:1788880172.779738:0:201287:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000c10b0087. 00000800:00000001:0.0:1788880172.779742:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00020000:00000010:3.0:1788880172.779744:0:201287:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 00000000b4bc14bc. 00008000:00000010:3.0:1788880172.779750:0:201287:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 0000000028ed48f8. 00000800:00000001:0.0:1788880172.779752:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000080:00000010:3.0:1788880172.779755:0:201287:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 000000004cbf12f5. 00000400:00000200:0.0:1788880172.779758:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880172.779761:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880172.779767:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:0.0:1788880172.779768:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880172.779774:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.779781:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780633856/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880182 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880172.779782:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880172.779821:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=416 offset=0 replen=544 req@00000000fd6b7ee1 x1875776780633856/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880182 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880172.779826:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880172.779830:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00080000:3.0:1788880172.779846:0:201287:0:(client.c:1832:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.779857:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880172.779858:0:201287:0:(client.c:1837:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.779866:0:201287:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.779868:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880172.779870:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880172.779874:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.779875:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.779902:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.779913:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.779922:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880172.779926:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.779932:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880172.779937:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880172.779941:0:201287:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.779946:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880172.779949:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880172.779954:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880172.779961:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880172.781186:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.781218:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880172.781221:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.781224:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.781260:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880172.781275:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 416 into portal 10 MB=0x6aa0250b00800 00000400:00000200:0.0:1788880172.781282:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 416/416 into md 0x18d [1] + 0 00000800:00000001:0.0:1788880172.781288:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.781329:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880172.781331:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880172.781340:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880172.781344:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880172.781349:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780634112/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880182 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880172.781358:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=416 offset=0 replen=544 req@00000000d838cab4 x1875776780634112/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880182 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880172.781372:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880172.781379:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880172.781382:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880172.783303:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305803888, at 4305803889 00000100:00000010:2.0:1788880172.783316:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008921ad2e. 00000100:00000040:3.0:1788880172.783322:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305803888, at 4305803889 00000100:00000010:3.0:1788880172.783338:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001a3d5793. 00000100:00000040:0.0:1788880172.783413:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305803888, at 4305803889 00000100:00000010:0.0:1788880172.783420:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e83229b9. 00000020:00000001:3.0:1788880173.758825:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880173.758829:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880173.758833:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000002:00000001:2.0:1788880173.781880:0:201284:0:(mdc_request.c:2953:mdc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880173.781890:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00000001:2.0:1788880173.781893:0:201284:0:(import.c:1241:ptlrpc_connect_interpret()) Process leaving via finish (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880173.781905:0:201284:0:(import.c:1648:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00080000:2.0:1788880173.781929:0:201284:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 0000000087a6f6bb): req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880173.781947:0:201284:0:(import.c:1754:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880173.781954:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880173.781959:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d838cab4 x1875776780634112/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880182 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880173.781966:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780634112:192.168.204.111@tcp:38: 00000100:00000001:2.0:1788880173.781972:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880173.781989:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880173.781993:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d838cab4 x1875776780634112/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880182 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880173.782000:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880173.782000:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880173.782005:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880173.782019:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000009ef860f0. 02000000:00000001:2.0:1788880173.782030:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880173.782030:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000001:2.0:1788880173.782035:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880173.782036:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880173.782037:0:201287:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788880173.782039:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880173.782042:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 00000000150c5893. 00000100:00000001:3.0:1788880173.782044:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:2.0:1788880173.782046:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880173.782049:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880173.782050:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880173.782052:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d838cab4. 00000100:00000001:2.0:1788880173.782060:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:3.0:1788880173.782060:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880173.782062:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880173.782065:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880173.782066:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880173.782071:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880173.782074:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880173.782077:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880173.782080:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880173.782082:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880173.782084:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880173.782087:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880173.782088:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880173.782092:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880173.782097:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000002a6ca715. 02000000:00000001:3.0:1788880173.782102:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880173.782108:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000ea759cb. 00000400:00000010:3.0:1788880173.782113:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000100:00000200:3.0:1788880173.782118:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780634240, portal 10 00000100:00000001:3.0:1788880173.782120:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880173.782121:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000020:00000010:3.0:1788880173.782127:0:201287:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 00000000a70dc68d. 00000100:00000040:3.0:1788880173.782137:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880189 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880173.782149:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880173.782152:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880173.782154:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007839347f. 00000100:00000200:3.0:1788880173.782157:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780634240, offset 0 00000400:00000200:3.0:1788880173.782164:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880173.782176:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880173.782182:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880173.782186:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880173.782189:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880173.782191:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880173.782195:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [20:20:256:4294967295] 192.168.204.111@tcp LPNI seq info [20:20:8:4294967295] 00000400:00000200:3.0:1788880173.782205:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880173.782211:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880173.782222:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000800:00000200:3.0:1788880173.782228:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880173.782278:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880173.782281:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880173.782291:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880173.782296:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 2998 usecs (3 jiffies) 00000100:00000001:3.0:1788880173.782300:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305804891 00000100:00000010:3.0:1788880173.782303:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000015631aee. 00000100:00000001:3.0:1788880173.782311:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880173.782324:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305804891 00000100:00000010:3.0:1788880173.782327:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e62b7132. 00000100:00000001:3.0:1788880173.782331:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880173.782335:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305804891 00000100:00000010:3.0:1788880173.782337:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014c076ca. 00000100:00000001:3.0:1788880173.782341:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880173.782343:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305804891 00000100:00000010:3.0:1788880173.782345:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c451d2b9. 00000100:00000001:3.0:1788880173.782347:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880173.782348:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880173.782351:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880173.782354:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880173.782361:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880173.782364:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880173.782366:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880173.782368:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880173.782369:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880173.782371:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880173.782404:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880173.782409:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000400:00000200:0.0:1788880173.782414:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880173.782419:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880173.782424:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007839347f 00000400:00000010:0.0:1788880173.782426:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007839347f. 00000100:00000001:0.0:1788880173.782430:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880173.782433:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880189 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880173.782444:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880173.782446:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880189 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880173.782453:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880173.782455:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880173.783799:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880173.783810:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880173.783813:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880173.783815:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880173.783823:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880173.783834:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 368 into portal 10 MB=0x6aa0250b00880 00000400:00000200:0.0:1788880173.783840:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 368/368 into md 0x195 [1] + 224 00000800:00000001:0.0:1788880173.783855:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880173.783861:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880173.783863:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880173.783866:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880173.783871:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880173.783875:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880189 ref 1 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880173.783884:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=368 offset=224 replen=368 req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880189 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880173.783898:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880173.783905:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880173.783907:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880173.783935:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880173.783939:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880173.783944:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880173.783946:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880173.783953:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000ea759cb. 00000400:00000200:3.0:1788880173.783957:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000010:3.0:1788880173.783959:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:3.0:1788880173.783962:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880173.783966:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880189 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880173.783976:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880189 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880173.783983:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880173.783985:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880173.783987:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880173.783989:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880173.783991:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880173.783993:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880173.783996:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880173.784003:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=1 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880173.784018:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=1 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880173.784021:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880173.784022:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880173.784025:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880173.784029:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880189 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880173.784041:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880173.784045:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880173.784047:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880173.784051:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880173.784053:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880173.784056:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880189 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880173.784063:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880173.784065:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880173.784067:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000002:00000020:3.0:1788880173.784071:0:201287:0:(mdc_request.c:1625:mdc_statfs_interpret()) blocks=23812 free=23439 avail=20275 objects=79992 free=79737 state=0 00000100:00000040:3.0:1788880173.784082:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880189 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880173.784089:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:0:1875776780634240:192.168.204.111@tcp:41: 00000100:00000001:3.0:1788880173.784094:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880173.784096:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880173.784098:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780634240/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880189 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880173.784104:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880173.784106:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880173.784109:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000002a6ca715. 02000000:00000001:3.0:1788880173.784112:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880173.784114:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880173.784116:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880173.784118:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880173.784120:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000ea04942e. 02000000:00000001:3.0:1788880173.784123:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880173.784125:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880173.784126:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:3.0:1788880173.784130:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880173.784131:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880173.784132:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880173.784137:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880173.784138:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880173.784140:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880173.784142:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880173.786301:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305804891, at 4305804892 00000100:00000010:0.0:1788880173.786308:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000015631aee. 00000100:00000040:2.0:1788880173.786380:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305804891, at 4305804892 00000100:00000040:3.0:1788880173.786387:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305804891, at 4305804892 00000100:00000010:2.0:1788880173.786398:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014c076ca. 00000100:00000010:3.0:1788880173.786399:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c451d2b9. 00000100:00000001:2.0:1788880173.822392:0:203392:0:(client.c:2554:ptlrpc_expired_set()) Process entered 00000100:00000001:2.0:1788880173.822404:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880173.822406:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880173.822411:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000009e67569f going to sleep for 0 seconds 00000100:00000001:2.0:1788880173.822417:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880173.822422:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880173.822425:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880173.822428:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880173.822430:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880174.782412:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880174.782417:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880174.782420:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:2.0:1788880174.846344:0:203392:0:(client.c:2554:ptlrpc_expired_set()) Process entered 00000100:00000001:2.0:1788880174.846353:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880174.846355:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880174.846358:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000009e67569f going to sleep for 0 seconds 00000100:00000001:2.0:1788880174.846362:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880174.846380:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880174.846382:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880174.846384:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880174.846386:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880175.806455:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880175.806462:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880175.806467:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:2.0:1788880175.870418:0:203392:0:(client.c:2554:ptlrpc_expired_set()) Process entered 00000100:00000001:2.0:1788880175.870427:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880175.870429:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880175.870432:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000009e67569f going to sleep for 0 seconds 00000100:00000001:2.0:1788880175.870437:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880175.870439:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880175.870441:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880175.870444:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880175.870446:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880176.830395:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880176.830398:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880176.830400:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:2.0:1788880176.894346:0:203392:0:(client.c:2554:ptlrpc_expired_set()) Process entered 00000100:00000001:2.0:1788880176.894355:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880176.894356:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880176.894360:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000009e67569f going to sleep for 0 seconds 00000100:00000001:2.0:1788880176.894365:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880176.894368:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880176.894371:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880176.894373:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880176.894376:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880177.790365:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880177.790376:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880177.790395:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001d70df98. 00000020:00000040:0.0:1788880177.790402:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880177.790406:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880177.790409:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880177.790414:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880177.790421:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b899fe8d. 00000100:00000001:0.0:1788880177.790429:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880177.790439:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000001d70df98 x1875776780634496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880177.790449:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880177.790465:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880177.790473:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001d70df98] to pc [ptlrpcd_00_03+3] req@000000001d70df98 x1875776780634496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880177.790505:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:0.0:1788880177.790508:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 0 suppress 1 00000100:00080000:0.0:1788880177.790512:0:53635:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00000001:0.0:1788880177.790514:0:53635:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:0.0:1788880177.790516:0:53635:0:(import.c:65:import_set_state_nolock()) 0000000095473c20 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:0.0:1788880177.790520:0:53635:0:(import.c:485:import_select_connection()) Process entered 00000100:00000040:0.0:1788880177.790524:0:53635:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 9 to 192.168.204.111@tcp 00000100:00000001:0.0:1788880177.790528:0:53635:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:0.0:1788880177.790531:0:53635:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 10 to 192.168.204.111@tcp 00000100:00000001:0.0:1788880177.790533:0:53635:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000001:0.0:1788880177.790536:0:53635:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:0.0:1788880177.790538:0:53635:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a8869e 00000020:00000001:0.0:1788880177.790541:0:53635:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880177.790547:0:53635:0:(lustre_handles.c:147:class_handle2object()) GET export 0000000017f88dbf refcount=5 00000020:00000001:0.0:1788880177.790551:0:53635:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948786688 : -126823760764928 : ffff8ca78aca6000) 00000020:00000001:0.0:1788880177.790565:0:53635:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948786688 : -126823760764928 : ffff8ca78aca6000) 00000100:00000040:0.0:1788880177.790573:0:53635:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 9 to 192.168.204.111@tcp 00000100:00000001:0.0:1788880177.790578:0:53635:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:0.0:1788880177.790584:0:53635:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 10 to 192.168.204.111@tcp 00000100:00000001:3.0:1788880177.790584:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880177.790594:0:53635:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000001:3.0:1788880177.790595:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880177.790603:0:53635:0:(genops.c:900:class_export_put()) PUTting export 0000000017f88dbf : new refcount 4 00000100:00080000:0.0:1788880177.790610:0:53635:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff8ca787527000: import 0000000095473c20 using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000100:00000001:0.0:1788880177.790617:0:53635:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880177.790622:0:53635:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:0.0:1788880177.790630:0:53635:0:(sec.c:1628:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880177.790639:0:53635:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880177.790643:0:53635:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:0.0:1788880177.790654:0:53635:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 134217728, lost: 0. 00000008:00000001:0.0:1788880177.790659:0:53635:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880177.790661:0:53635:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880177.790666:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006b32fd51. 00000020:00000040:0.0:1788880177.790670:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=7 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880177.790677:0:53635:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 02000000:00000001:0.0:1788880177.790681:0:53635:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880177.790685:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880177.790689:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880177.790692:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880177.790699:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 0000000025f082c3. 00000100:00000001:0.0:1788880177.790704:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880177.790711:0:53635:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880177.790728:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880177.790731:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880177.790737:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006b32fd51] to pc [ptlrpcd_rcv+-1] req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880177.790767:0:53635:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880177.790772:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880177.790779:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880177.790784:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000118779b0. 00000020:00000040:0.0:1788880177.790789:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880177.790794:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880177.790797:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880177.790799:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880177.790804:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000088d6bc1. 00000100:00000001:0.0:1788880177.790810:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880177.790811:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880177.790817:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880177.790818:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880177.790820:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880177.790828:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880177.790838:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880177.790842:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880177.790845:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880177.790848:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880177.790850:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880177.790854:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880177.790857:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000118779b0] to pc [ptlrpcd_00_00+0] req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880177.790859:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880177.790868:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780634624:192.168.204.111@tcp:38: 00000100:00000001:2.0:1788880177.790876:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880177.790878:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880177.790880:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880177.790892:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880177.790894:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880177.790898:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000da1bf4ae. 00000100:00000040:0.0:1788880177.790901:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11146) 02000000:00000001:2.0:1788880177.790903:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880177.790911:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000400:00000010:2.0:1788880177.790917:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000200:2.0:1788880177.790923:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780634624, portal 10 00000100:00000001:0.0:1788880177.790926:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880177.790928:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880177.790940:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849362304 : -126822860189312 : ffff8ca7c0781180) 00000100:00000001:0.0:1788880177.790942:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880177.790948:0:201286:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000020:00000010:2.0:1788880177.790952:0:201284:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 00000000c68794ed. 00000100:00000001:0.0:1788880177.790953:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880177.790956:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880177.790961:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880177.790963:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880187 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880177.790966:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788880177.790972:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880177.790979:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880177.790980:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880177.790985:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880177.790986:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880177.790991:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:2.0:1788880177.790993:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000001:0.0:1788880177.790995:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880177.790998:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:2.0:1788880177.790998:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780634624, offset 0 00000100:00000040:0.0:1788880177.791003:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880177.791008:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880177.791017:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:2.0:1788880177.791019:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880177.791020:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880177.791024:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788880177.791025:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880177.791029:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880177.791030:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880177.791034:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880177.791036:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:2.0:1788880177.791037:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00100000:0.0:1788880177.791039:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000118779b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780634752:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880177.791043:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880177.791048:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880177.791049:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [22:22:255:4294967295] 192.168.204.111@tcp LPNI seq info [22:22:7:4294967295] 02000000:00000001:0.0:1788880177.791053:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880177.791057:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880177.791058:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 02000000:00000001:0.0:1788880177.791064:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788880177.791071:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000010:0.0:1788880177.791072:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b5c40084. 00000400:00000200:2.0:1788880177.791072:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880177.791081:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880177.791088:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000e3619b0a. 00000400:00000200:3.0:1788880177.791091:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:0.0:1788880177.791095:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e0ace8ca. 00000800:00000200:2.0:1788880177.791097:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880177.791102:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000800:00000010:2.0:1788880177.791108:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006505de7c. 00000400:00000010:3.0:1788880177.791108:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000200:0.0:1788880177.791121:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780634752, portal 10 00000800:00000200:2.0:1788880177.791121:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880177.791127:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880177.791131:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006505de7c type 1, nob 616 niov 1 nkiov 1 00000100:00000001:0.0:1788880177.791132:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880177.791135:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880177.791136:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849360512 : -126822860191104 : ffff8ca7c0780a80) 00000100:00000040:2.0:1788880177.791140:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 2230 usecs (3 jiffies) 00000100:00000001:2.0:1788880177.791143:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305808899 00000020:00000010:0.0:1788880177.791144:0:201286:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 00000000a6467e0e. 00000100:00000001:2.0:1788880177.791153:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305808899 00000100:00000001:2.0:1788880177.791156:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305808899 00000100:00000001:2.0:1788880177.791158:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305808899 00000100:00000001:2.0:1788880177.791159:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305808899 00000100:00000001:2.0:1788880177.791162:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305808899 00000100:00000001:2.0:1788880177.791164:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305808899 00000100:00000040:0.0:1788880177.791166:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880193 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880177.791167:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305808899 00000100:00000001:2.0:1788880177.791172:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880177.791177:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880177.791181:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880177.791182:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880177.791185:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880177.791188:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880177.791191:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:0.0:1788880177.791193:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009fe01c6d. 00000100:00000001:2.0:1788880177.791194:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000200:0.0:1788880177.791199:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780634752, offset 0 00000100:00000001:2.0:1788880177.791199:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880177.791202:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880177.791204:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880177.791206:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880177.791208:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880177.791216:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880177.791219:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:0.0:1788880177.791224:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880177.791263:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880177.791267:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880177.791270:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880177.791274:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [23:23:254:4294967295] 192.168.204.111@tcp LPNI seq info [23:23:6:4294967295] 00000100:00000200:3.0:1788880177.791278:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001d70df98 x1875776780634496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880193 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880177.791301:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880177.791307:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000200:0.0:1788880177.791312:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880177.791316:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001d70df98 x1875776780634496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880193 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:0.0:1788880177.791320:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 00000100:00000001:3.0:1788880177.791338:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880177.791339:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880177.791346:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880177.791352:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880177.791357:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880177.791362:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880177.791365:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 629406 usecs (630 jiffies) 00000100:00000001:0.0:1788880177.791367:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305809527 00000100:00000001:0.0:1788880177.791373:0:201286:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, simply extend its deadline from 4305808899 00000800:00000200:3.0:1788880177.791375:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880177.791377:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305809527 00000800:00000010:3.0:1788880177.791380:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006505de7c. 00000100:00000001:0.0:1788880177.791385:0:201286:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, simply extend its deadline from 4305808899 00000400:00000200:3.0:1788880177.791387:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880177.791390:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305809527 00000400:00000200:3.0:1788880177.791396:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880177.791404:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000100:00000001:0.0:1788880177.791406:0:201286:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, simply extend its deadline from 4305808899 00000400:00000010:3.0:1788880177.791409:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:0.0:1788880177.791414:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305809527 00000100:00000001:3.0:1788880177.791416:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880177.791422:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880187 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880177.791423:0:201286:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, simply extend its deadline from 4305808899 00000100:00000001:0.0:1788880177.791434:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880177.791444:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880177.791448:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880177.791450:0:201286:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880177.791454:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880187 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880177.791456:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880177.791459:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880177.791465:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880177.791472:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880177.791476:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880177.791480:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880177.791518:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880177.791520:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000400:00000200:3.0:1788880177.791526:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880177.791530:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880177.791532:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009fe01c6d 00000400:00000010:3.0:1788880177.791534:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009fe01c6d. 00000100:00000001:3.0:1788880177.791547:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880177.791552:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880193 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880177.791560:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880177.791563:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880193 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880177.791569:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880177.791570:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880177.792508:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880177.792521:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880177.792525:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880177.792527:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880177.792535:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880177.792546:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b00980 00000400:00000200:0.0:1788880177.792553:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x19d [1] + 224 00000800:00000001:0.0:1788880177.792568:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880177.792579:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880177.792581:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880177.792586:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880177.792592:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880177.792597:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001d70df98 x1875776780634496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880193 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880177.792606:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001d70df98 x1875776780634496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880193 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880177.792623:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880177.792632:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880177.792635:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880177.793047:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880177.793053:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880177.793060:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880177.793063:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880177.793068:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880177.793074:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b00a80 00000400:00000200:0.0:1788880177.793079:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1ad [1] + 224 00000800:00000001:0.0:1788880177.793083:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880177.793088:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880177.793094:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880177.793097:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880177.793099:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880177.793103:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880193 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880177.793110:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880193 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880177.793124:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880177.793128:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880177.793130:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880177.793199:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880177.793203:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880177.793205:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1788880177.793206:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880177.793209:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880177.793210:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000e3619b0a. 00000800:00000001:0.0:1788880177.793213:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880177.793214:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e0ace8ca 00000400:00000010:3.0:1788880177.793217:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e0ace8ca. 00000800:00000001:0.0:1788880177.793218:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880177.793220:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:0.0:1788880177.793222:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880177.793225:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880193 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880177.793231:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880177.793254:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 416 into portal 10 MB=0x6aa0250b00a00 00000400:00000200:0.0:1788880177.793259:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 416/416 into md 0x1a5 [1] + 0 00000800:00000001:0.0:1788880177.793262:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880177.793267:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880177.793270:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880177.793274:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880177.793277:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880177.793281:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880187 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880177.793309:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880193 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880177.793310:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=416 offset=0 replen=544 req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880187 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880177.793327:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880177.793329:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880177.793332:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880177.793333:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880177.793334:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880177.793336:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880177.793338:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880177.793342:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=5 v=5 (5 0 0 0) 00000100:00000001:0.0:1788880177.793343:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:3.0:1788880177.793348:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=5 v=5 (1 0 0 0) 00000800:00000001:0.0:1788880177.793350:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880177.793351:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880177.793354:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880177.793355:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880177.793357:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880177.793362:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880193 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880177.793378:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880177.793380:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880177.793383:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880177.793386:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880177.793390:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:2.0:1788880177.793395:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00010000:00000001:3.0:1788880177.793396:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880177.793398:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:3.0:1788880177.793401:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880177.793403:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1788880177.793405:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880177.793408:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000200:2.0:1788880177.793410:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880187 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880177.793411:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880177.793419:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880193 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880177.793430:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880187 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880177.793432:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880177.793436:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880177.793438:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880177.793444:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880177.793445:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880193 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880177.793446:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880177.793449:0:201284:0:(client.c:1580:after_reply()) Process entered 00000100:00100000:3.0:1788880177.793456:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000118779b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780634752:192.168.204.111@tcp:400: 02000000:00000001:2.0:1788880177.793457:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880177.793460:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880177.793462:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880177.793465:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880177.793467:0:201286:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880177.793468:0:201284:0:(import.c:2149:obd_at_measure()) add 0 to 00000000468b166e time=1788880177 v=5 (0 0 0 0) 00000100:00000001:3.0:1788880177.793471:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00001000:2.0:1788880177.793472:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=5 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880177.793476:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880177.793476:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880177.793478:0:201284:0:(client.c:1493:ptlrpc_check_status()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000100:00000001:3.0:1788880177.793480:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880177.793481:0:201284:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880177.793483:0:201284:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880177.793486:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880177.793487:0:201284:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880177.793488:0:201284:0:(client.c:1784:after_reply()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000100:00000040:3.0:1788880177.793493:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000118779b0 x1875776780634752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880193 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880177.793494:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880187 ref 1 fl Rpc:RNQU/200/0 rc -11/-11 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880177.793508:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000100:00000001:2.0:1788880177.793513:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880177.793515:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880177.793516:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880177.793517:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:2.0:1788880177.793521:0:201284:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 02000000:00000001:3.0:1788880177.793521:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00080000:2.0:1788880177.793524:0:201284:0:(import.c:65:import_set_state_nolock()) 0000000095473c20 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 02000000:00000010:3.0:1788880177.793525:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b5c40084. 02000000:00000001:3.0:1788880177.793531:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00080000:2.0:1788880177.793532:0:201284:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.111@tcp failed (-11) 00000020:00000001:3.0:1788880177.793534:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880177.793536:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000020:00000040:3.0:1788880177.793537:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880177.793543:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000040:2.0:1788880177.793544:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880187 ref 1 fl Interpret:RNQU/200/0 rc -11/-11 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880177.793546:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000088d6bc1. 02000000:00000001:3.0:1788880177.793552:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880177.793555:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00100000:2.0:1788880177.793557:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780634624:192.168.204.111@tcp:38: 00000100:00000001:2.0:1788880177.793561:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:3.0:1788880177.793563:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000118779b0. 00000100:00000001:2.0:1788880177.793565:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880177.793569:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:2.0:1788880177.793570:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006b32fd51 x1875776780634624/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880187 ref 1 fl Complete:RNQU/200/0 rc -11/-11 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880177.793572:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880177.793576:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880177.793579:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880177.793581:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880177.793584:0:201286:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880177.793588:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880177.793589:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880177.793589:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880177.793591:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880177.793592:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880177.793597:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880177.793599:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000da1bf4ae. 02000000:00000001:2.0:1788880177.793604:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880177.793605:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880177.793607:0:201284:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=6 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880177.793609:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880177.793610:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 0000000025f082c3. 02000000:00000001:2.0:1788880177.793613:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880177.793614:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880177.793615:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006b32fd51. 00000100:00000001:2.0:1788880177.793618:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880177.793619:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880177.793621:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880177.793625:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880177.793627:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880177.793629:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880177.793630:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880177.854496:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880177.854499:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880177.854501:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:2.0:1788880177.918320:0:203392:0:(client.c:2554:ptlrpc_expired_set()) Process entered 00000100:00000001:2.0:1788880177.918326:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880177.918327:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880177.918330:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000009e67569f going to sleep for 0 seconds 00000100:00000001:2.0:1788880177.918335:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880177.918338:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880177.918339:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880177.918342:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880177.918345:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880178.430350:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305809527, at 4305809536 00000100:00000040:2.0:1788880178.430350:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305809527, at 4305809536 00000100:00000040:3.0:1788880178.430350:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305809527, at 4305809536 00000100:00000010:3.0:1788880178.430369:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000010:0.0:1788880178.430370:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000010:2.0:1788880178.430386:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000001:3.0:1788880178.814334:0:201286:0:(client.c:2554:ptlrpc_expired_set()) Process entered 00000100:00000001:3.0:1788880178.814344:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880178.814345:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880178.814348:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880178.814350:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880178.814353:0:201286:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880178.814355:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880178.814365:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880178.814368:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880178.814371:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880178.878392:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880178.878395:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880178.878397:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:2.0:1788880178.942409:0:203392:0:(client.c:2554:ptlrpc_expired_set()) Process entered 00000100:00000001:2.0:1788880178.942417:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880178.942419:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880178.942424:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000009e67569f going to sleep for 0 seconds 00000100:00000001:2.0:1788880178.942429:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880178.942432:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880178.942435:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880178.942437:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880178.942440:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880179.902386:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880179.902391:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880179.902393:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:2.0:1788880179.966343:0:203392:0:(client.c:2554:ptlrpc_expired_set()) Process entered 00000100:00000001:2.0:1788880179.966351:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880179.966353:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880179.966357:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000009e67569f going to sleep for 0 seconds 00000100:00000001:2.0:1788880179.966362:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880179.966365:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880179.966368:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880179.966371:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880179.966374:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:3.0:1788880180.350391:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000001:3.0:1788880180.350400:0:203401:0:(mgc_request.c:120:config_log_put()) Process leaving 00000020:00000001:0.0:1788880180.926570:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880180.926575:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880180.926578:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:2.0:1788880180.990353:0:203392:0:(client.c:2554:ptlrpc_expired_set()) Process entered 00000100:00000001:2.0:1788880180.990361:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880180.990363:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880180.990368:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000009e67569f going to sleep for 0 seconds 00000100:00000001:2.0:1788880180.990373:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880180.990377:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880180.990382:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880180.990384:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880180.990386:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880181.950438:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880181.950442:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880181.950445:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:2.0:1788880182.014368:0:203392:0:(client.c:2554:ptlrpc_expired_set()) Process entered 00000100:00000001:2.0:1788880182.014377:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880182.014380:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880182.014384:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000009e67569f going to sleep for 0 seconds 00000100:00000001:2.0:1788880182.014390:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880182.014395:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.014398:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880182.014401:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.014404:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.910531:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880182.910532:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:2.0:1788880182.910535:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:3.0:1788880182.910537:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880182.910538:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880182.910543:0:201286:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880182.910547:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880182.910557:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880182.910562:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.910567:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.910728:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880182.910733:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880182.910736:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880182.910743:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880182.910750:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880182.910751:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880182.910753:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880182.910755:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880182.910757:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880182.910759:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006a9c2e02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780635008:192.168.204.111@tcp:38: 00000100:00000001:3.0:1788880182.910762:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880182.910767:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880182.910770:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880182.910773:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880182.910777:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000002a6ca715. 02000000:00000001:3.0:1788880182.910780:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880182.910788:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000ea759cb. 00000100:00000001:0.0:1788880182.910795:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880182.910796:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000100:00000001:0.0:1788880182.910805:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880182.910805:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780635008, portal 10 00000100:00000001:3.0:1788880182.910812:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880182.910816:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938058624 : -126823771492992 : ffff8ca78a26ad80) 00000020:00000010:3.0:1788880182.910828:0:201284:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 0000000026c56e93. 00000100:00000040:3.0:1788880182.910854:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880192 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880182.910870:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880182.910878:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880182.910882:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8336eb2. 00000100:00000200:3.0:1788880182.910885:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780635008, offset 0 00000100:00000001:2.0:1788880182.910886:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880182.910889:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.910892:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880182.910894:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880182.910896:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880182.910897:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880182.910902:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000dc99d6ba x1875776780634880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.910911:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:3.0:1788880182.910915:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880182.910917:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.910921:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:3.0:1788880182.910921:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:2.0:1788880182.910923:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880182.910927:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880182.910928:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00100000:2.0:1788880182.910932:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000dc99d6ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780634880:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880182.910934:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880182.910939:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1788880182.910940:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:2.0:1788880182.910942:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880182.910945:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.910947:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788880182.910948:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [24:24:256:4294967295] 192.168.204.111@tcp LPNI seq info [24:24:8:4294967295] 02000000:00000010:2.0:1788880182.910951:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008f0935d3. 02000000:00000001:2.0:1788880182.910954:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880182.910958:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000400:00000010:2.0:1788880182.910962:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000400:00000200:3.0:1788880182.910967:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:2.0:1788880182.910971:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780634880, portal 25 00000800:00000200:3.0:1788880182.910974:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880182.910976:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880182.910978:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938052352 : -126823771499264 : ffff8ca78a269500) 00000800:00000010:3.0:1788880182.910980:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000100:00000040:2.0:1788880182.910984:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000dc99d6ba x1875776780634880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880198 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880182.910991:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880182.910998:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880182.911002:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880182.911005:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:2.0:1788880182.911007:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000078d4a4ff. 00000800:00000200:3.0:1788880182.911010:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 616 niov 1 nkiov 1 00000100:00000200:2.0:1788880182.911011:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780634880, offset 0 00000400:00000200:2.0:1788880182.911021:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880182.911026:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880182.911028:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880182.911033:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880182.911033:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 2807 usecs (3 jiffies) 00000400:00000200:2.0:1788880182.911038:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880182.911038:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305814019 00000400:00000200:2.0:1788880182.911041:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880182.911045:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:3.0:1788880182.911045:0:201284:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c451d2b9. 00000400:00000200:2.0:1788880182.911050:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [26:26:254:4294967295] 192.168.204.111@tcp LPNI seq info [26:26:6:4294967295] 00000100:00000001:3.0:1788880182.911054:0:201284:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000400:00000200:2.0:1788880182.911060:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880182.911062:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305814019 00000800:00000200:2.0:1788880182.911066:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788880182.911070:0:201284:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000625b9f8b. 00000800:00000010:2.0:1788880182.911071:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d24baaa0. 00000100:00000001:3.0:1788880182.911074:0:201284:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880182.911079:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305814019 00000800:00000200:2.0:1788880182.911083:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:3.0:1788880182.911086:0:201284:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000dc3cc638. 00000100:00000001:3.0:1788880182.911091:0:201284:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000800:00000200:0.0:1788880182.911093:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880182.911096:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305814019 00000800:00000200:2.0:1788880182.911099:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000010:0.0:1788880182.911100:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000100:00000010:3.0:1788880182.911103:0:201284:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001f846689. 00000800:00000200:2.0:1788880182.911107:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d24baaa0 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880182.911110:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880182.911110:0:201284:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880182.911112:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.911116:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305814646 00000400:00000200:0.0:1788880182.911120:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880182.911121:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880182.911122:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305814437 00000100:00000001:3.0:1788880182.911128:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880182.911130:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8336eb2 00000100:00000001:2.0:1788880182.911130:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305814646 00000100:00000001:2.0:1788880182.911135:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305814437 00000400:00000010:0.0:1788880182.911136:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8336eb2. 00000100:00000001:2.0:1788880182.911140:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305814646 00000100:00000001:3.0:1788880182.911141:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880182.911145:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880182.911145:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305814437 00000100:00000001:3.0:1788880182.911146:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.911148:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305814646 00000100:00000001:2.0:1788880182.911152:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305814437 00000100:00000001:3.0:1788880182.911153:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:0.0:1788880182.911154:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880192 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.911156:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880182.911157:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:2.0:1788880182.911161:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.911164:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880182.911168:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.911168:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880182.911172:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.911173:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880182.911175:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880182.911176:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880182.911181:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880192 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.911189:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880182.911191:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880182.911204:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880182.911206:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:0.0:1788880182.911208:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.911211:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880182.911213:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000007206624 00000400:00000010:0.0:1788880182.911222:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000007206624. 00000100:00000001:0.0:1788880182.911225:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880182.911228:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780635136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880198 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.911253:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880182.911256:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780635136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880198 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.911262:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880182.911263:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880182.911321:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880182.911323:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d24baaa0. 00000400:00000200:0.0:1788880182.911326:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.911329:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880182.911331:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000078d4a4ff 00000400:00000010:0.0:1788880182.911333:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000078d4a4ff. 00000100:00000001:0.0:1788880182.911335:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880182.911338:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000dc99d6ba x1875776780634880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880198 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.911345:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880182.911347:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000dc99d6ba x1875776780634880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880198 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.911352:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880182.911353:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880182.912881:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.912892:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880182.912896:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.912898:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.912905:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880182.912917:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b00c00 00000400:00000200:0.0:1788880182.912923:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1bd [1] + 224 00000800:00000001:0.0:1788880182.912929:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.912932:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.912934:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.912945:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880182.912950:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880182.912953:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780635136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880198 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880182.912962:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000077a1dd13 x1875776780635136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880198 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.912978:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880182.912996:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.913001:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880182.913003:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.913005:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.913009:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880182.913016:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b00b00 00000400:00000200:0.0:1788880182.913022:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1c5 [1] + 224 00000800:00000001:0.0:1788880182.913027:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.913036:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.913041:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.913051:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880182.913059:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880182.913067:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000dc99d6ba x1875776780634880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880198 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880182.913093:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000dc99d6ba x1875776780634880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880198 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.913121:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880182.913128:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.913133:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.913165:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880182.913169:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880182.913174:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880182.913179:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880182.913183:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:2.0:1788880182.913188:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788880182.913190:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:2.0:1788880182.913194:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880182.913202:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000dc99d6ba x1875776780634880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880198 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880182.913220:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000dc99d6ba x1875776780634880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880198 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.913257:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880182.913259:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.913262:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880182.913270:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880182.913273:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880182.913275:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.913281:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880182.913292:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=10 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880182.913298:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=10 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880182.913304:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880182.913308:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880182.913312:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880182.913315:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.913319:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.913327:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000dc99d6ba x1875776780634880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880198 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.913337:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880182.913341:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880182.913343:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880182.913348:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000dc99d6ba x1875776780634880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880198 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880182.913360:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000dc99d6ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780634880:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880182.913367:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.913371:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880182.913377:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000dc99d6ba x1875776780634880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880198 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.913386:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880182.913388:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880182.913390:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008f0935d3. 02000000:00000001:2.0:1788880182.913393:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880182.913395:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880182.913396:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880182.913398:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880182.913399:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 02000000:00000001:2.0:1788880182.913401:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880182.913402:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880182.913403:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000dc99d6ba. 00000100:00000001:2.0:1788880182.913410:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880182.913411:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.913412:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.913415:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880182.913416:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.913417:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880182.913418:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880182.913477:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.913483:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880182.913485:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.913487:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.913492:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880182.913499:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 416 into portal 10 MB=0x6aa0250b00b80 00000400:00000200:0.0:1788880182.913509:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 416/416 into md 0x1b5 [1] + 0 00000800:00000001:0.0:1788880182.913514:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.913519:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.913521:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.913525:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880182.913528:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880182.913532:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880192 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880182.913545:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=416 offset=0 replen=544 req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880192 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.913562:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880182.913566:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.913569:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880182.913628:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880182.913631:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880182.913635:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880182.913642:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880182.913647:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000ea759cb. 00000400:00000200:3.0:1788880182.913650:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000010:3.0:1788880182.913652:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:3.0:1788880182.913684:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880182.913691:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880192 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880182.913700:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880192 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880182.913706:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880182.913707:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880182.913709:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880182.913711:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880182.913712:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880182.913714:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880182.913715:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880182.913720:0:201284:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=1788880182 v=5 (0 0 0 0) 00000100:00001000:3.0:1788880182.913723:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=10 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880182.913725:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880182.913726:0:201284:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880182.913728:0:201284:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880182.913730:0:201284:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880192 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880182.913737:0:201284:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.913745:0:201284:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880182.913747:0:201284:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880182.913749:0:201284:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880182.913750:0:201284:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880182.913753:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880192 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880182.913760:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880182.913762:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880182.913763:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880182.913765:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00080000:3.0:1788880182.913770:0:201284:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff8ca787527000: connect to target with instance 3 00000020:00000001:3.0:1788880182.913772:0:201284:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:3.0:1788880182.913774:0:201284:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a8869e 00000020:00000001:3.0:1788880182.913776:0:201284:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880182.913779:0:201284:0:(lustre_handles.c:147:class_handle2object()) GET export 0000000017f88dbf refcount=5 00000020:00000001:3.0:1788880182.913781:0:201284:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948786688 : -126823760764928 : ffff8ca78aca6000) 00000020:00000001:3.0:1788880182.913784:0:201284:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948786688 : -126823760764928 : ffff8ca78aca6000) 00000100:00001000:3.0:1788880182.913787:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=1788880182 v=0 (0 0 0 0) 00000100:00001000:3.0:1788880182.913790:0:201284:0:(import.c:2215:obd_at_measure()) AT 0000000092db0829 change: old=0 new=5 delta=5 (val=1) hist 1 0 0 0 00000100:00000040:3.0:1788880182.913794:0:201284:0:(obd_cksum.h:77:obd_cksum_types_supported_client()) Crypto hash speed: crc 1651, crc32c 9963, adler 1371 00000100:00080000:3.0:1788880182.913798:0:201284:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-MDT0000-mdc-ffff8ca787527000: Resetting ns_connect_flags to server flags: 0xae7a7e7be344d3b8 00000020:00000040:3.0:1788880182.913801:0:201284:0:(genops.c:900:class_export_put()) PUTting export 0000000017f88dbf : new refcount 4 00000100:00000001:3.0:1788880182.913803:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 00000002:00000001:3.0:1788880182.913807:0:201284:0:(mdc_request.c:2879:mdc_import_event()) Process entered 00000008:00000020:3.0:1788880182.913810:0:201284:0:(osc_request.c:1062:osc_init_grant()) lustre-MDT0000-mdc-ffff8ca787527000, setting cl_avail_grant: 2146304 cl_lost_grant: 0. chunk bits: 12 cl_max_extent_pages: 16384 00000002:00000001:3.0:1788880182.913814:0:201284:0:(mdc_request.c:3107:mdc_init_ea_size()) Process entered 00000002:00000001:3.0:1788880182.913816:0:201284:0:(mdc_request.c:3114:mdc_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880182.913818:0:201284:0:(obd_class.h:1243:obd_notify()) Process entered 00800000:00000001:3.0:1788880182.913821:0:201284:0:(lmv_obd.c:188:lmv_notify()) Process entered 00000080:00000001:3.0:1788880182.913824:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:3.0:1788880182.913826:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:3.0:1788880182.913828:0:201284:0:(lmv_obd.c:223:lmv_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880182.913830:0:201284:0:(obd_class.h:1256:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880182.913952:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:3.0:1788880182.913953:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880182.913955:0:201284:0:(mdc_request.c:2953:mdc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.913957:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00080000:3.0:1788880182.913958:0:201284:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-MDT0000_UUID 00000100:00080000:3.0:1788880182.913962:0:201284:0:(import.c:65:import_set_state_nolock()) 0000000095473c20 lustre-MDT0000_UUID: changing import state from CONNECTING to FULL 00000100:00000001:3.0:1788880182.913968:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 00000002:00000001:3.0:1788880182.913969:0:201284:0:(mdc_request.c:2879:mdc_import_event()) Process entered 00000002:00000001:3.0:1788880182.913970:0:201284:0:(obd_class.h:1243:obd_notify()) Process entered 00800000:00000001:3.0:1788880182.913971:0:201284:0:(lmv_obd.c:188:lmv_notify()) Process entered 00800000:00000001:3.0:1788880182.913973:0:201284:0:(lmv_obd.c:82:lmv_set_mdc_active()) Process entered 00800000:00000040:3.0:1788880182.913974:0:201284:0:(lmv_obd.c:85:lmv_set_mdc_active()) Searching in lmv 00000000b999beb9 for uuid lustre-MDT0000_UUID (activate=1) 00800000:00000040:3.0:1788880182.913977:0:201284:0:(lmv_obd.c:91:lmv_set_mdc_active()) Target idx 0 is lustre-MDT0000_UUID conn 0x805d548a36a8869e 00800000:00000040:3.0:1788880182.913980:0:201284:0:(lmv_obd.c:106:lmv_set_mdc_active()) Found OBD lustre-MDT0000-mdc-ffff8ca787527000=ea27006f-52a8-4007-b69a-dcac8279445a device 3 (000000009cb5873b) type mdc at LMV idx 0 00800000:00000040:3.0:1788880182.913986:0:201284:0:(lmv_obd.c:111:lmv_set_mdc_active()) OBD 000000009cb5873b already active! 00800000:00000001:3.0:1788880182.913987:0:201284:0:(lmv_obd.c:112:lmv_set_mdc_active()) Process leaving via out_lmv_lock (rc=0 : 0 : 0x0) 00000080:00000001:3.0:1788880182.913989:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:3.0:1788880182.913991:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:3.0:1788880182.913992:0:201284:0:(lmv_obd.c:223:lmv_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880182.913994:0:201284:0:(obd_class.h:1256:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880182.913995:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:3.0:1788880182.913996:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880182.913998:0:201284:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=8 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000002:00000001:3.0:1788880182.914179:0:201284:0:(mdc_request.c:2953:mdc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.914181:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00000001:3.0:1788880182.914183:0:201284:0:(import.c:1241:ptlrpc_connect_interpret()) Process leaving via finish (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880182.914186:0:201284:0:(import.c:1648:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00080000:3.0:1788880182.914192:0:201284:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000009e67569f): req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788880182.914207:0:201284:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 00000000addf16a4): req@00000000175072aa x1875776780633984/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000002:00000001:0.0:1788880182.914209:0:203414:0:(mdc_request.c:2638:mdc_kuc_reregister_thread()) Process entered 00000020:00000001:0.0:1788880182.914213:0:203414:0:(kernelcomm.c:1325:libcfs_kkuc_group_foreach()) Process entered 00000020:00000001:0.0:1788880182.914216:0:203414:0:(kernelcomm.c:1339:libcfs_kkuc_group_foreach()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880182.914219:0:203414:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880182.914220:0:203414:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=7 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880182.914222:0:203414:0:(genops.c:1251:class_import_put()) Process leaving 00000002:00000001:0.0:1788880182.914223:0:203414:0:(mdc_request.c:2648:mdc_kuc_reregister_thread()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.914225:0:201284:0:(import.c:1754:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.914230:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.914255:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880182.914260:0:203392:0:(client.c:2980:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880182.914265:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880192 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.914267:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880182.914272:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880182.914284:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006a9c2e02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780635008:192.168.204.111@tcp:38: 02000000:00000001:2.0:1788880182.914289:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880182.914291:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880182.914293:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880182.914296:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:2.0:1788880182.914300:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880182.914303:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006a9c2e02 x1875776780635008/t0(0) o38->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880192 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.914305:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880182.914311:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880182.914316:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.914317:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880182.914320:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:2.0:1788880182.914321:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880182.914324:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000002a6ca715. 02000000:00000010:2.0:1788880182.914327:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000150c5893. 02000000:00000001:3.0:1788880182.914329:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:2.0:1788880182.914332:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880182.914333:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880182.914336:0:201284:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=6 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000010:2.0:1788880182.914337:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000020:00000001:3.0:1788880182.914342:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000010:2.0:1788880182.914345:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 02000000:00000010:3.0:1788880182.914346:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 000000003c073345. 00000100:00000200:2.0:1788880182.914352:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780634368, portal 10 02000000:00000001:3.0:1788880182.914352:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880182.914355:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788880182.914359:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000010:3.0:1788880182.914359:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006a9c2e02. 00000100:00000001:2.0:1788880182.914363:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000100:00000001:3.0:1788880182.914366:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880182.914369:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880182.914373:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880182.914374:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880182.914380:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880182.914384:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.914389:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880182.914390:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880182.914392:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.914396:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880182.914399:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a3b2cf4a. 00000100:00000200:2.0:1788880182.914402:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780634368, offset 0 00000400:00000200:2.0:1788880182.914409:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880182.914447:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880182.914454:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880182.914459:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880182.914462:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880182.914472:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880182.914478:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [28:28:255:4294967295] 192.168.204.111@tcp LPNI seq info [28:28:7:4294967295] 00000400:00000200:2.0:1788880182.914489:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880182.914492:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880182.914494:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b11601fe. 00000800:00000200:2.0:1788880182.914500:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880182.914508:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880182.914513:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b11601fe type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880182.914517:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.914522:0:203392:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 3130 usecs (4 jiffies) 00000100:00000001:2.0:1788880182.914526:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305814024 00000100:00000001:2.0:1788880182.914530:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305814646 00000800:00000200:0.0:1788880182.914533:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880182.914534:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305814024 00000800:00000010:0.0:1788880182.914538:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000100:00000001:2.0:1788880182.914541:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305814646 00000400:00000200:0.0:1788880182.914543:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880182.914547:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305814024 00000100:00000001:2.0:1788880182.914551:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305814646 00000400:00000200:0.0:1788880182.914553:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880182.914557:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305814024 00000400:00000200:0.0:1788880182.914562:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000100:00000001:2.0:1788880182.914564:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305814646 00000400:00000010:0.0:1788880182.914565:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:2.0:1788880182.914570:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880182.914571:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880182.914576:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880182.914578:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780633984/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.914582:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880182.914586:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:2.0:1788880182.914593:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 000000009e67569f going to sleep for 11 seconds 00000100:00000001:2.0:1788880182.914598:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880182.914600:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880182.914601:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880182.914606:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780633984/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.914627:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880182.914629:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880182.914666:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880182.914668:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b11601fe. 00000400:00000200:0.0:1788880182.914670:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.914674:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880182.914677:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a3b2cf4a 00000400:00000010:0.0:1788880182.914678:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a3b2cf4a. 00000100:00000001:0.0:1788880182.914680:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880182.914683:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.914689:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880182.914691:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.914697:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880182.914699:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880182.915880:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.915887:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880182.915889:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.915890:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.915895:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880182.915901:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 368 into portal 10 MB=0x6aa0250b00780 00000400:00000200:0.0:1788880182.915907:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 368/368 into md 0x1cd [1] + 224 00000800:00000001:0.0:1788880182.915911:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.915916:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.915918:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.915921:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880182.915924:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880182.915928:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780633984/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 1 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880182.915937:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=368 offset=224 replen=368 req@00000000175072aa x1875776780633984/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.915958:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880182.915965:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.915968:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880182.916054:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.916062:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880182.916067:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.916070:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.916083:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880182.916095:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 368 into portal 10 MB=0x6aa0250b00900 00000400:00000200:0.0:1788880182.916106:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 368/368 into md 0x1d1 [1] + 224 00000800:00000001:0.0:1788880182.916114:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.916120:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.916124:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.916132:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880182.916136:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880182.916141:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880182.916156:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=368 offset=224 replen=368 req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.916174:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880182.916178:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.916180:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.916212:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880182.916217:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880182.916219:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880182.916220:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:2.0:1788880182.916230:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788880182.916231:0:203392:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:2.0:1788880182.916257:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880182.916261:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880182.916270:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.916277:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880182.916278:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.916280:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880182.916282:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880182.916283:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880182.916285:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.916287:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880182.916290:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=0 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880182.916293:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=0 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880182.916295:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880182.916296:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880182.916298:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880182.916300:0:203392:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880182.916307:0:203392:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.916309:0:203392:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880182.916310:0:203392:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880182.916311:0:203392:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880182.916312:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.916315:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.916321:0:203392:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880182.916323:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880182.916324:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880182.916326:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880182.916343:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job mount.lustre:ea27006f-52a8-4007-b69a-dcac8279445a:203392:1875776780634368:192.168.204.111@tcp:41: 00000100:00000001:2.0:1788880182.916347:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.916349:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.916351:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:2.0:1788880182.916352:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880182.916354:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.916360:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880182.916362:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 000000009e67569f. 00000100:00000001:2.0:1788880182.916365:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:2.0:1788880182.916366:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880182.916370:0:203392:0:(mdc_request.c:1719:mdc_statfs()) Process leaving 00000100:00000001:2.0:1788880182.916371:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880182.916373:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780634368/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880198 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.916378:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880182.916380:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880182.916381:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000150c5893. 02000000:00000001:2.0:1788880182.916384:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880182.916385:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880182.916386:0:203392:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880182.916388:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880182.916389:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000002c5b8fef. 02000000:00000001:2.0:1788880182.916391:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880182.916392:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880182.916393:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:2.0:1788880182.916396:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880182.916397:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788880182.916398:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880182.916399:0:203392:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880182.916400:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 00800000:00000004:2.0:1788880182.916402:0:203392:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff8ca787527000: update 000000009485984e cache blocks 20244/23812 objects 79718/79992 00800000:00000001:2.0:1788880182.916407:0:203392:0:(obd_class.h:1136:obd_statfs()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880182.916419:0:203392:0:(lmv_obd.c:1487:lmv_statfs()) Process leaving via out_free_temp (rc=0 : 0 : 0x0) 00800000:00000010:2.0:1788880182.916422:0:203392:0:(lmv_obd.c:1511:lmv_statfs()) kfreed 'temp': 144 at 00000000126d8278. 00800000:00000001:2.0:1788880182.916425:0:203392:0:(lmv_obd.c:1512:lmv_statfs()) Process leaving (rc=0 : 0 : 0) 00000080:00000004:2.0:1788880182.916427:0:203392:0:(obd_class.h:1117:obd_statfs()) lustre-clilmv-ffff8ca787527000: update 00000000b239d812 cache blocks 20244/23812 objects 79718/79992 00000080:00000001:2.0:1788880182.916430:0:203392:0:(obd_class.h:1136:obd_statfs()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880182.916433:0:203392:0:(lmv_obd.c:1560:lmv_get_root()) Process entered 00000002:00000001:2.0:1788880182.916436:0:203392:0:(mdc_request.c:89:mdc_get_root()) Process entered 00000100:00000010:2.0:1788880182.916439:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880182.916441:0:203392:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880182.916444:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880182.916447:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880182.916449:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880182.916454:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000a581a743. 00000100:00000001:2.0:1788880182.916458:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.916463:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:2.0:1788880182.916465:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:2.0:1788880182.916468:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000126d8278. 00000100:00000001:2.0:1788880182.916470:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249931839360 : -126823777712256 : ffff8ca789c7c780) 00000100:00000001:2.0:1788880182.916472:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880182.916473:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000100:00000001:2.0:1788880182.916477:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880182.916479:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.916480:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:2.0:1788880182.916482:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880182.916485:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 0 ref 2 fl New:QU/209/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.916491:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880182.916493:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.916495:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880182.916497:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880182.916499:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880182.916501:0:203392:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job mount.lustre:ea27006f-52a8-4007-b69a-dcac8279445a:203392:1875776780635264:192.168.204.111@tcp:40: 00000100:00000001:2.0:1788880182.916504:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880182.916507:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880182.916512:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.916514:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880182.916516:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000150c5893. 02000000:00000001:2.0:1788880182.916518:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880182.916521:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000400:00000010:2.0:1788880182.916523:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000200:2.0:1788880182.916527:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780635264, portal 10 00000100:00000001:2.0:1788880182.916528:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880182.916529:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000100:00000040:2.0:1788880182.916533:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=209 req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 3 fl Rpc:r/209/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.916539:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880182.916542:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880182.916543:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ac9b6c42. 00000100:00000200:2.0:1788880182.916545:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 440 bytes to portal 12, xid 1875776780635264, offset 0 00000400:00000200:2.0:1788880182.916549:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880182.916556:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880182.916560:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880182.916563:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880182.916565:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880182.916568:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880182.916571:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [29:29:256:4294967295] 192.168.204.111@tcp LPNI seq info [29:29:8:4294967295] 00000400:00000200:2.0:1788880182.916578:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880182.916583:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 440 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880182.916586:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000002c5b8fef. 00000800:00000200:2.0:1788880182.916590:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880182.916595:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880182.916597:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000002c5b8fef type 1, nob 536 niov 1 nkiov 1 00000100:00000001:2.0:1788880182.916611:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.916614:0:203392:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 5070396 usecs (5071 jiffies) 00000100:00000001:2.0:1788880182.916616:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305819093 00000100:00000001:2.0:1788880182.916627:0:203392:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, simply extend its deadline from 4305814646 00000100:00000001:2.0:1788880182.916632:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305819093 00000100:00000001:2.0:1788880182.916634:0:203392:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, simply extend its deadline from 4305814646 00000100:00000001:2.0:1788880182.916637:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305819093 00000100:00000001:2.0:1788880182.916642:0:203392:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, simply extend its deadline from 4305814646 00000100:00000001:2.0:1788880182.916645:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305819093 00000100:00000001:2.0:1788880182.916649:0:203392:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, simply extend its deadline from 4305814646 00000100:00000001:2.0:1788880182.916651:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880182.916654:0:203392:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.916655:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880182.916657:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:2.0:1788880182.916658:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 00000000126d8278 going to sleep for 11 seconds 00000100:00000001:2.0:1788880182.916661:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:0.0:1788880182.916662:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880182.916663:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880182.916666:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000002c5b8fef. 00000400:00000200:0.0:1788880182.916670:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.916673:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880182.916675:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ac9b6c42 00000400:00000010:0.0:1788880182.916677:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ac9b6c42. 00000100:00000001:0.0:1788880182.916679:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880182.916682:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 3 fl Rpc:r/209/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.916695:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880182.916697:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 3 fl Rpc:Qr/209/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.916703:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880182.916705:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880182.917461:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.917468:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880182.917470:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.917473:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.917477:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880182.917483:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 448 into portal 10 MB=0x6aa0250b00c80 00000400:00000200:0.0:1788880182.917486:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 448/448 into md 0x1dd [1] + 224 00000800:00000001:0.0:1788880182.917489:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.917492:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.917493:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.917495:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880182.917497:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880182.917500:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 2 fl Rpc:Qr/209/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880182.917505:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=209 mlen=448 offset=224 replen=448 req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQ/209/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.917515:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880182.917519:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.917520:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.917546:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880182.917551:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880182.917552:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880182.917553:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:2.0:1788880182.917556:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788880182.917558:0:203392:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:2.0:1788880182.917561:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880182.917565:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQ/209/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880182.917573:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/209/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.917579:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880182.917581:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.917582:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880182.917584:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880182.917585:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880182.917587:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.917594:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880182.917598:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=0 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880182.917600:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=0 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880182.917603:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880182.917604:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880182.917605:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880182.917608:0:203392:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/209/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880182.917615:0:203392:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.917617:0:203392:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880182.917618:0:203392:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880182.917620:0:203392:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880182.917621:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.917623:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/209/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.917629:0:203392:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880182.917631:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880182.917632:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880182.917635:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 2 fl Interpret:RQU/209/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880182.917643:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job mount.lustre:ea27006f-52a8-4007-b69a-dcac8279445a:203392:1875776780635264:192.168.204.111@tcp:40: 00000100:00000001:2.0:1788880182.917647:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.917649:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.917651:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:2.0:1788880182.917652:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880182.917654:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 2 fl Complete:RQU/209/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.917660:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880182.917662:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000126d8278. 00000100:00000001:2.0:1788880182.917664:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:2.0:1788880182.917665:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000002:00000200:2.0:1788880182.917672:0:203392:0:(mdc_request.c:134:mdc_get_root()) root fid=[0x200000007:0x1:0x0], last_committed=0 00000002:00000001:2.0:1788880182.917675:0:203392:0:(mdc_request.c:135:mdc_get_root()) Process leaving 00000100:00000001:2.0:1788880182.917676:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880182.917678:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780635264/t0(0) o40->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 440/448 e 0 to 0 dl 1788880198 ref 1 fl Complete:RQU/209/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.917684:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880182.917685:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880182.917687:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000150c5893. 02000000:00000001:2.0:1788880182.917689:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880182.917690:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880182.917691:0:203392:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880182.917693:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880182.917702:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000a581a743. 02000000:00000001:2.0:1788880182.917705:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880182.917706:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880182.917707:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:2.0:1788880182.917710:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880182.917714:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00800000:00000001:2.0:1788880182.917715:0:203392:0:(lmv_obd.c:1568:lmv_get_root()) Process leaving (rc=0 : 0 : 0) 00000080:00000004:2.0:1788880182.917718:0:203392:0:(llite_lib.c:622:client_common_fill_super()) rootfid [0x200000007:0x1:0x0] 00000080:00000001:2.0:1788880182.917720:0:203392:0:(obd_class.h:511:obd_get_info()) Process entered 00800000:00000001:2.0:1788880182.917722:0:203392:0:(lmv_obd.c:3592:lmv_get_info()) Process entered 00800000:00000001:2.0:1788880182.917724:0:203392:0:(obd_class.h:511:obd_get_info()) Process entered 00000002:00000001:2.0:1788880182.917727:0:203392:0:(mdc_request.c:2756:mdc_get_info()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880182.917729:0:203392:0:(obd_class.h:525:obd_get_info()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880182.917731:0:203392:0:(lmv_obd.c:3624:lmv_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.917732:0:203392:0:(obd_class.h:525:obd_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000040:2.0:1788880182.917738:0:203392:0:(obd_cksum.h:77:obd_cksum_types_supported_client()) Crypto hash speed: crc 1651, crc32c 9963, adler 1371 00000080:00100000:2.0:1788880182.917741:0:203392:0:(llite_lib.c:784:client_common_fill_super()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34682648 ocd_grant: 2146304 00000080:00000001:2.0:1788880182.917743:0:203392:0:(obd_class.h:860:obd_connect()) Process entered 00020000:00000001:2.0:1788880182.917745:0:203392:0:(lov_obd.c:185:lov_connect()) Process entered 00020000:01000000:2.0:1788880182.917747:0:203392:0:(lov_obd.c:186:lov_connect()) connect #0 00000020:00000001:2.0:1788880182.917749:0:203392:0:(genops.c:1396:class_connect()) Process entered 00000020:00000001:2.0:1788880182.917751:0:203392:0:(genops.c:952:__class_new_export()) Process entered 00000020:00000010:2.0:1788880182.917754:0:203392:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 000000008c018d6d. 00000020:00000001:2.0:1788880182.917757:0:203392:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880182.917759:0:203392:0:(lustre_handles.c:93:class_handle_hash()) added object 000000008c018d6d with handle 0x805d548a36a886ac to hash 00000020:00000001:2.0:1788880182.917766:0:203392:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:2.0:1788880182.917770:0:203392:0:(obd_class.h:1016:obd_init_export()) Process entered 00000020:00000001:2.0:1788880182.917772:0:203392:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880182.917775:0:203392:0:(obd_config.c:994:class_incref()) incref lustre-clilov-ffff8ca787527000 (00000000998f2370) now 4 - export 00000020:00000001:2.0:1788880182.917778:0:203392:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617249948792832 : -126823760758784 : ffff8ca78aca7800) 00000020:00000040:2.0:1788880182.917780:0:203392:0:(genops.c:900:class_export_put()) PUTting export 000000008c018d6d : new refcount 1 00000020:00000080:2.0:1788880182.917782:0:203392:0:(genops.c:1406:class_connect()) connect: client ea27006f-52a8-4007-b69a-dcac8279445a, cookie 0x805d548a36a886ac 00000020:00000001:2.0:1788880182.917783:0:203392:0:(genops.c:1407:class_connect()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880182.917785:0:203392:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880182.917787:0:203392:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886ac 00000020:00000001:2.0:1788880182.917789:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880182.917790:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET export 000000008c018d6d refcount=2 00000020:00000001:2.0:1788880182.917792:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948792832 : -126823760758784 : ffff8ca78aca7800) 00000020:00000001:2.0:1788880182.917794:0:203392:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948792832 : -126823760758784 : ffff8ca78aca7800) 00020000:00000001:2.0:1788880182.917799:0:203392:0:(lov_obd.c:101:lov_connect_osc()) Process entered 00000100:00000001:2.0:1788880182.917802:0:203392:0:(obd_class.h:1223:obd_import_event()) Process entered 00000008:00000001:2.0:1788880182.917805:0:203392:0:(osc_request.c:3901:osc_import_event()) Process entered 00000008:00000001:2.0:1788880182.917808:0:203392:0:(osc_request.c:3968:osc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.917809:0:203392:0:(obd_class.h:1234:obd_import_event()) Process leaving 00020000:00000001:2.0:1788880182.917811:0:203392:0:(obd_class.h:1356:obd_register_observer()) Process entered 00020000:00000001:2.0:1788880182.917813:0:203392:0:(obd_class.h:1369:obd_register_observer()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.917815:0:203392:0:(obd_class.h:860:obd_connect()) Process entered 00010000:00000001:2.0:1788880182.917817:0:203392:0:(ldlm_lib.c:668:client_connect_import()) Process entered 00000020:00000001:2.0:1788880182.917819:0:203392:0:(genops.c:1396:class_connect()) Process entered 00000020:00000001:2.0:1788880182.917820:0:203392:0:(genops.c:952:__class_new_export()) Process entered 00000020:00000010:2.0:1788880182.917823:0:203392:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 000000003972ec14. 00000020:00000001:2.0:1788880182.917825:0:203392:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880182.917826:0:203392:0:(lustre_handles.c:93:class_handle_hash()) added object 000000003972ec14 with handle 0x805d548a36a886b3 to hash 00000020:00000001:2.0:1788880182.917828:0:203392:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:2.0:1788880182.917832:0:203392:0:(obd_class.h:1016:obd_init_export()) Process entered 00000020:00000001:2.0:1788880182.917847:0:203392:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880182.917849:0:203392:0:(genops.c:889:class_export_get()) GET export 000000003972ec14 refcount=3 00000020:00000040:2.0:1788880182.917852:0:203392:0:(obd_config.c:994:class_incref()) incref lustre-OST0000-osc-ffff8ca787527000 (00000000bd6a9017) now 5 - export 00000020:00000001:2.0:1788880182.917854:0:203392:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617249948788736 : -126823760762880 : ffff8ca78aca6800) 00000020:00000040:2.0:1788880182.917856:0:203392:0:(genops.c:900:class_export_put()) PUTting export 000000003972ec14 : new refcount 2 00000020:00000080:2.0:1788880182.917864:0:203392:0:(genops.c:1406:class_connect()) connect: client LOV_OSC_UUID, cookie 0x805d548a36a886b3 00000020:00000001:2.0:1788880182.917865:0:203392:0:(genops.c:1407:class_connect()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880182.917867:0:203392:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880182.917868:0:203392:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886b3 00000020:00000001:2.0:1788880182.917869:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880182.917871:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET export 000000003972ec14 refcount=3 00000020:00000001:2.0:1788880182.917872:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948788736 : -126823760762880 : ffff8ca78aca6800) 00000020:00000001:2.0:1788880182.917874:0:203392:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948788736 : -126823760762880 : ffff8ca78aca6800) 00000100:00000001:2.0:1788880182.917877:0:203392:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00000001:2.0:1788880182.917880:0:203392:0:(import.c:485:import_select_connection()) Process entered 00000100:00080000:2.0:1788880182.917883:0:203392:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 0 00000400:00000200:2.0:1788880182.917887:0:203392:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000100:00000001:2.0:1788880182.917890:0:203392:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880182.917892:0:203392:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880182.917894:0:203392:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000001:2.0:1788880182.917896:0:203392:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880182.917897:0:203392:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886b3 00000020:00000001:2.0:1788880182.917899:0:203392:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880182.917900:0:203392:0:(lustre_handles.c:147:class_handle2object()) GET export 000000003972ec14 refcount=4 00000020:00000001:2.0:1788880182.917902:0:203392:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948788736 : -126823760762880 : ffff8ca78aca6800) 00000020:00000001:2.0:1788880182.917903:0:203392:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948788736 : -126823760762880 : ffff8ca78aca6800) 00000100:00000001:2.0:1788880182.917905:0:203392:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880182.917906:0:203392:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880182.917908:0:203392:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000040:2.0:1788880182.917910:0:203392:0:(genops.c:900:class_export_put()) PUTting export 000000003972ec14 : new refcount 3 00000100:00080000:2.0:1788880182.917912:0:203392:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff8ca787527000: import 00000000beb50720 using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000100:00000001:2.0:1788880182.917914:0:203392:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.917916:0:203392:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:2.0:1788880182.917921:0:203392:0:(sec.c:1453:sptlrpc_sec_create()) Process entered 02000000:08000000:2.0:1788880182.917923:0:203392:0:(sec.c:1471:sptlrpc_sec_create()) osc lustre-OST0000-osc-ffff8ca787527000: select security flavor null 02000000:00000001:2.0:1788880182.917927:0:203392:0:(sec.c:1492:sptlrpc_sec_create()) Process leaving (rc=18446744072650609120 : -1058942496 : ffffffffc0e1d1e0) 02000000:00000001:2.0:1788880182.917935:0:203392:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.917937:0:203392:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:2.0:1788880182.917939:0:203392:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34682648 ocd_grant: 134217728, lost: 0. 00000008:00000001:2.0:1788880182.917942:0:203392:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.917943:0:203392:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880182.917945:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880182.917947:0:203392:0:(genops.c:1231:class_import_get()) import 00000000beb50720 refcount=2 obd=lustre-OST0000-osc-ffff8ca787527000 02000000:00000001:2.0:1788880182.917949:0:203392:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 02000000:00000001:2.0:1788880182.917949:0:203392:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.917951:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880182.917952:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880182.917954:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880182.917956:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 00000000150c5893. 00000100:00000001:2.0:1788880182.917959:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880182.917962:0:203392:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880182.917967:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880182.917968:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.917970:0:203392:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_rcv+-1] req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.917983:0:203392:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.917985:0:203392:0:(pinger.c:367:ptlrpc_pinger_add_import()) Process entered 00000100:00080000:2.0:1788880182.917987:0:203392:0:(pinger.c:373:ptlrpc_pinger_add_import()) adding pingable import ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID 00000020:00000040:2.0:1788880182.917989:0:203392:0:(genops.c:1231:class_import_get()) import 00000000beb50720 refcount=3 obd=lustre-OST0000-osc-ffff8ca787527000 00000100:00000001:2.0:1788880182.918008:0:203392:0:(pinger.c:384:ptlrpc_pinger_add_import()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880182.918011:0:203392:0:(ldlm_lib.c:725:client_connect_import()) Process leaving 00020000:00000001:2.0:1788880182.918015:0:203392:0:(obd_class.h:876:obd_connect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.918015:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00020000:01000000:2.0:1788880182.918017:0:203392:0:(lov_obd.c:159:lov_connect_osc()) Connected tgt idx 0 lustre-OST0000_UUID (lustre-OST0000-osc-ffff8ca787527000) active 00000100:00000001:3.0:1788880182.918019:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880182.918023:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880182.918027:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00020000:00000001:2.0:1788880182.918035:0:203392:0:(lov_obd.c:173:lov_connect_osc()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.918036:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880182.918039:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.918040:0:203392:0:(lov_obd.c:428:lov_notify()) Process entered 02000000:00000001:3.0:1788880182.918043:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000080:00000001:2.0:1788880182.918044:0:203392:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 02000000:00000001:3.0:1788880182.918046:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000080:00000001:2.0:1788880182.918047:0:203392:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.918051:0:203392:0:(lov_obd.c:465:lov_notify()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880182.918056:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.918062:0:203392:0:(lov_obd.c:235:lov_connect()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880182.918064:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780635392:192.168.204.111@tcp:8: 00000080:00000001:2.0:1788880182.918066:0:203392:0:(obd_class.h:876:obd_connect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.918072:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000080:00000010:2.0:1788880182.918073:0:203392:0:(llite_lib.c:843:client_common_fill_super()) kmalloced '(op_data)': 392 at 00000000a581a743. 02000000:00000001:3.0:1788880182.918075:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880182.918077:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880182.918080:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000020:00000010:2.0:1788880182.918081:0:203392:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 640 at 00000000df2b698c. 02000000:00000010:3.0:1788880182.918083:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000002a6ca715. 02000000:00000001:3.0:1788880182.918087:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880182.918090:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000ea759cb. 00000400:00000010:3.0:1788880182.918093:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00800000:00000001:2.0:1788880182.918095:0:203392:0:(lmv_obd.c:1622:lmv_getattr()) Process entered 00000100:00000200:3.0:1788880182.918095:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780635392, portal 4 00000100:00000001:3.0:1788880182.918099:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880182.918100:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00800000:00000001:2.0:1788880182.918102:0:203392:0:(lmv_fld.c:38:lmv_fld_lookup()) Process entered 00000020:00000010:3.0:1788880182.918105:0:201284:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 00000000a1774999. 80000000:00000001:2.0:1788880182.918106:0:203392:0:(fld_request.c:427:fld_client_lookup()) Process entered 80000000:00000001:2.0:1788880182.918110:0:203392:0:(fld_cache.c:479:fld_cache_lookup()) Process entered 00000100:00000040:3.0:1788880182.918111:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/544 e 0 to 0 dl 1788880192 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 80000000:00000001:2.0:1788880182.918114:0:203392:0:(fld_cache.c:502:fld_cache_lookup()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 80000000:00000001:2.0:1788880182.918120:0:203392:0:(fld_request.c:114:fld_client_get_target()) Process entered 00000100:00000001:3.0:1788880182.918121:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880182.918124:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 80000000:00000001:2.0:1788880182.918126:0:203392:0:(fld_request.c:45:fld_rrb_scan()) Process entered 00000400:00000010:3.0:1788880182.918126:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007fe20b6f. 80000000:00000001:2.0:1788880182.918130:0:203392:0:(fld_request.c:62:fld_rrb_scan()) Process leaving (rc=18446617250282350528 : -126823427201088 : ffff8ca79eac27c0) 00000100:00000200:3.0:1788880182.918130:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1875776780635392, offset 0 00000400:00000200:3.0:1788880182.918135:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 80000000:00000040:2.0:1788880182.918136:0:203392:0:(fld_request.c:124:fld_client_get_target()) cli-lustre-clilmv-ffff8ca787527000: Found target (idx 0) by seq 0x200000007 80000000:00000001:2.0:1788880182.918140:0:203392:0:(fld_request.c:127:fld_client_get_target()) Process leaving (rc=18446617250282350528 : -126823427201088 : ffff8ca79eac27c0) 00000400:00000200:3.0:1788880182.918142:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 80000000:00000040:2.0:1788880182.918143:0:203392:0:(fld_request.c:443:fld_client_lookup()) cli-lustre-clilmv-ffff8ca787527000: Lookup fld entry (seq: 0x200000007) on target lustre-MDT0000-mdc-ffff8ca787527000 (idx 0) 00000400:00000200:3.0:1788880182.918147:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 80000000:00000001:2.0:1788880182.918148:0:203392:0:(fld_request.c:312:fld_client_rpc()) Process entered 00000400:00000200:3.0:1788880182.918151:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880182.918154:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:2.0:1788880182.918155:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000400:00000200:3.0:1788880182.918157:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000020:00000040:2.0:1788880182.918160:0:203392:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000200:3.0:1788880182.918161:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [30:30:256:4294967295] 192.168.204.111@tcp LPNI seq info [30:30:8:4294967295] 00000100:00000001:2.0:1788880182.918168:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880182.918172:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000200:3.0:1788880182.918174:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788880182.918175:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880182.918180:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000010:2.0:1788880182.918183:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000e6a93e02. 00000800:00000010:3.0:1788880182.918183:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000800:00000200:3.0:1788880182.918189:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880182.918190:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.918194:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:2.0:1788880182.918196:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000800:00000200:3.0:1788880182.918199:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:2.0:1788880182.918200:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000126d8278. 00000800:00000200:3.0:1788880182.918203:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:2.0:1788880182.918204:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249931839360 : -126823777712256 : ffff8ca789c7c780) 00000100:00000001:2.0:1788880182.918208:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880182.918209:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000100:00000001:2.0:1788880182.918211:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880182.918213:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.918214:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:2.0:1788880182.918216:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880182.918219:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.918220:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880182.918224:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: using a timeout of 0 usecs (0 jiffies) 00000100:00000001:3.0:1788880182.918230:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305819094 00000100:00000001:2.0:1788880182.918260:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880182.918263:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.918264:0:201284:0:(niobuf.c:685:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 0, simply extend its deadline from 4305819093 02000000:00000001:2.0:1788880182.918266:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880182.918269:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880182.918269:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305819095 02000000:00000001:2.0:1788880182.918274:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.918275:0:201284:0:(niobuf.c:685:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 1, simply extend its deadline from 4305819093 00000100:00100000:2.0:1788880182.918278:0:203392:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job mount.lustre:ea27006f-52a8-4007-b69a-dcac8279445a:203392:1875776780635520:192.168.204.111@tcp:900: 00000100:00000001:3.0:1788880182.918279:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305819095 00000800:00000200:0.0:1788880182.918285:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880182.918285:0:201284:0:(niobuf.c:685:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 2, simply extend its deadline from 4305819093 00000100:00000001:2.0:1788880182.918286:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880182.918291:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880182.918291:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305819095 00000800:00000010:0.0:1788880182.918292:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 02000000:00000001:2.0:1788880182.918295:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.918296:0:201284:0:(niobuf.c:685:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 3, simply extend its deadline from 4305819093 00000400:00000200:0.0:1788880182.918299:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:2.0:1788880182.918299:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880182.918300:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000010:2.0:1788880182.918305:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000078217d0d. 00000400:00000200:0.0:1788880182.918307:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:2.0:1788880182.918310:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880182.918313:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007fe20b6f 00000100:00000001:3.0:1788880182.918314:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880182.918315:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000400:00000010:0.0:1788880182.918317:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007fe20b6f. 00000100:00000001:3.0:1788880182.918319:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880182.918323:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880182.918323:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880182.918326:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000001:3.0:1788880182.918328:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:2.0:1788880182.918331:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780635520, portal 10 00000100:00000001:3.0:1788880182.918331:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:2.0:1788880182.918336:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000200:0.0:1788880182.918337:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/544 e 0 to 0 dl 1788880192 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880182.918337:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880182.918340:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000100:00000001:3.0:1788880182.918341:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880182.918344:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880182.918349:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.918351:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.918354:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880182.918360:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/544 e 0 to 0 dl 1788880192 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.918362:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880182.918368:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880182.918372:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005071f3f0. 00000100:00000001:0.0:1788880182.918375:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880182.918376:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 264 bytes to portal 29, xid 1875776780635520, offset 0 00000100:00000001:0.0:1788880182.918378:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:2.0:1788880182.918383:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880182.918391:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880182.918393:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880182.918395:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880182.918397:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880182.918399:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880182.918401:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [31:31:256:4294967295] 192.168.204.111@tcp LPNI seq info [31:31:8:4294967295] 00000400:00000200:2.0:1788880182.918406:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880182.918413:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 264 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880182.918415:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000961894e8. 00000800:00000200:2.0:1788880182.918418:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880182.918421:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880182.918422:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000961894e8 type 1, nob 360 niov 1 nkiov 1 00000100:00000001:2.0:1788880182.918431:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.918433:0:203392:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 4225560 usecs (4226 jiffies) 00000100:00000001:2.0:1788880182.918435:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305818250 00000100:00000001:2.0:1788880182.918436:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305819094 00000100:00000001:2.0:1788880182.918438:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305818250 00000100:00000001:2.0:1788880182.918439:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305819095 00000100:00000001:2.0:1788880182.918439:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305818250 00000100:00000001:2.0:1788880182.918440:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305819095 00000100:00000001:2.0:1788880182.918441:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305818250 00000100:00000001:2.0:1788880182.918442:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305819095 00000100:00000001:2.0:1788880182.918443:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880182.918444:0:203392:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.918446:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880182.918446:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:2.0:1788880182.918448:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 00000000126d8278 going to sleep for 11 seconds 00000100:00000001:2.0:1788880182.918449:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880182.918450:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880182.918484:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880182.918487:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000961894e8. 00000400:00000200:0.0:1788880182.918490:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.918493:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880182.918495:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005071f3f0 00000400:00000010:0.0:1788880182.918496:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005071f3f0. 00000100:00000001:0.0:1788880182.918498:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880182.918500:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.918505:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880182.918506:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.918510:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880182.918511:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880182.919540:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.919546:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880182.919548:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.919549:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.919554:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880182.919560:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 248 into portal 10 MB=0x6aa0250b00d80 00000400:00000200:0.0:1788880182.919564:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 248/248 into md 0x1ed [1] + 224 00000800:00000001:0.0:1788880182.919567:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.919570:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.919571:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.919573:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880182.919580:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880182.919583:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880182.919588:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=248 offset=224 replen=248 req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880182.919599:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880182.919603:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.919605:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.919629:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880182.919635:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880182.919636:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880182.919638:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:2.0:1788880182.919642:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788880182.919643:0:203392:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:2.0:1788880182.919646:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880182.919651:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880182.919665:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.919671:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880182.919672:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.919675:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880182.919677:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880182.919678:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880182.919680:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.919682:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880182.919686:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 0000000061d3eef5 time=1788880182 v=5 (0 0 0 0) 00000100:00001000:2.0:1788880182.919689:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=0 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880182.919692:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880182.919693:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880182.919695:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880182.919698:0:203392:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880182.919702:0:203392:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.919705:0:203392:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880182.919706:0:203392:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880182.919707:0:203392:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880182.919708:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.919710:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.919713:0:203392:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880182.919714:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880182.919716:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880182.919717:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880182.919722:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job mount.lustre:ea27006f-52a8-4007-b69a-dcac8279445a:203392:1875776780635520:192.168.204.111@tcp:900: 00000100:00000001:2.0:1788880182.919725:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.919730:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.919732:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:2.0:1788880182.919732:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880182.919734:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.919738:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880182.919739:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000126d8278. 00000100:00000001:2.0:1788880182.919741:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:2.0:1788880182.919742:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 80000000:00000001:2.0:1788880182.919745:0:203392:0:(fld_request.c:407:fld_client_rpc()) Process leaving 00000100:00000001:2.0:1788880182.919746:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880182.919747:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d838cab4 x1875776780635520/t0(0) o900->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:29/10 lens 264/248 e 0 to 0 dl 1788880198 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.919751:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880182.919752:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880182.919753:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000078217d0d. 02000000:00000001:2.0:1788880182.919755:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880182.919756:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880182.919757:0:203392:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880182.919758:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880182.919759:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000e6a93e02. 02000000:00000001:2.0:1788880182.919762:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880182.919763:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880182.919764:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d838cab4. 00000100:00000001:2.0:1788880182.919766:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880182.919766:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 80000000:00000010:2.0:1788880182.919769:0:203392:0:(fld_cache.c:357:fld_cache_entry_create()) kmalloced '(f_new)': 56 at 000000000ea2f792. 80000000:00000001:2.0:1788880182.919770:0:203392:0:(fld_cache.c:362:fld_cache_entry_create()) Process leaving (rc=18446617250282350976 : -126823427200640 : ffff8ca79eac2980) 80000000:00000001:2.0:1788880182.919772:0:203392:0:(fld_cache.c:386:fld_cache_insert_nolock()) Process entered 80000000:00000001:2.0:1788880182.919773:0:203392:0:(fld_cache.c:198:fld_cache_shrink()) Process entered 80000000:00000001:2.0:1788880182.919774:0:203392:0:(fld_cache.c:203:fld_cache_shrink()) Process leaving (rc=0 : 0 : 0) 80000000:00000040:2.0:1788880182.919776:0:203392:0:(fld_cache.c:417:fld_cache_insert_nolock()) insert range [0x0000000200000007-0x0000000200000008]:0:mdt 80000000:00000001:2.0:1788880182.919777:0:203392:0:(fld_cache.c:110:fld_fix_new_list()) Process entered 80000000:00000001:2.0:1788880182.919778:0:203392:0:(fld_cache.c:165:fld_fix_new_list()) Process leaving 80000000:00000001:2.0:1788880182.919779:0:203392:0:(fld_cache.c:421:fld_cache_insert_nolock()) Process leaving (rc=0 : 0 : 0) 80000000:00000001:2.0:1788880182.919780:0:203392:0:(fld_cache.c:440:fld_cache_insert()) Process leaving (rc=0 : 0 : 0) 80000000:00000001:2.0:1788880182.919783:0:203392:0:(fld_request.c:485:fld_client_lookup()) Process leaving (rc=0 : 0 : 0) 00800000:00000002:2.0:1788880182.919785:0:203392:0:(lmv_fld.c:57:lmv_fld_lookup()) FLD lookup got mds #0 for fid=[0x200000007:0x1:0x0] 00800000:00000001:2.0:1788880182.919786:0:203392:0:(lmv_fld.c:65:lmv_fld_lookup()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880182.919790:0:203392:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 640 at 00000000da1bf4ae. 00000002:00000001:2.0:1788880182.919793:0:203392:0:(mdc_request.c:217:mdc_getattr()) Process entered 00000100:00000010:2.0:1788880182.919794:0:203392:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000020:00000040:2.0:1788880182.919796:0:203392:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880182.919798:0:203392:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880182.919800:0:203392:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880182.919801:0:203392:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880182.919804:0:203392:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 0000000078217d0d. 00000100:00000001:2.0:1788880182.919806:0:203392:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880182.919809:0:203392:0:(mdc_request.c:161:mdc_getattr_common()) Process entered 00000100:00000001:2.0:1788880182.919810:0:203392:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:2.0:1788880182.919811:0:203392:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:2.0:1788880182.919813:0:203392:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000126d8278. 00000100:00000001:2.0:1788880182.919814:0:203392:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249931839360 : -126823777712256 : ffff8ca789c7c780) 00000100:00000001:2.0:1788880182.919815:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880182.919816:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000100:00000001:2.0:1788880182.919818:0:203392:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880182.919819:0:203392:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.919820:0:203392:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:2.0:1788880182.919820:0:203392:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880182.919822:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/66264 e 0 to 0 dl 0 ref 2 fl New:QU/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:2.0:1788880182.919826:0:203392:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880182.919827:0:203392:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.919829:0:203392:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880182.919830:0:203392:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880182.919831:0:203392:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880182.919832:0:203392:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job mount.lustre:ea27006f-52a8-4007-b69a-dcac8279445a:203392:1875776780635648:192.168.204.111@tcp:33: 00000100:00000001:2.0:1788880182.919848:0:203392:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880182.919850:0:203392:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880182.919851:0:203392:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.919856:0:203392:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880182.919897:0:203392:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 131072 at 00000000f19652a7. 02000000:00000001:2.0:1788880182.919899:0:203392:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880182.919903:0:203392:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000400:00000010:2.0:1788880182.919907:0:203392:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 632 at 00000000a977081d. 00000100:00000200:2.0:1788880182.919910:0:203392:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 131072 bytes, xid 1875776780635648, portal 10 00000100:00000001:2.0:1788880182.919912:0:203392:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880182.919913:0:203392:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000100:00000040:2.0:1788880182.919917:0:203392:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=600 req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/66264 e 0 to 0 dl 1788880198 ref 3 fl Rpc:r/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:2.0:1788880182.919923:0:203392:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880182.919925:0:203392:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880182.919927:0:203392:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000200:2.0:1788880182.919929:0:203392:0:(niobuf.c:94:ptl_send_buf()) Sending 448 bytes to portal 23, xid 1875776780635648, offset 0 00000400:00000200:2.0:1788880182.919932:0:203392:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880182.919937:0:203392:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880182.919941:0:203392:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880182.919943:0:203392:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880182.919946:0:203392:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880182.919948:0:203392:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880182.919951:0:203392:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [32:32:256:4294967295] 192.168.204.111@tcp LPNI seq info [32:32:8:4294967295] 00000400:00000200:2.0:1788880182.919959:0:203392:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880182.919964:0:203392:0:(socklnd_cb.c:931:ksocknal_send()) sending 448 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880182.919967:0:203392:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008382090e. 00000800:00000200:2.0:1788880182.919971:0:203392:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880182.919977:0:203392:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880182.919980:0:203392:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008382090e type 1, nob 544 niov 1 nkiov 1 00000100:00000001:2.0:1788880182.919989:0:203392:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.919991:0:203392:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 3622170 usecs (3623 jiffies) 00000100:00000001:2.0:1788880182.919994:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305817648 00000100:00000001:2.0:1788880182.919995:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305819094 00000100:00000001:2.0:1788880182.920003:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305817648 00000100:00000001:2.0:1788880182.920006:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305819095 00000100:00000001:2.0:1788880182.920008:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305817648 00000100:00000001:2.0:1788880182.920010:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305819095 00000100:00000001:2.0:1788880182.920012:0:203392:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305817648 00000100:00000001:2.0:1788880182.920014:0:203392:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305819095 00000100:00000001:2.0:1788880182.920015:0:203392:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880182.920017:0:203392:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.920020:0:203392:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880182.920021:0:203392:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:2.0:1788880182.920023:0:203392:0:(client.c:2702:ptlrpc_set_wait()) set 00000000126d8278 going to sleep for 11 seconds 00000100:00000001:2.0:1788880182.920026:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880182.920027:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880182.920043:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000200:0.0:1788880182.920057:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880182.920061:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008382090e. 00000400:00000200:0.0:1788880182.920065:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.920068:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880182.920071:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000800:00000001:3.0:1788880182.920071:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880182.920074:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:0.0:1788880182.920077:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880182.920079:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/66264 e 0 to 0 dl 1788880198 ref 3 fl Rpc:r/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:0.0:1788880182.920084:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880182.920086:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/66264 e 0 to 0 dl 1788880198 ref 3 fl Rpc:Qr/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:0.0:1788880182.920101:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880182.920103:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880182.920137:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880182.920142:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880182.920148:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880182.920156:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 416 into portal 4 MB=0x6aa0250b00d00 00000400:00000200:3.0:1788880182.920162:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.204.111@tcp of length 416/416 into md 0x1e5 [1] + 0 00000800:00000001:3.0:1788880182.920166:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880182.920177:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880182.920180:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880182.920183:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880182.920187:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880182.920191:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/544 e 0 to 0 dl 1788880192 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880182.920199:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=416 offset=0 replen=544 req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/544 e 0 to 0 dl 1788880192 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880182.920217:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880182.920222:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880182.920224:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.920252:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880182.920255:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880182.920258:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880182.920259:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880182.920261:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000ea759cb. 00000400:00000200:2.0:1788880182.920264:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000010:2.0:1788880182.920266:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:2.0:1788880182.920268:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880182.920271:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/544 e 0 to 0 dl 1788880192 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880182.920278:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/544 e 0 to 0 dl 1788880192 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.920283:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880182.920284:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.920285:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880182.920286:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880182.920287:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880182.920288:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.920290:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880182.920302:0:201284:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 000000009679bc01. 00000100:00001000:2.0:1788880182.920306:0:201284:0:(import.c:2149:obd_at_measure()) add 5 to 0000000040ba1a3b time=1788880182 v=5 (0 0 0 0) 00000100:00001000:2.0:1788880182.920309:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 000000003184a8ce time=1788880182 v=0 (0 0 0 0) 00000100:00001000:2.0:1788880182.920312:0:201284:0:(import.c:2215:obd_at_measure()) AT 000000003184a8ce change: old=0 new=5 delta=5 (val=1) hist 1 0 0 0 00000100:00000001:2.0:1788880182.920315:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880182.920316:0:201284:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880182.920318:0:201284:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880182.920321:0:201284:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/416 e 0 to 0 dl 1788880192 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880182.920329:0:201284:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.920332:0:201284:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00100000:2.0:1788880182.920333:0:201284:0:(client.c:3112:ptlrpc_free_committed()) lustre-OST0000-osc-ffff8ca787527000: committing for last_committed 0 gen 1 00000100:00000040:2.0:1788880182.920335:0:201284:0:(client.c:3222:ptlrpc_free_committed()) lustre-OST0000-osc-ffff8ca787527000: normal: skip=0 replay=0/0 committed=0/0 00000100:00000001:2.0:1788880182.920338:0:201284:0:(client.c:3223:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880182.920339:0:201284:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.920342:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/416 e 0 to 0 dl 1788880192 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.920352:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880182.920354:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880182.920355:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.920357:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00080000:2.0:1788880182.920362:0:201284:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff8ca787527000: connect to target with instance 3 00000020:00000001:2.0:1788880182.920364:0:201284:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880182.920366:0:201284:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886b3 00000020:00000001:2.0:1788880182.920368:0:201284:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880182.920371:0:201284:0:(lustre_handles.c:147:class_handle2object()) GET export 000000003972ec14 refcount=4 00000020:00000001:2.0:1788880182.920373:0:201284:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948788736 : -126823760762880 : ffff8ca78aca6800) 00000020:00000001:2.0:1788880182.920375:0:201284:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948788736 : -126823760762880 : ffff8ca78aca6800) 00000100:00001000:2.0:1788880182.920379:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 000000003184a8ce time=1788880182 v=0 (0 0 0 0) 00000100:00001000:2.0:1788880182.920382:0:201284:0:(import.c:2215:obd_at_measure()) AT 000000003184a8ce change: old=0 new=5 delta=5 (val=1) hist 1 0 0 0 00000100:00000040:2.0:1788880182.920386:0:201284:0:(obd_cksum.h:77:obd_cksum_types_supported_client()) Crypto hash speed: crc 1651, crc32c 9963, adler 1371 00000100:00080000:2.0:1788880182.920396:0:201284:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff8ca787527000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000020:00000040:2.0:1788880182.920399:0:201284:0:(genops.c:900:class_export_put()) PUTting export 000000003972ec14 : new refcount 3 00000100:00000001:2.0:1788880182.920400:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 00000008:00000001:2.0:1788880182.920402:0:201284:0:(osc_request.c:3901:osc_import_event()) Process entered 00000008:00000020:2.0:1788880182.920404:0:201284:0:(osc_request.c:1062:osc_init_grant()) lustre-OST0000-osc-ffff8ca787527000, setting cl_avail_grant: 8437760 cl_lost_grant: 0. chunk bits: 12 cl_max_extent_pages: 16384 00000008:00000001:2.0:1788880182.920407:0:201284:0:(obd_class.h:1243:obd_notify()) Process entered 00020000:00000001:2.0:1788880182.920408:0:201284:0:(lov_obd.c:428:lov_notify()) Process entered 00000080:00000001:2.0:1788880182.920410:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:2.0:1788880182.920413:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.920414:0:201284:0:(lov_obd.c:465:lov_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1788880182.920415:0:201284:0:(obd_class.h:1256:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1788880182.920417:0:201284:0:(osc_request.c:3968:osc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.920418:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00080000:2.0:1788880182.920419:0:201284:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:2.0:1788880182.920422:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000beb50720 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00000100:00000001:2.0:1788880182.920423:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 00000008:00000001:2.0:1788880182.920424:0:201284:0:(osc_request.c:3901:osc_import_event()) Process entered 00000008:00000001:2.0:1788880182.920424:0:201284:0:(obd_class.h:1243:obd_notify()) Process entered 00020000:00000001:2.0:1788880182.920425:0:201284:0:(lov_obd.c:428:lov_notify()) Process entered 00020000:00000001:2.0:1788880182.920426:0:201284:0:(lov_obd.c:349:lov_set_osc_active()) Process entered 00020000:00000040:2.0:1788880182.920427:0:201284:0:(lov_obd.c:351:lov_set_osc_active()) Searching in lov 00000000e0e80469 for uuid lustre-OST0000_UUID event(1) 00020000:01000000:2.0:1788880182.920429:0:201284:0:(lov_obd.c:398:lov_set_osc_active()) Marking OSC lustre-OST0000_UUID active 00020000:00000040:2.0:1788880182.920431:0:201284:0:(lov_obd.c:415:lov_set_osc_active()) lustre-OST0000_UUID: lov idx 0 conn 805d548a36a886b3 00020000:00000001:2.0:1788880182.920432:0:201284:0:(lov_obd.c:419:lov_set_osc_active()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.920433:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:2.0:1788880182.920435:0:201284:0:(lcommon_misc.c:85:cl_ocd_update()) Process entered 00000080:00000004:2.0:1788880182.920436:0:201284:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0467af2e3640038 -> 0xa0427af2e3440038 00000080:00000001:2.0:1788880182.920438:0:201284:0:(lcommon_misc.c:37:cl_init_ea_size()) Process entered 00000080:00000001:2.0:1788880182.920438:0:201284:0:(obd_class.h:511:obd_get_info()) Process entered 00020000:00000001:2.0:1788880182.920440:0:201284:0:(lov_obd.c:1158:lov_get_info()) Process entered 00020000:00000001:2.0:1788880182.920442:0:201284:0:(lov_obd.c:1209:lov_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.920443:0:201284:0:(obd_class.h:525:obd_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.920444:0:201284:0:(obd_class.h:511:obd_get_info()) Process entered 00020000:00000001:2.0:1788880182.920444:0:201284:0:(lov_obd.c:1158:lov_get_info()) Process entered 00020000:00000001:2.0:1788880182.920445:0:201284:0:(lov_obd.c:1209:lov_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.920445:0:201284:0:(obd_class.h:525:obd_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00080000:2.0:1788880182.920450:0:201284:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00800000:00000001:2.0:1788880182.920452:0:201284:0:(lmv_obd.c:283:lmv_init_ea_size()) Process entered 00000002:00000001:2.0:1788880182.920454:0:201284:0:(mdc_request.c:3107:mdc_init_ea_size()) Process entered 00000002:00000001:2.0:1788880182.920455:0:201284:0:(mdc_request.c:3114:mdc_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880182.920456:0:201284:0:(mdc_request.c:3107:mdc_init_ea_size()) Process entered 00000002:00000001:2.0:1788880182.920457:0:201284:0:(mdc_request.c:3114:mdc_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880182.920458:0:201284:0:(lmv_obd.c:311:lmv_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.920459:0:201284:0:(lcommon_misc.c:59:cl_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.920460:0:201284:0:(lcommon_misc.c:112:cl_ocd_update()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.920461:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.920462:0:201284:0:(lov_obd.c:465:lov_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1788880182.920463:0:201284:0:(obd_class.h:1256:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1788880182.920463:0:201284:0:(osc_request.c:3968:osc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.920464:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00000001:2.0:1788880182.920465:0:201284:0:(import.c:1241:ptlrpc_connect_interpret()) Process leaving via finish (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880182.920467:0:201284:0:(import.c:1648:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00000001:2.0:1788880182.920469:0:201284:0:(import.c:1754:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.920471:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.920475:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/416 e 0 to 0 dl 1788880192 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880182.920481:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780635392:192.168.204.111@tcp:8: 00000100:00000001:2.0:1788880182.920484:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.920485:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880182.920487:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780635392/t0(0) o8->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 520/416 e 0 to 0 dl 1788880192 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880182.920494:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880182.920495:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880182.920496:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000002a6ca715. 02000000:00000001:2.0:1788880182.920499:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880182.920500:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880182.920501:0:201284:0:(genops.c:1244:class_import_put()) import 00000000beb50720 refcount=2 obd=lustre-OST0000-osc-ffff8ca787527000 00000020:00000001:2.0:1788880182.920503:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880182.920504:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 00000000150c5893. 02000000:00000001:2.0:1788880182.920507:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880182.920512:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880182.920513:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:2.0:1788880182.920516:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880182.920517:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.920519:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.920523:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880182.920524:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.920525:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880182.920526:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880182.929257:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.929273:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880182.929278:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.929280:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.929291:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880182.929308:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 464 into portal 10 MB=0x6aa0250b00e00 00000400:00000200:0.0:1788880182.929317:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 464/464 into md 0x1f5 [32] + 224 00000800:00000001:0.0:1788880182.929324:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.929330:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880182.929333:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880182.929348:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880182.929356:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880182.929363:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/66264 e 0 to 0 dl 1788880198 ref 2 fl Rpc:Qr/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000040:0.0:1788880182.929373:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=600 mlen=464 offset=224 replen=66264 req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/66264 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQ/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:0.0:1788880182.929393:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880182.929402:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880182.929405:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.929444:0:203392:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880182.929452:0:203392:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880182.929455:0:203392:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880182.929457:0:203392:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:2.0:1788880182.929462:0:203392:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a977081d 00000400:00000010:2.0:1788880182.929464:0:203392:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 632 at 00000000a977081d. 00000100:00000001:2.0:1788880182.929468:0:203392:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880182.929472:0:203392:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/66264 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQ/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000200:2.0:1788880182.929481:0:203392:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/66264 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:2.0:1788880182.929488:0:203392:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880182.929489:0:203392:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.929492:0:203392:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880182.929494:0:203392:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880182.929497:0:203392:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880182.929499:0:203392:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880182.929502:0:203392:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880182.929510:0:203392:0:(import.c:2149:obd_at_measure()) add 5 to 00000000ae179dea time=1788880182 v=5 (0 0 0 0) 00000100:00001000:2.0:1788880182.929514:0:203392:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=0 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880182.929517:0:203392:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880182.929518:0:203392:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880182.929520:0:203392:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880182.929525:0:203392:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/464 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/600/0 rc 0/0 job:'' uid:0 gid:0 projid:0 00010000:00000001:2.0:1788880182.929532:0:203392:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.929535:0:203392:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880182.929543:0:203392:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880182.929545:0:203392:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880182.929547:0:203392:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880182.929551:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/464 e 0 to 0 dl 1788880198 ref 2 fl Rpc:RQU/600/0 rc 0/0 job:'' uid:0 gid:0 projid:0 00000100:00000001:2.0:1788880182.929568:0:203392:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880182.929571:0:203392:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880182.929573:0:203392:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880182.929576:0:203392:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/464 e 0 to 0 dl 1788880198 ref 2 fl Interpret:RQU/600/0 rc 0/0 job:'' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788880182.929582:0:203392:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job mount.lustre:ea27006f-52a8-4007-b69a-dcac8279445a:203392:1875776780635648:192.168.204.111@tcp:33: 00000100:00000001:2.0:1788880182.929588:0:203392:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880182.929590:0:203392:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.929592:0:203392:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:2.0:1788880182.929594:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880182.929596:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/464 e 0 to 0 dl 1788880198 ref 2 fl Complete:RQU/600/0 rc 0/0 job:'' uid:0 gid:0 projid:0 00000100:00000001:2.0:1788880182.929602:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880182.929605:0:203392:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000126d8278. 00000100:00000001:2.0:1788880182.929608:0:203392:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:2.0:1788880182.929609:0:203392:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000002:00000200:2.0:1788880182.929614:0:203392:0:(mdc_request.c:172:mdc_getattr_common()) mode: 40755 00000002:00000001:2.0:1788880182.929617:0:203392:0:(mdc_request.c:194:mdc_getattr_common()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880182.929620:0:203392:0:(mdc_request.c:268:mdc_getattr()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880182.929622:0:203392:0:(lmv_obd.c:1635:lmv_getattr()) Process leaving (rc=0 : 0 : 0) 00000080:00000010:2.0:1788880182.929625:0:203392:0:(llite_lib.c:858:client_common_fill_super()) kfreed 'op_data': 392 at 00000000a581a743. 00000002:00000001:2.0:1788880182.929631:0:203392:0:(mdc_request.c:554:mdc_get_lustre_md()) Process entered 00000002:00000001:2.0:1788880182.929634:0:203392:0:(mdc_request.c:646:mdc_get_lustre_md()) Process leaving 00000020:00000001:2.0:1788880182.929636:0:203392:0:(cl_object.c:762:cl_env_obtain()) Process entered 00000020:00000010:2.0:1788880182.929654:0:203392:0:(cl_object.c:715:cl_env_new()) slab-alloced 'cle': 144 at 000000006f1031c1. 00000020:00000010:2.0:1788880182.929659:0:203392:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000e6a93e02. 00000020:00000010:2.0:1788880182.929664:0:203392:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000c0495ef7. 00000020:00000010:2.0:1788880182.929667:0:203392:0:(cl_object.c:1137:cl_key_init()) kmalloced '(value)': 72 at 00000000a3bf26bf. 00000008:00000010:2.0:1788880182.929678:0:203392:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 0000000052826747. 00020000:00000010:2.0:1788880182.929682:0:203392:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000b84b6ce0. 00008000:00000010:2.0:1788880182.929687:0:203392:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 000000001603dae8. 00000080:00000010:2.0:1788880182.929692:0:203392:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 000000008b72261e. 00000080:00000010:2.0:1788880182.929696:0:203392:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 000000009fafce3b. 00000020:00000010:2.0:1788880182.929699:0:203392:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000058b4ee38. 00000008:00000010:2.0:1788880182.929703:0:203392:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000752c82ca. 00020000:00000010:2.0:1788880182.929705:0:203392:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 000000000f82ec17. 00008000:00000010:2.0:1788880182.929709:0:203392:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 0000000066fffc68. 00000080:00000010:2.0:1788880182.929712:0:203392:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 000000009a966891. 00000020:00000001:2.0:1788880182.929715:0:203392:0:(cl_object.c:793:cl_env_obtain()) Process leaving (rc=18446617250042534008 : -126823667017608 : ffff8ca79060d878) 00000020:00001000:2.0:1788880182.929717:0:203392:0:(cl_object.c:831:cl_env_get()) 1@000000006f1031c1 00000080:00000001:2.0:1788880182.929719:0:203392:0:(vvp_dev.c:172:vvp_device_alloc()) Process entered 00000080:00000010:2.0:1788880182.929721:0:203392:0:(vvp_dev.c:174:vvp_device_alloc()) kmalloced '(vdv)': 72 at 000000001465fece. 00000080:00000010:2.0:1788880182.929726:0:203392:0:(vvp_dev.c:182:vvp_device_alloc()) kmalloced '(site)': 576 at 00000000a977081d. 00000020:00000001:2.0:1788880182.929728:0:203392:0:(lu_object.c:1137:lu_site_init()) Process entered 00000020:00000010:2.0:1788880182.929781:0:203392:0:(lu_object.c:1156:lu_site_init()) kmalloced '(s->ls_bkts)': 18432 at 00000000faa23ffb. 00000020:00000010:2.0:1788880182.929791:0:203392:0:(lprocfs_status.c:1447:lprocfs_stats_alloc()) alloc '(stats)': 608 at 00000000dda0c0cd. 00000020:00000010:2.0:1788880182.929795:0:203392:0:(lprocfs_status.c:1459:lprocfs_stats_alloc()) alloc '(stats->ls_cnt_header)': 192 at 00000000126d8278. 00000020:00000001:2.0:1788880182.929799:0:203392:0:(lu_object.c:1199:lu_site_init()) Process leaving (rc=0 : 0 : 0) 00000008:00000010:2.0:1788880182.929804:0:203392:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000db08619a. 00020000:00000010:2.0:1788880182.929806:0:203392:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 0000000048c442e4. 00008000:00000010:2.0:1788880182.929809:0:203392:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 000000007b0c4715. 00000080:00000010:2.0:1788880182.929812:0:203392:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 00000000502d0171. 00000080:00000010:2.0:1788880182.929814:0:203392:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 00000000bb6f1707. 00000008:00000010:2.0:1788880182.929826:0:203392:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000b47e82cb. 00020000:00000010:2.0:1788880182.929829:0:203392:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 00000000f00a81e2. 00008000:00000010:2.0:1788880182.929831:0:203392:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 000000009238e91b. 00000080:00000010:2.0:1788880182.929847:0:203392:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 00000000585cf905. 00000008:00000010:2.0:1788880182.929851:0:203392:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000e035dee2. 00020000:00000010:2.0:1788880182.929853:0:203392:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000247c01dd. 00008000:00000010:2.0:1788880182.929856:0:203392:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000634ba2cc. 00000080:00000010:2.0:1788880182.929858:0:203392:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 00000000edd0ead3. 00000080:00000010:2.0:1788880182.929865:0:203392:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 00000000fb25baf7. 00000008:00000010:2.0:1788880182.929869:0:203392:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 0000000014f5f732. 00020000:00000010:2.0:1788880182.929871:0:203392:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 000000003f2da66b. 00008000:00000010:2.0:1788880182.929873:0:203392:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 00000000dbc9c821. 00000080:00000010:2.0:1788880182.929876:0:203392:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 00000000b27bbe9e. 00000008:00000010:2.0:1788880182.929880:0:203392:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 00000000a9422caa. 00020000:00000010:2.0:1788880182.929881:0:203392:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 00000000e48f543b. 00008000:00000010:2.0:1788880182.929884:0:203392:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000eae75791. 00000080:00000010:2.0:1788880182.929886:0:203392:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 00000000bdd66818. 00000080:00000010:2.0:1788880182.929888:0:203392:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 00000000e0c908af. 00000008:00000010:2.0:1788880182.929891:0:203392:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 000000000ae13907. 00020000:00000010:2.0:1788880182.929893:0:203392:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 000000006f7fd20d. 00008000:00000010:2.0:1788880182.929895:0:203392:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 00000000a4c0564c. 00000080:00000010:2.0:1788880182.929897:0:203392:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 000000005e67cab5. 00000008:00000010:2.0:1788880182.929901:0:203392:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2864 at 000000008d7e3a78. 00020000:00000010:2.0:1788880182.929904:0:203392:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 0000000042819046. 00008000:00000010:2.0:1788880182.929906:0:203392:0:(echo_client.c:413:echo_thread_key_init()) slab-alloced 'info': 904 at 00000000e0ff896b. 00000080:00000010:2.0:1788880182.929908:0:203392:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 000000009c384b06. 00000080:00000010:2.0:1788880182.929910:0:203392:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 592 at 000000007fa847b1. 00000008:00000010:2.0:1788880182.929913:0:203392:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000936429d9. 00020000:00000010:2.0:1788880182.929915:0:203392:0:(lov_dev.c:117:lov_session_key_init()) slab-alloced 'info': 584 at 0000000065b7c3bf. 00008000:00000010:2.0:1788880182.929917:0:203392:0:(echo_client.c:438:echo_session_key_init()) slab-alloced 'session': 8 at 000000007f54063d. 00000080:00000010:2.0:1788880182.929919:0:203392:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 00000000c4ba6781. 00000080:00000001:2.0:1788880182.929924:0:203392:0:(vvp_dev.c:198:vvp_device_alloc()) Process leaving (rc=18446617250404476512 : -126823305075104 : ffff8ca7a5f3a660) 00000080:00000001:2.0:1788880182.929926:0:203392:0:(vvp_dev.c:207:vvp_device_init()) Process entered 00020000:00000001:2.0:1788880182.929929:0:203392:0:(lov_dev.c:169:lov_mdc_dev_init()) Process entered 00020000:00000010:2.0:1788880182.929933:0:203392:0:(lovsub_dev.c:84:lovsub_device_alloc()) kmalloced '(lsd)': 72 at 00000000a08cf606. 00020000:00000001:2.0:1788880182.929936:0:203392:0:(lovsub_dev.c:31:lovsub_device_init()) Process entered 00000002:00000001:2.0:1788880182.929938:0:203392:0:(mdc_dev.c:1758:mdc_device_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.929940:0:203392:0:(lovsub_dev.c:43:lovsub_device_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.929943:0:203392:0:(lov_dev.c:177:lov_mdc_dev_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.929944:0:203392:0:(lov_dev.c:169:lov_mdc_dev_init()) Process entered 00020000:00000010:2.0:1788880182.929946:0:203392:0:(lovsub_dev.c:84:lovsub_device_alloc()) kmalloced '(lsd)': 72 at 000000002810fe3e. 00020000:00000001:2.0:1788880182.929954:0:203392:0:(lovsub_dev.c:31:lovsub_device_init()) Process entered 00000002:00000001:2.0:1788880182.929956:0:203392:0:(mdc_dev.c:1758:mdc_device_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.929957:0:203392:0:(lovsub_dev.c:43:lovsub_device_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.929959:0:203392:0:(lov_dev.c:177:lov_mdc_dev_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000010:2.0:1788880182.929962:0:203392:0:(lovsub_dev.c:84:lovsub_device_alloc()) kmalloced '(lsd)': 72 at 00000000a0799007. 00020000:00000001:2.0:1788880182.929963:0:203392:0:(lovsub_dev.c:31:lovsub_device_init()) Process entered 00000008:00000001:2.0:1788880182.929965:0:203392:0:(osc_dev.c:155:osc_device_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.929967:0:203392:0:(lovsub_dev.c:43:lovsub_device_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:2.0:1788880182.929969:0:203392:0:(lov_dev.c:281:lov_device_init()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.929971:0:203392:0:(vvp_dev.c:218:vvp_device_init()) Process leaving (rc=0 : 0 : 0) 00000020:00001000:2.0:1788880182.929973:0:203392:0:(cl_object.c:925:cl_env_put()) 1@000000006f1031c1 00000020:00000010:2.0:1788880182.929976:0:203392:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000c0495ef7. 00000020:00000010:2.0:1788880182.929980:0:203392:0:(cl_object.c:1137:cl_key_fini()) kfreed 'info': 72 at 00000000a3bf26bf. 00000008:00000010:2.0:1788880182.929983:0:203392:0:(osc_dev.c:100:osc_key_fini()) slab-freed '(info)': 2864 at 0000000052826747. 00020000:00000010:2.0:1788880182.929986:0:203392:0:(lov_dev.c:95:lov_key_fini()) slab-freed '(info)': 192 at 00000000b84b6ce0. 00008000:00000010:2.0:1788880182.929989:0:203392:0:(echo_client.c:424:echo_thread_key_fini()) slab-freed '(info)': 904 at 000000001603dae8. 00000080:00000010:2.0:1788880182.929993:0:203392:0:(vvp_dev.c:80:ll_thread_key_fini()) slab-freed '(lti)': 168 at 000000008b72261e. 00000080:00000010:2.0:1788880182.929995:0:203392:0:(vvp_dev.c:130:vvp_thread_key_fini()) slab-freed '(vti)': 592 at 000000009fafce3b. 00000020:00000010:2.0:1788880182.929998:0:203392:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000e6a93e02. 00000008:00000010:2.0:1788880182.930001:0:203392:0:(osc_dev.c:125:osc_session_fini()) slab-freed '(info)': 352 at 00000000752c82ca. 00020000:00000010:2.0:1788880182.930004:0:203392:0:(lov_dev.c:136:lov_session_key_fini()) slab-freed '(info)': 584 at 000000000f82ec17. 00008000:00000010:2.0:1788880182.930006:0:203392:0:(echo_client.c:449:echo_session_key_fini()) slab-freed '(session)': 8 at 0000000066fffc68. 00000080:00000010:2.0:1788880182.930009:0:203392:0:(vvp_dev.c:105:vvp_session_key_fini()) slab-freed '(session)': 240 at 000000009a966891. 00000020:00000010:2.0:1788880182.930010:0:203392:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000058b4ee38. 00000020:00000010:2.0:1788880182.930012:0:203392:0:(cl_object.c:750:cl_env_fini()) slab-freed '(cle)': 144 at 000000006f1031c1. 00000080:00000001:2.0:1788880182.930015:0:203392:0:(vvp_dev.c:302:cl_sb_init()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.930018:0:203392:0:(lcommon_cl.c:275:cl_fid_build_ino()) Process leaving (rc=144115188193296385 : 144115188193296385 : 200000007000001) 00000080:00000001:2.0:1788880182.930020:0:203392:0:(namei.c:113:ll_iget()) Process entered 00000080:00000010:2.0:1788880182.930042:0:203392:0:(super25.c:41:ll_alloc_inode()) slab-alloced 'lli': 2072 at 0000000003d40655. 00000080:00000001:2.0:1788880182.930052:0:203392:0:(llite_lib.c:3360:ll_read_inode2()) Process entered 00000080:00200000:2.0:1788880182.930053:0:203392:0:(llite_lib.c:3363:ll_read_inode2()) VFS Op:inode=[0x200000007:0x1:0x0](00000000f8fa31aa) 00000080:00000001:2.0:1788880182.930056:0:203392:0:(llite_lib.c:1965:ll_update_lsm_md()) Process entered 00000080:00000002:2.0:1788880182.930057:0:203392:0:(llite_lib.c:1969:ll_update_lsm_md()) update lsm_obj 0000000000000000 of [0x200000007:0x1:0x0] 00000080:00000001:2.0:1788880182.930060:0:203392:0:(llite_lib.c:1986:ll_update_lsm_md()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.930062:0:203392:0:(lcommon_cl.c:275:cl_fid_build_ino()) Process leaving (rc=144115188193296385 : 144115188193296385 : 200000007000001) 00000080:00000001:2.0:1788880182.930069:0:203392:0:(lcommon_cl.c:294:cl_fid_build_gen()) Process leaving (rc=33554432 : 33554432 : 2000000) 00000080:00200000:2.0:1788880182.930073:0:203392:0:(llite_lib.c:3126:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 4096 00000080:00000001:2.0:1788880182.930075:0:203392:0:(llite_lib.c:3389:ll_read_inode2()) Process leaving 00000080:00000001:2.0:1788880182.930077:0:203392:0:(namei.c:156:ll_iget()) Process leaving (rc=18446617250750562504 : -126822958989112 : ffff8ca7ba9480c8) 00800000:00000001:2.0:1788880182.930080:0:203392:0:(lmv_obd.c:4159:lmv_put_lustre_md()) Process entered 00800000:00000001:2.0:1788880182.930082:0:203392:0:(lmv_obd.c:4166:lmv_put_lustre_md()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880182.930085:0:203392:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880182.930089:0:203392:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d838cab4 x1875776780635648/t0(0) o33->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:23/10 lens 448/464 e 0 to 0 dl 1788880198 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'' uid:0 gid:0 projid:0 00000100:00000001:2.0:1788880182.930096:0:203392:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880182.930099:0:203392:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880182.930101:0:203392:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 131072 at 00000000f19652a7. 02000000:00000001:2.0:1788880182.930121:0:203392:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880182.930123:0:203392:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880182.930125:0:203392:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880182.930127:0:203392:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880182.930129:0:203392:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 0000000078217d0d. 02000000:00000001:2.0:1788880182.930134:0:203392:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880182.930136:0:203392:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880182.930137:0:203392:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d838cab4. 00000100:00000001:2.0:1788880182.930140:0:203392:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880182.930141:0:203392:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000080:00000010:2.0:1788880182.930148:0:203392:0:(dcache.c:138:ll_d_init()) kmalloced '(lld)': 24 at 000000008214bad1. 00000080:00000001:2.0:1788880182.930151:0:203392:0:(obd_class.h:836:obd_get_uuid()) Process entered 00800000:00000001:2.0:1788880182.930153:0:203392:0:(obd_class.h:836:obd_get_uuid()) Process entered 00800000:00000001:2.0:1788880182.930155:0:203392:0:(obd_class.h:843:obd_get_uuid()) Process leaving (rc=18446617250653066664 : -126823056484952 : ffff8ca7b4c4d5a8) 00000080:00000001:2.0:1788880182.930158:0:203392:0:(obd_class.h:843:obd_get_uuid()) Process leaving (rc=18446617250653066664 : -126823056484952 : ffff8ca7b4c4d5a8) 00000080:00000010:2.0:1788880182.930161:0:203392:0:(llite_lib.c:947:client_common_fill_super()) kfreed 'data': 192 at 0000000000211877. 00000080:00000010:2.0:1788880182.930164:0:203392:0:(llite_lib.c:948:client_common_fill_super()) kfreed 'osfs': 144 at 0000000055a084bf. 00000080:00000001:2.0:1788880182.930166:0:203392:0:(obd_class.h:511:obd_get_info()) Process entered 00020000:00000001:2.0:1788880182.930168:0:203392:0:(lov_obd.c:1158:lov_get_info()) Process entered 00020000:00000001:2.0:1788880182.930172:0:203392:0:(lov_obd.c:1209:lov_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.930173:0:203392:0:(obd_class.h:525:obd_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880182.930190:0:203392:0:(llite_lib.c:992:client_common_fill_super()) Process leaving (rc=0 : 0 : 0) 00000080:00000010:2.0:1788880182.930193:0:203392:0:(llite_lib.c:1645:ll_fill_super()) kfreed 'md': 33 at 0000000057834ed5. 00000080:00000010:2.0:1788880182.930201:0:203392:0:(llite_lib.c:1647:ll_fill_super()) kfreed 'dt': 33 at 00000000f0a428ff. 00000080:00000010:2.0:1788880182.930204:0:203392:0:(llite_lib.c:1655:ll_fill_super()) kfreed 'cfg': 88 at 00000000f51165b6. 00000080:02000400:2.0:1788880182.930207:0:203392:0:(llite_lib.c:1661:ll_fill_super()) Mounted lustre-client 00000080:00000001:2.0:1788880182.931729:0:203392:0:(llite_lib.c:1662:ll_fill_super()) Process leaving (rc=0 : 0 : 0) 00000080:00000004:2.0:1788880182.931735:0:203392:0:(super25.c:181:lustre_fill_super()) 192.168.204.111@tcp:/lustre: Mount complete 00000080:00000001:0.0:1788880182.932542:0:1:0:(llite_lib.c:4244:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:0.0:1788880182.942525:0:1:0:(llite_lib.c:4244:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880182.974536:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880182.974543:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880182.974547:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880183.998364:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880183.998367:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880183.998370:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880184.290910:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 0000000016be5b21 00000080:00200008:3.0:1788880184.290918:0:203453:0:(file.c:6283:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](00000000f8fa31aa), request_mask 2047, flags 0, foreign 0 00000080:00000001:3.0:1788880184.290926:0:203453:0:(file.c:6176:ll_inode_revalidate()) Process entered 00000020:00000001:3.0:1788880184.290927:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 00000000f165bcf1 00000080:00200000:3.0:1788880184.290929:0:203453:0:(file.c:6178:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000f8fa31aa),name=/ 00000080:00000010:3.0:1788880184.290934:0:203453:0:(llite_lib.c:4055:ll_prep_md_op_data()) kmalloced '(op_data)': 392 at 00000000115c4ce2. 00000080:00000001:3.0:1788880184.290940:0:203453:0:(namei.c:944:ll_intent_lock()) Process entered 00000020:00000010:3.0:1788880184.290945:0:203453:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 640 at 00000000352996e3. 00800000:00000001:3.0:1788880184.290949:0:203453:0:(lmv_intent.c:566:lmv_intent_lock()) Process entered 00000020:00000001:3.0:1788880184.290951:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 0000000023d77907 00800000:00000002:3.0:1788880184.290953:0:203453:0:(lmv_intent.c:573:lmv_intent_lock()) INTENT LOCK 'getattr' for [0x200000007:0x1:0x0] '' on [0x200000007:0x1:0x0] 00800000:00000001:3.0:1788880184.290956:0:203453:0:(lmv_intent.c:447:lmv_intent_lookup()) Process entered 00800000:00000001:3.0:1788880184.290958:0:203453:0:(lmv_fld.c:38:lmv_fld_lookup()) Process entered 80000000:00000001:3.0:1788880184.290961:0:203453:0:(fld_request.c:427:fld_client_lookup()) Process entered 80000000:00000001:3.0:1788880184.290962:0:203453:0:(fld_cache.c:479:fld_cache_lookup()) Process entered 80000000:00000001:3.0:1788880184.290965:0:203453:0:(fld_cache.c:498:fld_cache_lookup()) Process leaving (rc=0 : 0 : 0) 80000000:00000001:3.0:1788880184.290967:0:203453:0:(fld_request.c:434:fld_client_lookup()) Process leaving (rc=0 : 0 : 0) 00800000:00000002:3.0:1788880184.290968:0:203453:0:(lmv_fld.c:57:lmv_fld_lookup()) FLD lookup got mds #0 for fid=[0x200000007:0x1:0x0] 00800000:00000001:3.0:1788880184.290971:0:203453:0:(lmv_fld.c:65:lmv_fld_lookup()) Process leaving (rc=0 : 0 : 0) 00800000:00000002:3.0:1788880184.290974:0:203453:0:(lmv_intent.c:500:lmv_intent_lookup()) LOOKUP_INTENT with fid1=[0x200000007:0x1:0x0], fid2=[0x200000007:0x1:0x0], name='' -> mds #0 00000020:00000010:3.0:1788880184.290983:0:203453:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 640 at 000000002a6ca715. 00000002:00000001:3.0:1788880184.290987:0:203453:0:(mdc_locks.c:1500:mdc_intent_lock()) Process entered 00000020:00000001:3.0:1788880184.290989:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 00000000395658e9 00000002:00010000:3.0:1788880184.290990:0:203453:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00000002:00000001:3.0:1788880184.290993:0:203453:0:(mdc_locks.c:1404:mdc_revalidate_lock()) Process entered 00000002:00000001:3.0:1788880184.290995:0:203453:0:(mdc_locks.c:135:mdc_lock_match()) Process entered 00010000:00000001:3.0:1788880184.290997:0:203453:0:(ldlm_lock.c:1446:ldlm_lock_match_with_skip()) Process entered 00010000:00000001:3.0:1788880184.291003:0:203453:0:(ldlm_lock.c:1461:ldlm_lock_match_with_skip()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880184.291006:0:203453:0:(mdc_locks.c:141:mdc_lock_match()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880184.291008:0:203453:0:(mdc_locks.c:1456:mdc_revalidate_lock()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880184.291010:0:203453:0:(mdc_locks.c:1025:mdc_enqueue_base()) Process entered 00000002:00000001:3.0:1788880184.291012:0:203453:0:(mdc_locks.c:621:mdc_intent_getattr_pack()) Process entered 00000100:00000010:3.0:1788880184.291016:0:203453:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880184.291019:0:203453:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00010000:00000001:3.0:1788880184.291025:0:203453:0:(ldlm_request.c:923:ldlm_prep_elc_req()) Process entered 00010000:00000001:3.0:1788880184.291029:0:203453:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:3.0:1788880184.291032:0:203453:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.291034:0:203453:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880184.291037:0:203453:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880184.291040:0:203453:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880184.291045:0:203453:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 000000006fc9dc72. 00000100:00000001:3.0:1788880184.291052:0:203453:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880184.291056:0:203453:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:3.0:1788880184.291058:0:203453:0:(ldlm_request.c:2459:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880184.291059:0:203453:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:3.0:1788880184.291060:0:203453:0:(ldlm_request.c:2459:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880184.291062:0:203453:0:(ldlm_request.c:979:ldlm_prep_elc_req()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880184.291067:0:203453:0:(mdc_locks.c:692:mdc_intent_getattr_pack()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00010000:00000001:3.0:1788880184.291070:0:203453:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:3.0:1788880184.291072:0:203453:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:3.0:1788880184.291075:0:203453:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at 0000000049bd8a46. 00010000:00000010:3.0:1788880184.291078:0:203453:0:(ldlm_resource.c:1487:ldlm_resource_inodebits_new()) kmalloced '(res->lr_ibits_queues)': 112 at 00000000bb86fbaf. 00010000:00000001:3.0:1788880184.291083:0:203453:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:3.0:1788880184.291085:0:203453:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 0000000046030e36. 00000020:00000010:3.0:1788880184.291088:0:203453:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 128 at 00000000285741dd. 00000020:00000001:3.0:1788880184.291096:0:203453:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:3.0:1788880184.291099:0:203453:0:(lustre_handles.c:93:class_handle_hash()) added object 0000000046030e36 with handle 0x805d548a36a886ba to hash 00000020:00000001:3.0:1788880184.291101:0:203453:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:3.0:1788880184.291103:0:203453:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:3.0:1788880184.291106:0:203453:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:3.0:1788880184.291110:0:203453:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:3.0:1788880184.291112:0:203453:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880184.291115:0:203453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 203453 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788880184.291123:0:203453:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 203453 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788880184.291133:0:203453:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 203453 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00000001:3.0:1788880184.291138:0:203453:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:3.0:1788880184.291139:0:203453:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:3.0:1788880184.291146:0:203453:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000fa054719. 00000100:00000001:3.0:1788880184.291148:0:203453:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063673792 : -126823645877824 : ffff8ca791a369c0) 00000100:00000001:3.0:1788880184.291151:0:203453:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880184.291152:0:203453:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000001:3.0:1788880184.291156:0:203453:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880184.291158:0:203453:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.291160:0:203453:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:3.0:1788880184.291162:0:203453:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880184.291171:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl New:QU/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:3.0:1788880184.291179:0:203453:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880184.291181:0:203453:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880184.291185:0:203453:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880184.291187:0:203453:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880184.291189:0:203453:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880184.291192:0:203453:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job lfs:ea27006f-52a8-4007-b69a-dcac8279445a:203453:1875776780635776:192.168.204.111@tcp:101: 00000100:00000001:3.0:1788880184.291200:0:203453:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880184.291205:0:203453:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880184.291207:0:203453:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880184.291210:0:203453:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880184.291213:0:203453:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 2048 at 00000000fb4f27d2. 02000000:00000001:3.0:1788880184.291215:0:203453:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880184.291220:0:203453:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006f95a419. 00000400:00000010:3.0:1788880184.291226:0:203453:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f73021f1. 00000100:00000200:3.0:1788880184.291231:0:203453:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 2048 bytes, xid 1875776780635776, portal 10 00000100:00000001:3.0:1788880184.291359:0:203453:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880184.291360:0:203453:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000040:3.0:1788880184.291367:0:203453:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=600 req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/1152 e 0 to 0 dl 1788880200 ref 3 fl Rpc:r/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:3.0:1788880184.291375:0:203453:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880184.291378:0:203453:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880184.291381:0:203453:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005f2b805f. 00000100:00000200:3.0:1788880184.291384:0:203453:0:(niobuf.c:94:ptl_send_buf()) Sending 576 bytes to portal 12, xid 1875776780635776, offset 0 00000400:00000200:3.0:1788880184.291391:0:203453:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880184.291406:0:203453:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880184.291412:0:203453:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880184.291417:0:203453:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880184.291421:0:203453:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880184.291425:0:203453:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880184.291430:0:203453:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [33:33:256:4294967295] 192.168.204.111@tcp LPNI seq info [33:33:8:4294967295] 00000400:00000200:3.0:1788880184.291442:0:203453:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880184.291447:0:203453:0:(socklnd_cb.c:931:ksocknal_send()) sending 576 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880184.291452:0:203453:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000800:00000200:3.0:1788880184.291458:0:203453:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880184.291465:0:203453:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880184.291469:0:203453:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 672 niov 1 nkiov 1 00000100:00000001:3.0:1788880184.291483:0:203453:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880184.291491:0:203453:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 3170911 usecs (3171 jiffies) 00000100:00000001:3.0:1788880184.291495:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305818568 00000100:00000001:3.0:1788880184.291498:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305819094 00000100:00000001:3.0:1788880184.291500:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305818568 00000100:00000001:3.0:1788880184.291503:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305819095 00000100:00000001:3.0:1788880184.291505:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305818568 00000100:00000001:3.0:1788880184.291507:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305819095 00000100:00000001:3.0:1788880184.291510:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305818568 00000100:00000001:3.0:1788880184.291511:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305819095 00000100:00000001:3.0:1788880184.291513:0:203453:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880184.291515:0:203453:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.291517:0:203453:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880184.291519:0:203453:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:3.0:1788880184.291521:0:203453:0:(client.c:2702:ptlrpc_set_wait()) set 00000000fa054719 going to sleep for 11 seconds 00000100:00000001:3.0:1788880184.291524:0:203453:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880184.291526:0:203453:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880184.291580:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880184.291584:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000400:00000200:0.0:1788880184.291588:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880184.291592:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880184.291596:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005f2b805f 00000400:00000010:0.0:1788880184.291597:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005f2b805f. 00000100:00000001:0.0:1788880184.291600:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880184.291603:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/1152 e 0 to 0 dl 1788880200 ref 3 fl Rpc:r/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:0.0:1788880184.291610:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880184.291612:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/1152 e 0 to 0 dl 1788880200 ref 3 fl Rpc:Qr/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:0.0:1788880184.291616:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880184.291617:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880184.293697:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880184.293710:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880184.293713:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880184.293714:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880184.293723:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880184.293736:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 688 into portal 10 MB=0x6aa0250b00e80 00000400:00000200:3.0:1788880184.293825:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 688/688 into md 0x1fd [1] + 224 00000800:00000001:3.0:1788880184.293833:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880184.293840:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880184.293842:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880184.293846:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880184.293852:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880184.293857:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/1152 e 0 to 0 dl 1788880200 ref 2 fl Rpc:Qr/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000040:3.0:1788880184.293882:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=600 mlen=688 offset=224 replen=1152 req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/1152 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQ/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:3.0:1788880184.293901:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880184.293909:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880184.293912:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880184.293953:0:203453:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880184.293960:0:203453:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880184.293962:0:203453:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880184.293964:0:203453:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006f95a419. 00000400:00000200:0.0:1788880184.293968:0:203453:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f73021f1 00000400:00000010:0.0:1788880184.293974:0:203453:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f73021f1. 00000100:00000001:0.0:1788880184.293977:0:203453:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880184.293982:0:203453:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/1152 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQ/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000200:0.0:1788880184.293991:0:203453:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/1152 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQU/600/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 00000100:00000001:0.0:1788880184.293997:0:203453:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880184.293998:0:203453:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880184.294001:0:203453:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880184.294004:0:203453:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880184.294005:0:203453:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880184.294007:0:203453:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880184.294010:0:203453:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:0.0:1788880184.294017:0:203453:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 00000000bc786407. 00000100:00001000:0.0:1788880184.294023:0:203453:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=2 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880184.294027:0:203453:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=2 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880184.294029:0:203453:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880184.294030:0:203453:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.294032:0:203453:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880184.294037:0:203453:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880184.294040:0:203453:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880184.294041:0:203453:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880184.294043:0:203453:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880184.294044:0:203453:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880184.294047:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/688 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQU/600/0 rc 0/0 job:'' uid:0 gid:0 projid:0 00000100:00000001:0.0:1788880184.294053:0:203453:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880184.294055:0:203453:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880184.294057:0:203453:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880184.294060:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/688 e 0 to 0 dl 1788880200 ref 2 fl Interpret:RQU/600/0 rc 0/0 job:'' uid:0 gid:0 projid:0 00000100:00100000:0.0:1788880184.294066:0:203453:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job lfs:ea27006f-52a8-4007-b69a-dcac8279445a:203453:1875776780635776:192.168.204.111@tcp:101: 00000100:00000001:0.0:1788880184.294077:0:203453:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880184.294079:0:203453:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880184.294081:0:203453:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:0.0:1788880184.294083:0:203453:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880184.294086:0:203453:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/688 e 0 to 0 dl 1788880200 ref 2 fl Complete:RQU/600/0 rc 0/0 job:'' uid:0 gid:0 projid:0 00000100:00000001:0.0:1788880184.294092:0:203453:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880184.294094:0:203453:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000fa054719. 00000100:00000001:0.0:1788880184.294097:0:203453:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:0.0:1788880184.294098:0:203453:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.294101:0:203453:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:0.0:1788880184.294104:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.294106:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.294109:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=4 00000020:00000001:0.0:1788880184.294112:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294114:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000040:0.0:1788880184.294120:0:203453:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 0000000046030e36, remote cookie: 0x92f11317f60a7941, flags: 0x1001 00010000:00010000:0.0:1788880184.294123:0:203453:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788880184.294131:0:203453:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880184.294138:0:203453:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:0.0:1788880184.294142:0:203453:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:0.0:1788880184.294143:0:203453:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:0.0:1788880184.294145:0:203453:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:0.0:1788880184.294146:0:203453:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:0.0:1788880184.294148:0:203453:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x200000007:0x1:0x0].0x0 (0000000049bd8a46) refcount = 1 00010000:00010000:0.0:1788880184.294151:0:203453:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880184.294165:0:203453:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00000020:00000010:0.0:1788880184.294168:0:203453:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 448 at 00000000b5c40084. 00010000:00000001:0.0:1788880184.294171:0:203453:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:0.0:1788880184.294172:0:203453:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:0.0:1788880184.294175:0:203453:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:0.0:1788880184.294177:0:203453:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880184.294179:0:203453:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880184.294184:0:203453:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:0.0:1788880184.294185:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294187:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880184.294188:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294188:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880184.294190:0:203453:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:0.0:1788880184.294194:0:203453:0:(mdc_locks.c:778:mdc_finish_enqueue()) Process entered 00010000:00000001:0.0:1788880184.294195:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.294197:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.294198:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=3 00000020:00000001:0.0:1788880184.294199:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294201:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294203:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.294204:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.294205:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=4 00000020:00000001:0.0:1788880184.294207:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294216:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294219:0:203453:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:0.0:1788880184.294221:0:203453:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880184.294223:0:203453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880184.294265:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294266:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880184.294268:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.294270:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.294271:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=5 00000020:00000001:0.0:1788880184.294273:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294275:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294278:0:203453:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:0.0:1788880184.294279:0:203453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880184.294285:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294286:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:0.0:1788880184.294287:0:203453:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880184.294293:0:203453:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:0.0:1788880184.294294:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294295:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880184.294296:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294297:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000002:00100000:0.0:1788880184.294302:0:203453:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/688 e 0 to 0 dl 1788880200 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'' uid:0 gid:0 projid:0 00010000:00000001:0.0:1788880184.294310:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.294311:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.294312:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=3 00000020:00000001:0.0:1788880184.294313:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294314:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294316:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294316:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000002:00000001:0.0:1788880184.294317:0:203453:0:(mdc_locks.c:981:mdc_finish_enqueue()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:0.0:1788880184.294322:0:203453:0:(mdc_locks.c:1186:mdc_enqueue_base()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:0.0:1788880184.294325:0:203453:0:(mdc_locks.c:1288:mdc_finish_intent_lock()) Process entered 00010000:00000001:0.0:1788880184.294326:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.294327:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.294328:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=3 00000020:00000001:0.0:1788880184.294329:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294330:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00000002:00010000:0.0:1788880184.294331:0:203453:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880184.294336:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294336:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880184.294337:0:203453:0:(ldlm_lock.c:1446:ldlm_lock_match_with_skip()) Process entered 00010000:00000001:0.0:1788880184.294338:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.294339:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.294339:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=3 00000020:00000001:0.0:1788880184.294340:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294341:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000040:0.0:1788880184.294345:0:203453:0:(ldlm_resource.c:1656:ldlm_resource_getref()) getref res: 0000000049bd8a46 count: 2 00010000:00000001:0.0:1788880184.294346:0:203453:0:(ldlm_resource.c:1595:ldlm_resource_get()) Process leaving via found (rc=0 : 0 : 0x0) 00010000:00010000:0.0:1788880184.294349:0:203453:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns 00000000c56413c7 type 13 mode 4 res 8589934599/1 (0 0) 00010000:00000001:0.0:1788880184.294350:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294351:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000002:00000001:0.0:1788880184.294351:0:203453:0:(mdc_locks.c:1383:mdc_finish_intent_lock()) Process leaving 00000020:00000001:0.0:1788880184.294352:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 000000006f47e987 00000002:00002000:0.0:1788880184.294353:0:203453:0:(mdc_locks.c:1388:mdc_finish_intent_lock()) D_IT dentry= intent=getattr status=0 disp=0xb: rc = 0 00000002:00000001:0.0:1788880184.294355:0:203453:0:(mdc_locks.c:1543:mdc_intent_lock()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:0.0:1788880184.294357:0:203453:0:(lmv_intent.c:556:lmv_intent_lookup()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:0.0:1788880184.294359:0:203453:0:(lmv_intent.c:606:lmv_intent_lock()) Process leaving (rc=0 : 0 : 0) 00000080:00200000:0.0:1788880184.294361:0:203453:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00000001:0.0:1788880184.294363:0:203453:0:(namei.c:990:ll_intent_lock()) Process leaving (rc=0 : 0 : 0) 00000080:00000010:0.0:1788880184.294366:0:203453:0:(llite_lib.c:4196:ll_finish_md_op_data()) kfreed 'op_data': 392 at 00000000115c4ce2. 00000080:00000001:0.0:1788880184.294369:0:203453:0:(dcache.c:220:ll_revalidate_it_finish()) Process entered 00000080:00000001:0.0:1788880184.294371:0:203453:0:(llite_lib.c:3837:ll_prep_inode()) Process entered 00000002:00000001:0.0:1788880184.294377:0:203453:0:(mdc_request.c:554:mdc_get_lustre_md()) Process entered 00800000:00000001:0.0:1788880184.294381:0:203453:0:(lmv_obd.c:3922:lmv_stripe_object_create()) Process entered 00800000:00000010:0.0:1788880184.294384:0:203453:0:(lmv_obd.c:3901:lmv_stripe_object_alloc()) kmalloced '(lsm_obj)': 56 at 00000000a6e808b7. 00800000:00000001:0.0:1788880184.294385:0:203453:0:(lmv_obd.c:3906:lmv_stripe_object_alloc()) Process leaving (rc=18446617250279283776 : -126823430267840 : ffff8ca79e7d5c40) 00800000:00000001:0.0:1788880184.294386:0:203453:0:(lmv_obd.c:3983:lmv_stripe_object_create()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:0.0:1788880184.294388:0:203453:0:(mdc_request.c:646:mdc_get_lustre_md()) Process leaving 00000080:00000001:0.0:1788880184.294390:0:203453:0:(llite_lib.c:1965:ll_update_lsm_md()) Process entered 00000080:00000002:0.0:1788880184.294391:0:203453:0:(llite_lib.c:1969:ll_update_lsm_md()) update lsm_obj 0000000000000000 of [0x200000007:0x1:0x0] 00000080:00000001:0.0:1788880184.294393:0:203453:0:(llite_lib.c:1920:ll_update_default_lsm_md()) Process entered 00000080:00000002:0.0:1788880184.294394:0:203453:0:(lustre_lmv.h:160:lmv_stripe_object_dump()) dump LMV: magic=0xcd30cd0 refs=1 count=1 index=4294967295 hash=invalid:0x0 max_inherit=0 max_inherit_rr=3 version=0 migrate_offset=0 migrate_hash=invalid:0 pool= 00000080:00000001:0.0:1788880184.294397:0:203453:0:(llite_lib.c:1955:ll_update_default_lsm_md()) Process leaving 00000080:00000001:0.0:1788880184.294398:0:203453:0:(llite_lib.c:1986:ll_update_lsm_md()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:0.0:1788880184.294401:0:203453:0:(lcommon_cl.c:275:cl_fid_build_ino()) Process leaving (rc=144115188193296385 : 144115188193296385 : 200000007000001) 00000080:00000001:0.0:1788880184.294403:0:203453:0:(lcommon_cl.c:294:cl_fid_build_gen()) Process leaving (rc=33554432 : 33554432 : 2000000) 00000080:00200000:0.0:1788880184.294406:0:203453:0:(llite_lib.c:3126:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 4096 00010000:00000001:0.0:1788880184.294408:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.294409:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.294410:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=3 00000020:00000001:0.0:1788880184.294411:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294412:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294413:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294414:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000080:00000001:0.0:1788880184.294414:0:203453:0:(llite_lib.c:3935:ll_prep_inode()) Process leaving via out (rc=0 : 0 : 0x0) 00800000:00000001:0.0:1788880184.294416:0:203453:0:(lmv_obd.c:4159:lmv_put_lustre_md()) Process entered 00800000:00000001:0.0:1788880184.294417:0:203453:0:(lmv_obd.c:4166:lmv_put_lustre_md()) Process leaving (rc=0 : 0 : 0) 00000080:00010000:0.0:1788880184.294418:0:203453:0:(llite_internal.h:2014:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000f8fa31aa) for lock 0x805d548a36a886ba 00800000:00000001:0.0:1788880184.294420:0:203453:0:(lmv_obd.c:4079:lmv_set_lock_data()) Process entered 00000002:00000001:0.0:1788880184.294421:0:203453:0:(mdc_locks.c:94:mdc_set_lock_data()) Process entered 00010000:00000001:0.0:1788880184.294422:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.294422:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.294423:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=3 00000020:00000001:0.0:1788880184.294424:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294425:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294429:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294429:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000002:00000001:0.0:1788880184.294430:0:203453:0:(mdc_locks.c:122:mdc_set_lock_data()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:0.0:1788880184.294431:0:203453:0:(lmv_obd.c:4084:lmv_set_lock_data()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:0.0:1788880184.294440:0:203453:0:(dcache.c:242:ll_revalidate_it_finish()) Process leaving (rc=0 : 0 : 0) 00000080:00010000:0.0:1788880184.294441:0:203453:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000f8fa31aa) 00000080:00010000:0.0:1788880184.294443:0:203453:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x805d548a36a886ba from it 00000000b77e1de1 00010000:00000001:0.0:1788880184.294444:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.294445:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.294445:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=3 00000020:00000001:0.0:1788880184.294446:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294447:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.294449:0:203453:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:0.0:1788880184.294450:0:203453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880184.294453:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294454:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880184.294454:0:203453:0:(ldlm_lock.c:302:ldlm_lock_add_to_lru()) Process entered 00010000:00000001:0.0:1788880184.294456:0:203453:0:(ldlm_lock.c:306:ldlm_lock_add_to_lru()) Process leaving 00010000:00010000:0.0:1788880184.294457:0:203453:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00000001:0.0:1788880184.294461:0:203453:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1788880184.294463:0:203453:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1788880184.294464:0:203453:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1788880184.294466:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294467:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880184.294468:0:203453:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.294470:0:203453:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1788880184.294471:0:203453:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.294472:0:203453:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.294474:0:203453:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.294474:0:203453:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:0.0:1788880184.294478:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.294479:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000100:00000001:0.0:1788880184.294480:0:203453:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880184.294482:0:203453:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780635776/t0(0) o101->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 576/688 e 0 to 0 dl 1788880200 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'' uid:0 gid:0 projid:0 00000100:00000001:0.0:1788880184.294486:0:203453:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880184.294488:0:203453:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880184.294490:0:203453:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 2048 at 00000000fb4f27d2. 02000000:00000001:0.0:1788880184.294492:0:203453:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880184.294494:0:203453:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880184.294495:0:203453:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880184.294497:0:203453:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880184.294499:0:203453:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 000000006fc9dc72. 02000000:00000001:0.0:1788880184.294502:0:203453:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880184.294503:0:203453:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880184.294504:0:203453:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:0.0:1788880184.294506:0:203453:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880184.294507:0:203453:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000080:00000001:0.0:1788880184.294510:0:203453:0:(file.c:6236:ll_merge_md_attr()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:0.0:1788880184.294514:0:203453:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 1856 at 000000006330c641. 00000020:00000001:0.0:1788880184.294516:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 0000000016be5b21 00000080:00000008:0.0:1788880184.294517:0:203453:0:(file.c:6459:ll_getattr_dentry()) COMPLETED file /:[0x200000007:0x1:0x0](00000000f8fa31aa), request_mask 2047, flags 0, foreign 0 00000080:00000001:0.0:1788880184.294987:0:203453:0:(llite_lib.c:4244:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880184.295008:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 0000000016be5b21 00000080:00200008:0.0:1788880184.295009:0:203453:0:(file.c:6283:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](00000000f8fa31aa), request_mask 0, flags 0, foreign 0 00000080:00000001:0.0:1788880184.295013:0:203453:0:(file.c:6176:ll_inode_revalidate()) Process entered 00000020:00000001:0.0:1788880184.295014:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 00000000f165bcf1 00000080:00200000:0.0:1788880184.295015:0:203453:0:(file.c:6178:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000f8fa31aa),name=/ 00000080:00000010:0.0:1788880184.295018:0:203453:0:(llite_lib.c:4055:ll_prep_md_op_data()) kmalloced '(op_data)': 392 at 0000000030a8fe2c. 00800000:00000002:0.0:1788880184.295020:0:203453:0:(lmv_obd.c:3994:lmv_stripe_object_get()) get 00000000a6e808b7 2 00000080:00000001:0.0:1788880184.295022:0:203453:0:(namei.c:944:ll_intent_lock()) Process entered 00000020:00000010:0.0:1788880184.295024:0:203453:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 640 at 0000000025f082c3. 00800000:00000001:0.0:1788880184.295026:0:203453:0:(lmv_intent.c:566:lmv_intent_lock()) Process entered 00000020:00000001:0.0:1788880184.295028:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 0000000023d77907 00800000:00000002:0.0:1788880184.295028:0:203453:0:(lmv_intent.c:573:lmv_intent_lock()) INTENT LOCK 'getattr' for [0x200000007:0x1:0x0] '' on [0x200000007:0x1:0x0] 00800000:00000001:0.0:1788880184.295034:0:203453:0:(lmv_intent.c:447:lmv_intent_lookup()) Process entered 00800000:00000001:0.0:1788880184.295035:0:203453:0:(lmv_fld.c:38:lmv_fld_lookup()) Process entered 80000000:00000001:0.0:1788880184.295037:0:203453:0:(fld_request.c:427:fld_client_lookup()) Process entered 80000000:00000001:0.0:1788880184.295038:0:203453:0:(fld_cache.c:479:fld_cache_lookup()) Process entered 80000000:00000001:0.0:1788880184.295040:0:203453:0:(fld_cache.c:498:fld_cache_lookup()) Process leaving (rc=0 : 0 : 0) 80000000:00000001:0.0:1788880184.295041:0:203453:0:(fld_request.c:434:fld_client_lookup()) Process leaving (rc=0 : 0 : 0) 00800000:00000002:0.0:1788880184.295043:0:203453:0:(lmv_fld.c:57:lmv_fld_lookup()) FLD lookup got mds #0 for fid=[0x200000007:0x1:0x0] 00800000:00000001:0.0:1788880184.295044:0:203453:0:(lmv_fld.c:65:lmv_fld_lookup()) Process leaving (rc=0 : 0 : 0) 00800000:00000002:0.0:1788880184.295045:0:203453:0:(lmv_intent.c:500:lmv_intent_lookup()) LOOKUP_INTENT with fid1=[0x200000007:0x1:0x0], fid2=[0x200000007:0x1:0x0], name='' -> mds #0 00000020:00000010:0.0:1788880184.295049:0:203453:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 640 at 000000006ece4223. 00000002:00000001:0.0:1788880184.295051:0:203453:0:(mdc_locks.c:1500:mdc_intent_lock()) Process entered 00000020:00000001:0.0:1788880184.295052:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 00000000395658e9 00000002:00010000:0.0:1788880184.295053:0:203453:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00000002:00000001:0.0:1788880184.295055:0:203453:0:(mdc_locks.c:1404:mdc_revalidate_lock()) Process entered 00000002:00000001:0.0:1788880184.295056:0:203453:0:(mdc_locks.c:135:mdc_lock_match()) Process entered 00010000:00000001:0.0:1788880184.295057:0:203453:0:(ldlm_lock.c:1446:ldlm_lock_match_with_skip()) Process entered 00010000:00000040:0.0:1788880184.295059:0:203453:0:(ldlm_resource.c:1656:ldlm_resource_getref()) getref res: 0000000049bd8a46 count: 2 00010000:00000001:0.0:1788880184.295060:0:203453:0:(ldlm_resource.c:1595:ldlm_resource_get()) Process leaving via found (rc=0 : 0 : 0x0) 00010000:00000001:0.0:1788880184.295062:0:203453:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00000020:00000010:0.0:1788880184.295064:0:203453:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 128 at 00000000954a6e70. 00010000:00000001:0.0:1788880184.295065:0:203453:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=1 : 1 : 1) 00010000:00010000:0.0:1788880184.295067:0:203453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 02000000:00000001:0.0:1788880184.295072:0:203453:0:(sec.c:1034:sptlrpc_import_check_ctx()) Process entered 02000000:00000001:0.0:1788880184.295074:0:203453:0:(sec.c:1050:sptlrpc_import_check_ctx()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880184.295075:0:203453:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00000001:0.0:1788880184.295079:0:203453:0:(mdc_locks.c:141:mdc_lock_match()) Process leaving (rc=4 : 4 : 4) 00000002:00000001:0.0:1788880184.295080:0:203453:0:(mdc_locks.c:1456:mdc_revalidate_lock()) Process leaving (rc=1 : 1 : 1) 00000002:00000001:0.0:1788880184.295081:0:203453:0:(mdc_locks.c:1523:mdc_intent_lock()) Process leaving (rc=1 : 1 : 1) 00800000:00000001:0.0:1788880184.295082:0:203453:0:(lmv_intent.c:521:lmv_intent_lookup()) Process leaving (rc=1 : 1 : 1) 00800000:00000001:0.0:1788880184.295083:0:203453:0:(lmv_intent.c:606:lmv_intent_lock()) Process leaving (rc=1 : 1 : 1) 00000080:00200000:0.0:1788880184.295086:0:203453:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00000001:0.0:1788880184.295088:0:203453:0:(namei.c:990:ll_intent_lock()) Process leaving (rc=1 : 1 : 1) 00800000:00000002:0.0:1788880184.295089:0:203453:0:(lmv_obd.c:4039:lmv_stripe_object_put()) put 00000000a6e808b7 2 00000080:00000010:0.0:1788880184.295090:0:203453:0:(llite_lib.c:4196:ll_finish_md_op_data()) kfreed 'op_data': 392 at 0000000030a8fe2c. 00000080:00000001:0.0:1788880184.295092:0:203453:0:(dcache.c:220:ll_revalidate_it_finish()) Process entered 00000080:00000001:0.0:1788880184.295093:0:203453:0:(dcache.c:223:ll_revalidate_it_finish()) Process leaving (rc=0 : 0 : 0) 00000080:00010000:0.0:1788880184.295094:0:203453:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000f8fa31aa) 00000080:00010000:0.0:1788880184.295096:0:203453:0:(llite_internal.h:2014:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000f8fa31aa) for lock 0x805d548a36a886ba 00800000:00000001:0.0:1788880184.295097:0:203453:0:(lmv_obd.c:4079:lmv_set_lock_data()) Process entered 00000002:00000001:0.0:1788880184.295098:0:203453:0:(mdc_locks.c:94:mdc_set_lock_data()) Process entered 00010000:00000001:0.0:1788880184.295099:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.295100:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.295100:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=3 00000020:00000001:0.0:1788880184.295101:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.295102:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.295104:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.295104:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000002:00000001:0.0:1788880184.295105:0:203453:0:(mdc_locks.c:122:mdc_set_lock_data()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:0.0:1788880184.295106:0:203453:0:(lmv_obd.c:4084:lmv_set_lock_data()) Process leaving (rc=0 : 0 : 0) 00000080:00010000:0.0:1788880184.295107:0:203453:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x805d548a36a886ba from it 00000000b77e1de1 00010000:00000001:0.0:1788880184.295108:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.295109:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.295109:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=3 00000020:00000001:0.0:1788880184.295110:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.295111:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.295112:0:203453:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:0.0:1788880184.295113:0:203453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00000001:0.0:1788880184.295116:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.295116:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880184.295117:0:203453:0:(ldlm_lock.c:302:ldlm_lock_add_to_lru()) Process entered 00010000:00000001:0.0:1788880184.295117:0:203453:0:(ldlm_lock.c:306:ldlm_lock_add_to_lru()) Process leaving 00010000:00010000:0.0:1788880184.295118:0:203453:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 1 00010000:00000001:0.0:1788880184.295124:0:203453:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1788880184.295125:0:203453:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1788880184.295126:0:203453:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1788880184.295127:0:203453:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.295129:0:203453:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1788880184.295129:0:203453:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.295130:0:203453:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.295131:0:203453:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.295132:0:203453:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:0.0:1788880184.295133:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.295134:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000100:00000001:0.0:1788880184.295135:0:203453:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880184.295136:0:203453:0:(client.c:2856:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000080:00000001:0.0:1788880184.295137:0:203453:0:(file.c:6236:ll_merge_md_attr()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880184.295139:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 0000000016be5b21 00000080:00000008:0.0:1788880184.295139:0:203453:0:(file.c:6459:ll_getattr_dentry()) COMPLETED file /:[0x200000007:0x1:0x0](00000000f8fa31aa), request_mask 0, flags 0, foreign 0 00000080:00000001:0.0:1788880184.295202:0:203453:0:(file.c:6635:ll_inode_permission()) Process entered 00000080:00000001:0.0:1788880184.295203:0:203453:0:(file.c:6176:ll_inode_revalidate()) Process entered 00000020:00000001:0.0:1788880184.295204:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 000000005a61c864 00000080:00200000:0.0:1788880184.295205:0:203453:0:(file.c:6178:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000f8fa31aa),name=/ 00000080:00000010:0.0:1788880184.295207:0:203453:0:(llite_lib.c:4055:ll_prep_md_op_data()) kmalloced '(op_data)': 392 at 0000000030a8fe2c. 00800000:00000002:0.0:1788880184.295208:0:203453:0:(lmv_obd.c:3994:lmv_stripe_object_get()) get 00000000a6e808b7 2 00000080:00000001:0.0:1788880184.295210:0:203453:0:(namei.c:944:ll_intent_lock()) Process entered 00800000:00000001:0.0:1788880184.295211:0:203453:0:(lmv_intent.c:566:lmv_intent_lock()) Process entered 00000020:00000001:0.0:1788880184.295212:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 0000000000eb5cb3 00800000:00000002:0.0:1788880184.295213:0:203453:0:(lmv_intent.c:573:lmv_intent_lock()) INTENT LOCK 'getattr' for [0x200000007:0x1:0x0] '' on [0x200000007:0x1:0x0] 00800000:00000001:0.0:1788880184.295214:0:203453:0:(lmv_intent.c:447:lmv_intent_lookup()) Process entered 00800000:00000001:0.0:1788880184.295215:0:203453:0:(lmv_fld.c:38:lmv_fld_lookup()) Process entered 80000000:00000001:0.0:1788880184.295216:0:203453:0:(fld_request.c:427:fld_client_lookup()) Process entered 80000000:00000001:0.0:1788880184.295217:0:203453:0:(fld_cache.c:479:fld_cache_lookup()) Process entered 80000000:00000001:0.0:1788880184.295218:0:203453:0:(fld_cache.c:498:fld_cache_lookup()) Process leaving (rc=0 : 0 : 0) 80000000:00000001:0.0:1788880184.295218:0:203453:0:(fld_request.c:434:fld_client_lookup()) Process leaving (rc=0 : 0 : 0) 00800000:00000002:0.0:1788880184.295219:0:203453:0:(lmv_fld.c:57:lmv_fld_lookup()) FLD lookup got mds #0 for fid=[0x200000007:0x1:0x0] 00800000:00000001:0.0:1788880184.295220:0:203453:0:(lmv_fld.c:65:lmv_fld_lookup()) Process leaving (rc=0 : 0 : 0) 00800000:00000002:0.0:1788880184.295221:0:203453:0:(lmv_intent.c:500:lmv_intent_lookup()) LOOKUP_INTENT with fid1=[0x200000007:0x1:0x0], fid2=[0x200000007:0x1:0x0], name='' -> mds #0 00000002:00000001:0.0:1788880184.295227:0:203453:0:(mdc_locks.c:1500:mdc_intent_lock()) Process entered 00000020:00000001:0.0:1788880184.295227:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 00000000cd743751 00000002:00010000:0.0:1788880184.295228:0:203453:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00000002:00000001:0.0:1788880184.295230:0:203453:0:(mdc_locks.c:1404:mdc_revalidate_lock()) Process entered 00000002:00000001:0.0:1788880184.295231:0:203453:0:(mdc_locks.c:135:mdc_lock_match()) Process entered 00010000:00000001:0.0:1788880184.295251:0:203453:0:(ldlm_lock.c:1446:ldlm_lock_match_with_skip()) Process entered 00010000:00000040:0.0:1788880184.295254:0:203453:0:(ldlm_resource.c:1656:ldlm_resource_getref()) getref res: 0000000049bd8a46 count: 2 00010000:00000001:0.0:1788880184.295257:0:203453:0:(ldlm_resource.c:1595:ldlm_resource_get()) Process leaving via found (rc=0 : 0 : 0x0) 00010000:00000001:0.0:1788880184.295259:0:203453:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:0.0:1788880184.295262:0:203453:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=1 : 1 : 1) 00010000:00010000:0.0:1788880184.295265:0:203453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 1 02000000:00000001:0.0:1788880184.295271:0:203453:0:(sec.c:1034:sptlrpc_import_check_ctx()) Process entered 02000000:00000001:0.0:1788880184.295274:0:203453:0:(sec.c:1050:sptlrpc_import_check_ctx()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880184.295276:0:203453:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 1 00000002:00000001:0.0:1788880184.295282:0:203453:0:(mdc_locks.c:141:mdc_lock_match()) Process leaving (rc=4 : 4 : 4) 00000002:00000001:0.0:1788880184.295284:0:203453:0:(mdc_locks.c:1456:mdc_revalidate_lock()) Process leaving (rc=1 : 1 : 1) 00000002:00000001:0.0:1788880184.295286:0:203453:0:(mdc_locks.c:1523:mdc_intent_lock()) Process leaving (rc=1 : 1 : 1) 00800000:00000001:0.0:1788880184.295288:0:203453:0:(lmv_intent.c:521:lmv_intent_lookup()) Process leaving (rc=1 : 1 : 1) 00800000:00000001:0.0:1788880184.295290:0:203453:0:(lmv_intent.c:606:lmv_intent_lock()) Process leaving (rc=1 : 1 : 1) 00000080:00200000:0.0:1788880184.295292:0:203453:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00000001:0.0:1788880184.295294:0:203453:0:(namei.c:990:ll_intent_lock()) Process leaving (rc=1 : 1 : 1) 00800000:00000002:0.0:1788880184.295296:0:203453:0:(lmv_obd.c:4039:lmv_stripe_object_put()) put 00000000a6e808b7 2 00000080:00000010:0.0:1788880184.295298:0:203453:0:(llite_lib.c:4196:ll_finish_md_op_data()) kfreed 'op_data': 392 at 0000000030a8fe2c. 00000080:00000001:0.0:1788880184.295302:0:203453:0:(dcache.c:220:ll_revalidate_it_finish()) Process entered 00000080:00000001:0.0:1788880184.295303:0:203453:0:(dcache.c:223:ll_revalidate_it_finish()) Process leaving (rc=0 : 0 : 0) 00000080:00010000:0.0:1788880184.295305:0:203453:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000f8fa31aa) 00000080:00010000:0.0:1788880184.295307:0:203453:0:(llite_internal.h:2014:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000f8fa31aa) for lock 0x805d548a36a886ba 00800000:00000001:0.0:1788880184.295309:0:203453:0:(lmv_obd.c:4079:lmv_set_lock_data()) Process entered 00000002:00000001:0.0:1788880184.295311:0:203453:0:(mdc_locks.c:94:mdc_set_lock_data()) Process entered 00010000:00000001:0.0:1788880184.295316:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.295317:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.295318:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=3 00000020:00000001:0.0:1788880184.295319:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.295321:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.295322:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.295323:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000002:00000001:0.0:1788880184.295323:0:203453:0:(mdc_locks.c:122:mdc_set_lock_data()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:0.0:1788880184.295324:0:203453:0:(lmv_obd.c:4084:lmv_set_lock_data()) Process leaving (rc=0 : 0 : 0) 00000080:00010000:0.0:1788880184.295325:0:203453:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x805d548a36a886ba from it 0000000069de55b0 00010000:00000001:0.0:1788880184.295326:0:203453:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880184.295327:0:203453:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880184.295327:0:203453:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000046030e36 refcount=3 00000020:00000001:0.0:1788880184.295328:0:203453:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.295329:0:203453:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661944320 : -126823047607296 : ffff8ca7b54c4c00) 00010000:00000001:0.0:1788880184.295331:0:203453:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:0.0:1788880184.295332:0:203453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 1 00010000:00000001:0.0:1788880184.295335:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.295336:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880184.295336:0:203453:0:(ldlm_lock.c:302:ldlm_lock_add_to_lru()) Process entered 00010000:00000001:0.0:1788880184.295337:0:203453:0:(ldlm_lock.c:306:ldlm_lock_add_to_lru()) Process leaving 00010000:00010000:0.0:1788880184.295338:0:203453:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8ca787527000 lock: 0000000046030e36/0x805d548a36a886ba lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x92f11317f60a7941 expref: -99 pid: 203453 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00000001:0.0:1788880184.295341:0:203453:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1788880184.295342:0:203453:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1788880184.295342:0:203453:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1788880184.295343:0:203453:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.295344:0:203453:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1788880184.295345:0:203453:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.295346:0:203453:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.295347:0:203453:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880184.295351:0:203453:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:0.0:1788880184.295352:0:203453:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880184.295353:0:203453:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000100:00000001:0.0:1788880184.295354:0:203453:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880184.295354:0:203453:0:(client.c:2856:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000080:00200000:0.0:1788880184.295355:0:203453:0:(file.c:6650:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](00000000f8fa31aa), inode mode 41ed mask 44 00000080:00000001:0.0:1788880184.295357:0:203453:0:(file.c:6689:ll_inode_permission()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:0.0:1788880184.295360:0:203453:0:(dir.c:3010:ll_dir_open()) Process entered 00000080:00000001:0.0:1788880184.295361:0:203453:0:(file.c:1086:ll_file_open()) Process entered 00000020:00000001:0.0:1788880184.295362:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 000000005a61c864 00000080:00200008:0.0:1788880184.295363:0:203453:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](00000000f8fa31aa), flags 100000 00000080:00000010:0.0:1788880184.295366:0:203453:0:(file.c:70:ll_file_data_get()) slab-alloced 'lfd': 288 at 00000000d4d75441. 00000080:00000001:0.0:1788880184.295369:0:203453:0:(file.c:1115:ll_file_open()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:0.0:1788880184.295369:0:203453:0:(dir.c:3011:ll_dir_open()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:0.0:1788880184.295431:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:0.0:1788880184.295433:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:0.0:1788880184.295438:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:0.0:1788880184.295440:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:0.0:1788880184.295443:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 0000000018c57bb0. 00000020:00000001:0.0:1788880184.295446:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:0.0:1788880184.295447:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:0.0:1788880184.295449:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:0.0:1788880184.295450:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca7919a7800 uarg=0000000000000000 00800000:00000001:0.0:1788880184.295454:0:203453:0:(obd_class.h:1075:obd_statfs()) Process entered 00800000:00000004:0.0:1788880184.295455:0:203453:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0000-mdc-ffff8ca787527000: age 11146, max_age 11147 00000002:00000001:0.0:1788880184.295458:0:203453:0:(mdc_request.c:1667:mdc_statfs()) Process entered 00000020:00000040:0.0:1788880184.295459:0:203453:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000010:0.0:1788880184.295463:0:203453:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000009ee4c6bf. 00000020:00000040:0.0:1788880184.295465:0:203453:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=5 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880184.295468:0:203453:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880184.295470:0:203453:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880184.295472:0:203453:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880184.295475:0:203453:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000088542ca3. 00000100:00000001:0.0:1788880184.295480:0:203453:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880184.295482:0:203453:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:0.0:1788880184.295483:0:203453:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:0.0:1788880184.295487:0:203453:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000095d2d2a1. 00000100:00000001:0.0:1788880184.295491:0:203453:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250052435328 : -126823657116288 : ffff8ca790f7ed80) 00000100:00000001:0.0:1788880184.295492:0:203453:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880184.295493:0:203453:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849358720 : -126822860192896 : ffff8ca7c0780380) 00000100:00000001:0.0:1788880184.295496:0:203453:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880184.295498:0:203453:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880184.295499:0:203453:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:0.0:1788880184.295500:0:203453:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880184.295504:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000009ee4c6bf x1875776780635904/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880184.295509:0:203453:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880184.295510:0:203453:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880184.295513:0:203453:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880184.295514:0:203453:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880184.295516:0:203453:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880184.295518:0:203453:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job lfs:ea27006f-52a8-4007-b69a-dcac8279445a:203453:1875776780635904:192.168.204.111@tcp:41: 00000100:00000001:0.0:1788880184.295520:0:203453:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880184.295525:0:203453:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880184.295526:0:203453:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880184.295527:0:203453:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880184.295530:0:203453:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000006fc9dc72. 02000000:00000001:0.0:1788880184.295531:0:203453:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880184.295534:0:203453:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ca37d043. 00000400:00000010:0.0:1788880184.295538:0:203453:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a08ac001. 00000100:00000200:0.0:1788880184.295543:0:203453:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780635904, portal 10 00000100:00000001:0.0:1788880184.295544:0:203453:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880184.295545:0:203453:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849358720 : -126822860192896 : ffff8ca7c0780380) 00000100:00000040:0.0:1788880184.295549:0:203453:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000009ee4c6bf x1875776780635904/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880184.295554:0:203453:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880184.295557:0:203453:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880184.295558:0:203453:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000100:00000200:0.0:1788880184.295561:0:203453:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780635904, offset 0 00000400:00000200:0.0:1788880184.295566:0:203453:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880184.295578:0:203453:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880184.295582:0:203453:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880184.295586:0:203453:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880184.295589:0:203453:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880184.295591:0:203453:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880184.295594:0:203453:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [34:34:256:4294967295] 192.168.204.111@tcp LPNI seq info [34:34:8:4294967295] 00000400:00000200:0.0:1788880184.295600:0:203453:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880184.295605:0:203453:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880184.295608:0:203453:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000088d6bc1. 00000800:00000200:0.0:1788880184.295612:0:203453:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880184.295616:0:203453:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880184.295619:0:203453:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000088d6bc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880184.295634:0:203453:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880184.295637:0:203453:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 2818982 usecs (2819 jiffies) 00000100:00000001:0.0:1788880184.295639:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305818220 00000100:00000001:0.0:1788880184.295641:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305819094 00000100:00000001:0.0:1788880184.295642:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305818220 00000100:00000001:0.0:1788880184.295643:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305819095 00000100:00000001:0.0:1788880184.295644:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305818220 00000100:00000001:0.0:1788880184.295645:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305819095 00000100:00000001:0.0:1788880184.295646:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305818220 00000100:00000001:0.0:1788880184.295647:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305819095 00000100:00000001:0.0:1788880184.295648:0:203453:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880184.295650:0:203453:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880184.295651:0:203453:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880184.295652:0:203453:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:0.0:1788880184.295654:0:203453:0:(client.c:2702:ptlrpc_set_wait()) set 0000000095d2d2a1 going to sleep for 11 seconds 00000100:00000001:0.0:1788880184.295656:0:203453:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880184.295661:0:203453:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880184.296832:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880184.296839:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880184.296849:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880184.296851:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880184.296858:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880184.296869:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 368 into portal 10 MB=0x6aa0250b00f00 00000400:00000200:0.0:1788880184.296876:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 368/368 into md 0x205 [1] + 224 00000800:00000001:0.0:1788880184.296881:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880184.296886:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880184.296888:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880184.296891:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880184.296896:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880184.296899:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000009ee4c6bf x1875776780635904/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880184.296911:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=368 offset=224 replen=368 req@000000009ee4c6bf x1875776780635904/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880184.296921:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880184.296927:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880184.296929:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880184.296946:0:203453:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880184.296952:0:203453:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880184.296954:0:203453:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880184.296955:0:203453:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ca37d043. 00000400:00000200:3.0:1788880184.296959:0:203453:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a08ac001 00000400:00000010:3.0:1788880184.296961:0:203453:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a08ac001. 00000100:00000001:3.0:1788880184.296964:0:203453:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880184.296967:0:203453:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000009ee4c6bf x1875776780635904/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880184.296976:0:203453:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000009ee4c6bf x1875776780635904/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.296983:0:203453:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880184.296984:0:203453:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880184.296986:0:203453:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880184.296988:0:203453:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880184.296990:0:203453:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880184.296992:0:203453:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880184.296994:0:203453:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880184.296999:0:203453:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=2 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880184.297003:0:203453:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=2 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880184.297005:0:203453:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880184.297006:0:203453:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880184.297008:0:203453:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880184.297010:0:203453:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.297013:0:203453:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880184.297015:0:203453:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880184.297017:0:203453:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880184.297018:0:203453:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880184.297027:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000009ee4c6bf x1875776780635904/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.297034:0:203453:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880184.297037:0:203453:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880184.297039:0:203453:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880184.297041:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000009ee4c6bf x1875776780635904/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880184.297048:0:203453:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job lfs:ea27006f-52a8-4007-b69a-dcac8279445a:203453:1875776780635904:192.168.204.111@tcp:41: 00000100:00000001:3.0:1788880184.297052:0:203453:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880184.297054:0:203453:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.297056:0:203453:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:3.0:1788880184.297058:0:203453:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880184.297060:0:203453:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000009ee4c6bf x1875776780635904/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.297067:0:203453:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880184.297069:0:203453:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000095d2d2a1. 00000100:00000001:3.0:1788880184.297072:0:203453:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:3.0:1788880184.297073:0:203453:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880184.297077:0:203453:0:(mdc_request.c:1719:mdc_statfs()) Process leaving 00000100:00000001:3.0:1788880184.297078:0:203453:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880184.297080:0:203453:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000009ee4c6bf x1875776780635904/t0(0) o41->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.297086:0:203453:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880184.297088:0:203453:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880184.297090:0:203453:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000006fc9dc72. 02000000:00000001:3.0:1788880184.297092:0:203453:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880184.297094:0:203453:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880184.297095:0:203453:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880184.297097:0:203453:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880184.297099:0:203453:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000088542ca3. 02000000:00000001:3.0:1788880184.297103:0:203453:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880184.297104:0:203453:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880184.297106:0:203453:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000009ee4c6bf. 00000100:00000001:3.0:1788880184.297114:0:203453:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880184.297115:0:203453:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880184.297117:0:203453:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880184.297118:0:203453:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880184.297119:0:203453:0:(genops.c:1251:class_import_put()) Process leaving 00800000:00000004:3.0:1788880184.297121:0:203453:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0000-mdc-ffff8ca787527000: update 000000009485984e cache blocks 20244/23812 objects 79718/79992 00800000:00000001:3.0:1788880184.297124:0:203453:0:(obd_class.h:1136:obd_statfs()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:3.0:1788880184.297128:0:203453:0:(lmv_obd.c:1194:lmv_iocontrol()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.297131:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=0 : 0 : 0) 00000080:00000010:3.0:1788880184.297133:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 0000000018c57bb0. 00000080:00000001:3.0:1788880184.297136:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.297138:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.297177:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.297179:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000020:00000010:3.0:1788880184.297186:0:203453:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 1856 at 000000008158dd15. 00000080:00000001:3.0:1788880184.297189:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.297191:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.297195:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.297197:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.297199:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.297202:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.297203:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.297208:0:203453:0:(obd_class.h:1075:obd_statfs()) Process entered 00800000:00000004:3.0:1788880184.297210:0:203453:0:(obd_class.h:1091:obd_statfs()) lustre-MDT0001-mdc-ffff8ca787527000: age 10136, max_age 11147 00000002:00000001:3.0:1788880184.297212:0:203453:0:(mdc_request.c:1667:mdc_statfs()) Process entered 00000020:00000040:3.0:1788880184.297215:0:203453:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000010:3.0:1788880184.297218:0:203453:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000020:00000040:3.0:1788880184.297221:0:203453:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=5 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880184.297224:0:203453:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880184.297226:0:203453:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880184.297228:0:203453:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880184.297254:0:203453:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:3.0:1788880184.297259:0:203453:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.297262:0:203453:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:3.0:1788880184.297263:0:203453:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:3.0:1788880184.297272:0:203453:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000e1cb172e. 00000100:00000001:3.0:1788880184.297274:0:203453:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063674560 : -126823645877056 : ffff8ca791a36cc0) 00000100:00000001:3.0:1788880184.297277:0:203453:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880184.297279:0:203453:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000100:00000001:3.0:1788880184.297282:0:203453:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880184.297284:0:203453:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.297285:0:203453:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:3.0:1788880184.297287:0:203453:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880184.297290:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780636032/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.297296:0:203453:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880184.297298:0:203453:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880184.297301:0:203453:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880184.297303:0:203453:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880184.297305:0:203453:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880184.297308:0:203453:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job lfs:ea27006f-52a8-4007-b69a-dcac8279445a:203453:1875776780636032:192.168.204.111@tcp:41: 00000100:00000001:3.0:1788880184.297312:0:203453:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880184.297315:0:203453:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880184.297316:0:203453:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880184.297318:0:203453:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880184.297322:0:203453:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000d7b2c6b7. 02000000:00000001:3.0:1788880184.297324:0:203453:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880184.297328:0:203453:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004100a205. 00000400:00000010:3.0:1788880184.297332:0:203453:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 00000100:00000200:3.0:1788880184.297336:0:203453:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780636032, portal 10 00000100:00000001:3.0:1788880184.297338:0:203453:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880184.297340:0:203453:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000100:00000040:3.0:1788880184.297345:0:203453:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780636032/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.297352:0:203453:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880184.297355:0:203453:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880184.297357:0:203453:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000082116f14. 00000100:00000200:3.0:1788880184.297359:0:203453:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780636032, offset 0 00000400:00000200:3.0:1788880184.297369:0:203453:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880184.297376:0:203453:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880184.297380:0:203453:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880184.297384:0:203453:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880184.297387:0:203453:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880184.297390:0:203453:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880184.297393:0:203453:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [35:35:256:4294967295] 192.168.204.111@tcp LPNI seq info [35:35:8:4294967295] 00000400:00000200:3.0:1788880184.297403:0:203453:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880184.297409:0:203453:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880184.297413:0:203453:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000800:00000200:3.0:1788880184.297418:0:203453:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880184.297424:0:203453:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880184.297427:0:203453:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880184.297440:0:203453:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880184.297444:0:203453:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 316211 usecs (317 jiffies) 00000100:00000001:3.0:1788880184.297447:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305815720 00000100:00000001:3.0:1788880184.297450:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305819094 00000100:00000001:3.0:1788880184.297452:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305815720 00000100:00000001:3.0:1788880184.297454:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305819095 00000100:00000001:3.0:1788880184.297455:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305815720 00000100:00000001:3.0:1788880184.297457:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305819095 00000100:00000001:3.0:1788880184.297459:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305815720 00000100:00000001:3.0:1788880184.297461:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305819095 00000100:00000001:3.0:1788880184.297463:0:203453:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880184.297466:0:203453:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.297469:0:203453:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880184.297470:0:203453:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:3.0:1788880184.297473:0:203453:0:(client.c:2702:ptlrpc_set_wait()) set 00000000e1cb172e going to sleep for 11 seconds 00000100:00000001:3.0:1788880184.297480:0:203453:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880184.297482:0:203453:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880184.297494:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880184.297498:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000400:00000200:0.0:1788880184.297503:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880184.297507:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880184.297510:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000082116f14 00000400:00000010:0.0:1788880184.297512:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000082116f14. 00000100:00000001:0.0:1788880184.297515:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880184.297518:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780636032/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880184.297526:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880184.297528:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@000000002a42fbe1 x1875776780636032/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880184.297544:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880184.297546:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880184.298720:0:203453:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880184.298768:0:203453:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880184.298770:0:203453:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880184.298771:0:203453:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004100a205. 00000400:00000200:3.0:1788880184.298774:0:203453:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000400:00000010:3.0:1788880184.298776:0:203453:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:3.0:1788880184.298778:0:203453:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880184.298782:0:203453:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780636032/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880184.298799:0:203453:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780636032/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.298805:0:203453:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880184.298806:0:203453:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880184.298820:0:203453:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880184.298822:0:203453:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880184.298824:0:203453:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880184.298825:0:203453:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880184.298827:0:203453:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880184.298832:0:203453:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=12 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880184.298835:0:203453:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=12 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880184.298837:0:203453:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880184.298838:0:203453:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880184.298840:0:203453:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880184.298844:0:203453:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002a42fbe1 x1875776780636032/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880184.298850:0:203453:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.298852:0:203453:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880184.298853:0:203453:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880184.298855:0:203453:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880184.298856:0:203453:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880184.298859:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780636032/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.298875:0:203453:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880184.298877:0:203453:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880184.298879:0:203453:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880184.298882:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780636032/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880184.298888:0:203453:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job lfs:ea27006f-52a8-4007-b69a-dcac8279445a:203453:1875776780636032:192.168.204.111@tcp:41: 00000100:00000001:3.0:1788880184.298892:0:203453:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880184.298894:0:203453:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.298896:0:203453:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:3.0:1788880184.298897:0:203453:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880184.298899:0:203453:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780636032/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.298904:0:203453:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880184.298906:0:203453:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000e1cb172e. 00000100:00000001:3.0:1788880184.298909:0:203453:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:3.0:1788880184.298910:0:203453:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880184.298914:0:203453:0:(mdc_request.c:1719:mdc_statfs()) Process leaving 00000100:00000001:3.0:1788880184.298915:0:203453:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880184.298917:0:203453:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780636032/t0(0) o41->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/368 e 0 to 0 dl 1788880200 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.298922:0:203453:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880184.298924:0:203453:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880184.298926:0:203453:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000d7b2c6b7. 02000000:00000001:3.0:1788880184.298928:0:203453:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880184.298929:0:203453:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880184.298931:0:203453:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880184.298933:0:203453:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880184.298934:0:203453:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 02000000:00000001:3.0:1788880184.298937:0:203453:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880184.298938:0:203453:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880184.298939:0:203453:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:3.0:1788880184.298942:0:203453:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880184.298947:0:203453:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880184.298948:0:203453:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880184.298949:0:203453:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880184.298951:0:203453:0:(genops.c:1251:class_import_put()) Process leaving 00800000:00000004:3.0:1788880184.298952:0:203453:0:(obd_class.h:1117:obd_statfs()) lustre-MDT0001-mdc-ffff8ca787527000: update 00000000032a86de cache blocks 20275/23812 objects 79737/79992 00800000:00000001:3.0:1788880184.298955:0:203453:0:(obd_class.h:1136:obd_statfs()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:3.0:1788880184.298959:0:203453:0:(lmv_obd.c:1194:lmv_iocontrol()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.298961:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=0 : 0 : 0) 00000080:00000010:3.0:1788880184.298963:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.298965:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.298967:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299001:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299003:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299008:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299009:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299013:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299016:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299018:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299021:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299022:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299026:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299028:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299030:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299033:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299035:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299036:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299040:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299041:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299043:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299045:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299046:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299048:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299050:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299051:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299052:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299058:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299060:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299062:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299064:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299066:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299068:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299072:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299073:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299075:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299076:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299079:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299081:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299083:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299084:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299085:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299087:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299088:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299090:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299092:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299095:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299096:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299104:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299106:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299109:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299110:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299112:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299114:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299116:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299117:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299118:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299120:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299122:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299128:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299130:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299133:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299135:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299139:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299140:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299143:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299144:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299146:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299148:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299150:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299152:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299153:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299156:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299158:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299160:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299161:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299163:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299165:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299168:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299169:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299170:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299171:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299173:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299175:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299176:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299177:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299178:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299181:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299183:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299184:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299186:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299192:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299194:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299198:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299199:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299202:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299203:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299205:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299207:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299209:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299210:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299211:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299213:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299215:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299216:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299218:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299220:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299222:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299225:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299226:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299229:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299230:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299316:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299320:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299321:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299323:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299324:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299326:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299328:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299330:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299333:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299335:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299338:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299342:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299347:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299350:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299351:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299353:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299355:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299356:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299358:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299359:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299361:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299363:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299364:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299366:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299368:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299370:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299374:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299375:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299377:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299378:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299379:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299381:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299382:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299383:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299384:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299386:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299388:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299389:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299391:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299393:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299394:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299396:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299397:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299399:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299399:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299405:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299407:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299408:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299410:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299411:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299413:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299415:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299416:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299418:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299420:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299421:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299425:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299426:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299429:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299430:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299432:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299433:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299435:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299436:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299436:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299438:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299440:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299442:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299443:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299445:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299446:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299449:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299450:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299451:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299452:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299454:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299456:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299457:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299461:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299462:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299465:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299466:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299468:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299470:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299471:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299473:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299477:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299478:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299480:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299481:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299483:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299484:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299486:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299487:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299487:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299489:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299491:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299492:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299494:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299496:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299497:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299500:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299500:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299503:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299503:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299505:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299506:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299508:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299509:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299509:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299511:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299517:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299519:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299520:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299522:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299523:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299527:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299528:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299530:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299531:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299533:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299535:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299536:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299537:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299538:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299540:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299542:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299543:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299545:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299546:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299547:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299550:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299551:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299553:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299554:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299555:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299557:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299558:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299560:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299561:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299563:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299564:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299565:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299567:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299572:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299574:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299578:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299578:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299581:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299582:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299584:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299585:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299587:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299588:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299589:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299591:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299592:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299594:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299595:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299597:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299599:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299601:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299602:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299604:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299605:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299606:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299608:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299610:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299611:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299611:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299614:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299615:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299617:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299618:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299620:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299621:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299624:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299628:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299631:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299632:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299634:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299635:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299637:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299638:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299639:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299641:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299643:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299644:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299646:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299648:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299649:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299652:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299653:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299655:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299656:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299658:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299660:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299661:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299662:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299663:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299665:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299666:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299668:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299670:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299671:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299673:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299675:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299676:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299678:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299679:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299683:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299685:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299687:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299688:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299689:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299691:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299693:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299695:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299696:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299697:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299699:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299703:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299703:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299706:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299707:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299708:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299710:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299711:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299712:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299713:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299714:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299716:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299717:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299719:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299720:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299722:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299724:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299725:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299727:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299728:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299730:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299731:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299732:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299738:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299739:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299741:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299743:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299744:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299746:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299747:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299749:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299752:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299753:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299755:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299756:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299758:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299760:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299762:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299763:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299764:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299766:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299767:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299769:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299770:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299772:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299774:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299776:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299777:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299779:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299780:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299782:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299784:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299785:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299786:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299787:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299789:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299794:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299796:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299798:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299799:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299801:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299805:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299806:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299822:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299824:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299826:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299828:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299829:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299831:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299832:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299834:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299836:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299837:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299839:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299840:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299842:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299845:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299845:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299848:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299849:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299850:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299852:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299853:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299854:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299855:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299857:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299859:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299860:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299862:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299867:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299869:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299873:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299874:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299877:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299878:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299879:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299881:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299883:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299884:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299885:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299887:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299888:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299890:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299891:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299893:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299894:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299896:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299897:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299899:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299900:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299901:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299903:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299904:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299905:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299906:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299908:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299910:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299911:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299913:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299914:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299916:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299918:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299923:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299925:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299926:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299928:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299929:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299931:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299932:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299933:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299935:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299937:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299938:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299940:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299942:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299943:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299946:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299947:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299950:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299951:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299952:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299954:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299956:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299956:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299957:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299960:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299961:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299963:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299964:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299966:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299967:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299970:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299971:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.299973:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.299973:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.299978:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.299981:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.299982:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.299983:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.299984:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.299987:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299988:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299990:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.299991:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.299993:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299995:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.299998:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.299999:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300002:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300003:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300004:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300006:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300008:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300008:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300010:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300012:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300014:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300016:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300017:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300019:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300020:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300022:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300023:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300025:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300026:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300027:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300029:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300030:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300035:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300036:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300038:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300040:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300041:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300043:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300045:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300047:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300050:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300051:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300053:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300054:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300055:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300057:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300058:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300059:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300060:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300062:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300064:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300065:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300067:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300068:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300070:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300072:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300073:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300075:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300075:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300077:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300078:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300080:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300081:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300081:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300083:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300089:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300091:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300093:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300095:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300096:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300100:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300101:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300104:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300105:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300107:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300109:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300110:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300111:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300112:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300115:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300116:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300118:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300119:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300121:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300123:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300126:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300126:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300129:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300130:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300132:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300134:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300135:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300136:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300137:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300139:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300140:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300142:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300143:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300149:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300150:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300154:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300155:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300157:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300158:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300160:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300162:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300163:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300164:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300165:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300167:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300169:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300170:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300172:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300173:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300175:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300177:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300178:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300180:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300181:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300182:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300183:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300184:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300186:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300186:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300189:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300190:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300192:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300194:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300195:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300197:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300200:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300204:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300207:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300209:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300210:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300212:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300214:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300215:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300216:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300219:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300221:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300222:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300225:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300226:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300228:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300265:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300267:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300270:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300272:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300274:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300277:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300279:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300280:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300281:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300284:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300286:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300288:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300290:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300293:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300295:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300298:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300300:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300302:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300303:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300310:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300312:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300314:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300316:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300317:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300319:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300321:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300323:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300324:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300327:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300328:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300332:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300333:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300335:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300336:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300338:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300339:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300341:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300342:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300343:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300345:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300347:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300348:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300350:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300351:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300353:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300355:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300356:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300358:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300359:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300360:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300362:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300363:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300368:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300368:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300371:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300372:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300374:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300376:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300377:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300379:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300382:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300383:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300385:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300387:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300388:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300390:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300391:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300392:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300393:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300395:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300397:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300399:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300401:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300402:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300404:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300406:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300407:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300408:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300409:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300411:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300412:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300414:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300415:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300416:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300418:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300422:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300424:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300426:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300428:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300430:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300433:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300434:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300436:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300437:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300439:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300441:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300442:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300443:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300444:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300446:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300448:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300449:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300451:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300452:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300454:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300456:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300456:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300458:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300459:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300461:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300463:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300464:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300465:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300466:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300468:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300469:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300471:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300473:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300479:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300480:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300484:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300485:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300487:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300488:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300490:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300492:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300493:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300494:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300495:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300497:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300499:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300500:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300502:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300504:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300506:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300508:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300509:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300511:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300512:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300514:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300516:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300517:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300518:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300519:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300521:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300523:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300524:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300525:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300527:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300528:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300531:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300535:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300538:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300540:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300541:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300543:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300545:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300546:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300547:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300549:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300551:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300552:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300554:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300556:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300557:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300561:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300562:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300564:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300565:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300566:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300568:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300569:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300570:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300571:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300573:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300575:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300577:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300578:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300580:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300582:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300584:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300585:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300587:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300588:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300593:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300595:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300596:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300598:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300598:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300601:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300602:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300604:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300606:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300607:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300609:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300612:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300613:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300615:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300616:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300617:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300619:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300620:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300621:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300622:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300624:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300626:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300627:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300628:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300630:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300631:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300633:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300634:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300636:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300636:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300638:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300639:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300641:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300646:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300647:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300649:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300650:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300652:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300654:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300656:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300657:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300661:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300661:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300664:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300665:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300667:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300669:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300671:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300672:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300673:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300675:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300676:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300678:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300680:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300681:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300683:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300685:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300685:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300687:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300688:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300690:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300691:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300692:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300693:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300694:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300696:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300701:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300703:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300705:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300707:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300708:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300711:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300712:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300715:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300716:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300717:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300719:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300721:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300722:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300722:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300725:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300726:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300727:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300729:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300731:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300732:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300735:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300736:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300738:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300739:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300741:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300743:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300744:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300745:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300746:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300748:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300750:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300751:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300753:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300758:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300759:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300763:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300764:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300766:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300767:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300769:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300770:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300772:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300773:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300774:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300777:0:203453:0:(lmv_obd.c:966:lmv_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300778:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300779:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.300781:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300783:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300785:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.300787:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300788:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300790:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300791:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300793:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300795:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300796:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00800000:00000001:3.0:1788880184.300797:0:203453:0:(lmv_obd.c:929:lmv_iocontrol()) Process entered 00800000:00000080:3.0:1788880184.300798:0:203453:0:(lmv_obd.c:931:lmv_iocontrol()) lustre-clilmv-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00800000:00000001:3.0:1788880184.300800:0:203453:0:(lmv_obd.c:962:lmv_iocontrol()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000080:00000001:3.0:1788880184.300802:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000080:00000001:3.0:1788880184.300803:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000080:00000010:3.0:1788880184.300805:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.300819:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000080:00000001:3.0:1788880184.300821:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000080:00000001:3.0:1788880184.300825:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.300831:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.300835:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.300836:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.300838:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.300840:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.300842:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.300845:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.300847:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.300853:0:203453:0:(obd_class.h:1075:obd_statfs()) Process entered 00020000:00000004:3.0:1788880184.300854:0:203453:0:(obd_class.h:1091:obd_statfs()) lustre-OST0000-osc-ffff8ca787527000: age 10136, max_age 11147 00000008:00000001:3.0:1788880184.300858:0:203453:0:(osc_request.c:3470:osc_statfs()) Process entered 00000020:00000040:3.0:1788880184.300861:0:203453:0:(genops.c:1231:class_import_get()) import 00000000beb50720 refcount=3 obd=lustre-OST0000-osc-ffff8ca787527000 00000100:00000010:3.0:1788880184.300866:0:203453:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000020:00000040:3.0:1788880184.300868:0:203453:0:(genops.c:1231:class_import_get()) import 00000000beb50720 refcount=4 obd=lustre-OST0000-osc-ffff8ca787527000 00000020:00000001:3.0:1788880184.300872:0:203453:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880184.300873:0:203453:0:(genops.c:1244:class_import_put()) import 00000000beb50720 refcount=3 obd=lustre-OST0000-osc-ffff8ca787527000 00000020:00000001:3.0:1788880184.300875:0:203453:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880184.300876:0:203453:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880184.300879:0:203453:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880184.300883:0:203453:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880184.300887:0:203453:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:3.0:1788880184.300893:0:203453:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.300896:0:203453:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:3.0:1788880184.300898:0:203453:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:3.0:1788880184.300902:0:203453:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000e1cb172e. 00000100:00000001:3.0:1788880184.300905:0:203453:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063674560 : -126823645877056 : ffff8ca791a36cc0) 00000100:00000001:3.0:1788880184.300907:0:203453:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880184.300908:0:203453:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000100:00000001:3.0:1788880184.300912:0:203453:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880184.300914:0:203453:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.300915:0:203453:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:3.0:1788880184.300917:0:203453:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880184.300924:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.300932:0:203453:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880184.300939:0:203453:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880184.300943:0:203453:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880184.300945:0:203453:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880184.300947:0:203453:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880184.300950:0:203453:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job lfs:ea27006f-52a8-4007-b69a-dcac8279445a:203453:1875776780636160:192.168.204.111@tcp:13: 00000100:00000001:3.0:1788880184.300954:0:203453:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880184.300959:0:203453:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880184.300961:0:203453:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880184.300963:0:203453:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880184.300966:0:203453:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000d7b2c6b7. 02000000:00000001:3.0:1788880184.300968:0:203453:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880184.300972:0:203453:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004100a205. 00000400:00000010:3.0:1788880184.300976:0:203453:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 00000100:00000200:3.0:1788880184.300982:0:203453:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780636160, portal 4 00000100:00000001:3.0:1788880184.300984:0:203453:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880184.300985:0:203453:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000100:00000040:3.0:1788880184.300990:0:203453:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.300997:0:203453:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880184.300999:0:203453:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880184.301001:0:203453:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005b101ac6. 00000100:00000200:3.0:1788880184.301017:0:203453:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 7, xid 1875776780636160, offset 0 00000400:00000200:3.0:1788880184.301023:0:203453:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880184.301032:0:203453:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880184.301037:0:203453:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880184.301041:0:203453:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880184.301044:0:203453:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880184.301046:0:203453:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880184.301049:0:203453:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [36:36:256:4294967295] 192.168.204.111@tcp LPNI seq info [36:36:8:4294967295] 00000400:00000200:3.0:1788880184.301059:0:203453:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880184.301065:0:203453:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880184.301068:0:203453:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000800:00000200:3.0:1788880184.301078:0:203453:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880184.301085:0:203453:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880184.301089:0:203453:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880184.301106:0:203453:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880184.301110:0:203453:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: using a timeout of 2888 usecs (3 jiffies) 00000100:00000001:3.0:1788880184.301113:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305815409 00000100:00000001:3.0:1788880184.301116:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305819094 00000100:00000001:3.0:1788880184.301118:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305815409 00000100:00000001:3.0:1788880184.301120:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305819095 00000100:00000001:3.0:1788880184.301121:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305815409 00000100:00000001:3.0:1788880184.301123:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305819095 00000100:00000001:3.0:1788880184.301125:0:203453:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305815409 00000100:00000001:3.0:1788880184.301127:0:203453:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305819095 00000100:00000001:3.0:1788880184.301129:0:203453:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880184.301132:0:203453:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.301135:0:203453:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880184.301136:0:203453:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:3.0:1788880184.301138:0:203453:0:(client.c:2702:ptlrpc_set_wait()) set 00000000e1cb172e going to sleep for 11 seconds 00000100:00000001:3.0:1788880184.301141:0:203453:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880184.301143:0:203453:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880184.301189:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880184.301193:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000400:00000200:0.0:1788880184.301198:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880184.301203:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880184.301208:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005b101ac6 00000400:00000010:0.0:1788880184.301210:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005b101ac6. 00000100:00000001:0.0:1788880184.301213:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880184.301217:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880184.301226:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880184.301228:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880184.301280:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880184.301281:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880184.302503:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880184.302513:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880184.302516:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880184.302518:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880184.302525:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880184.302535:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 368 into portal 4 MB=0x6aa0250b01000 00000400:00000200:0.0:1788880184.302550:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.204.111@tcp of length 368/368 into md 0x215 [1] + 224 00000800:00000001:0.0:1788880184.302555:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880184.302561:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880184.302563:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880184.302566:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880184.302570:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880184.302573:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880184.302582:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=368 offset=224 replen=368 req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880184.302605:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880184.302611:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880184.302614:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880184.302627:0:203453:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880184.302633:0:203453:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880184.302634:0:203453:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880184.302636:0:203453:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004100a205. 00000400:00000200:3.0:1788880184.302639:0:203453:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000400:00000010:3.0:1788880184.302640:0:203453:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:3.0:1788880184.302641:0:203453:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880184.302644:0:203453:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880184.302650:0:203453:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.302659:0:203453:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880184.302660:0:203453:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880184.302662:0:203453:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880184.302664:0:203453:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880184.302666:0:203453:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880184.302667:0:203453:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880184.302669:0:203453:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880184.302674:0:203453:0:(import.c:2149:obd_at_measure()) add 5 to 00000000090ffc3f time=1788880184 v=5 (0 0 0 0) 00000100:00001000:3.0:1788880184.302676:0:203453:0:(import.c:2149:obd_at_measure()) add 1 to 000000003184a8ce time=2 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880184.302678:0:203453:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880184.302679:0:203453:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880184.302681:0:203453:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880184.302683:0:203453:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880184.302687:0:203453:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.302689:0:203453:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880184.302691:0:203453:0:(client.c:3108:ptlrpc_free_committed()) lustre-OST0000-osc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880184.302692:0:203453:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880184.302693:0:203453:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880184.302694:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.302698:0:203453:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880184.302700:0:203453:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880184.302702:0:203453:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880184.302704:0:203453:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880184.302711:0:203453:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job lfs:ea27006f-52a8-4007-b69a-dcac8279445a:203453:1875776780636160:192.168.204.111@tcp:13: 00000100:00000001:3.0:1788880184.302715:0:203453:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880184.302718:0:203453:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880184.302720:0:203453:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:3.0:1788880184.302721:0:203453:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880184.302728:0:203453:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.302735:0:203453:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880184.302737:0:203453:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000e1cb172e. 00000100:00000001:3.0:1788880184.302740:0:203453:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:3.0:1788880184.302742:0:203453:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:3.0:1788880184.302746:0:203453:0:(osc_request.c:3518:osc_statfs()) Process leaving 00000100:00000001:3.0:1788880184.302747:0:203453:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880184.302749:0:203453:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780636160/t0(0) o13->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:7/4 lens 224/368 e 0 to 0 dl 1788880200 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880184.302755:0:203453:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880184.302758:0:203453:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880184.302759:0:203453:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000d7b2c6b7. 02000000:00000001:3.0:1788880184.302762:0:203453:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880184.302764:0:203453:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880184.302765:0:203453:0:(genops.c:1244:class_import_put()) import 00000000beb50720 refcount=2 obd=lustre-OST0000-osc-ffff8ca787527000 00000020:00000001:3.0:1788880184.302767:0:203453:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880184.302769:0:203453:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 02000000:00000001:3.0:1788880184.302773:0:203453:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880184.302774:0:203453:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880184.302776:0:203453:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:3.0:1788880184.302779:0:203453:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880184.302780:0:203453:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00020000:00000004:3.0:1788880184.302783:0:203453:0:(obd_class.h:1117:obd_statfs()) lustre-OST0000-osc-ffff8ca787527000: update 00000000d48156b5 cache blocks 31672/35554 objects 49503/50000 00020000:00000001:3.0:1788880184.302786:0:203453:0:(obd_class.h:1136:obd_statfs()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:3.0:1788880184.302790:0:203453:0:(lov_obd.c:1147:lov_iocontrol()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.302792:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=0 : 0 : 0) 00000080:00000010:3.0:1788880184.302794:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.302797:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.302798:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.302862:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.302867:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.302875:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.302877:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.302881:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.302884:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.302891:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.302894:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.302895:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.302898:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302900:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302902:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.302904:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.302907:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302909:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302914:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.302915:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.302918:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.302919:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.302921:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.302923:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.302925:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.302926:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.302927:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.302929:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302931:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302932:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.302933:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.302935:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302936:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302938:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.302939:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.302940:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.302941:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.302942:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.302944:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.302945:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.302946:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.302947:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.302954:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302955:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302956:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.302957:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.302958:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302959:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302962:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.302962:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.302964:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.302964:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.302965:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.302967:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.302968:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.302969:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.302970:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.302972:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302973:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302975:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.302976:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.302978:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302980:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302983:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.302984:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.302986:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.302987:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.302989:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.302991:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.302993:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.302994:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.302995:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.302997:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.302998:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303004:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303006:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303008:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303010:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303015:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303017:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303019:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303020:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303023:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303026:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303027:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303029:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303029:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303032:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303033:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303035:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303045:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303051:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303054:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303058:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303059:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303061:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303062:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303064:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303066:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303067:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303068:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303069:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303070:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303071:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303072:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303073:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303078:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303079:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303082:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303082:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303084:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303085:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303086:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303087:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303088:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303089:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303090:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303091:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303092:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303093:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303094:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303095:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303096:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303098:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303099:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303102:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303102:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303104:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303106:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303107:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303108:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303108:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303110:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303112:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303113:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303116:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303119:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303121:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303124:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303131:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303134:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303135:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303137:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303140:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303141:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303143:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303144:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303146:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303148:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303149:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303151:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303153:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303155:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303166:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303167:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303168:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303169:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303171:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303173:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303174:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303175:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303176:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303178:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303179:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303181:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303182:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303183:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303184:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303186:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303187:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303188:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303188:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303193:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303194:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303196:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303196:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303197:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303198:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303199:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303200:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303201:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303202:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303203:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303206:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303207:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303208:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303209:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303210:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303211:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303212:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303212:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303212:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303213:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303214:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303215:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303216:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303218:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303218:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303220:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303221:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303222:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303222:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303223:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303224:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303225:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303228:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303228:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303230:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303231:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303259:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303261:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303264:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303266:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303271:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303272:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303278:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303280:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303282:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303284:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303286:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303287:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303288:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303290:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303292:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303293:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303295:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303296:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303297:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303300:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303301:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303303:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303303:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303305:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303306:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303307:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303308:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303308:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303310:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303315:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303316:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303317:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303319:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303320:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303323:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303324:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303327:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303328:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303329:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303331:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303333:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303334:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303335:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303337:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303339:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303340:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303342:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303343:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303345:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303348:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303349:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303352:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303352:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303354:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303356:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303358:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303359:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303360:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303362:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303363:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303365:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303367:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303374:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303376:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303381:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303382:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303385:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303386:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303388:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303390:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303392:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303393:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303394:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303396:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303398:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303399:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303402:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303404:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303405:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303408:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303409:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303411:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303412:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303414:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303416:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303417:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303418:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303419:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303422:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303423:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303425:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303426:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303428:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303430:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303432:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303437:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303440:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303441:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303443:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303445:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303446:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303447:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303448:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303451:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303452:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303454:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303455:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303457:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303459:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303463:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303464:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303466:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303467:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303469:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303471:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303472:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303473:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303475:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303477:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303479:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303480:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303482:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303483:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303485:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303488:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303488:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303491:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303492:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303499:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303501:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303502:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303504:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303504:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303507:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303508:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303509:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303511:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303513:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303515:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303519:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303520:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303522:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303523:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303525:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303527:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303528:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303529:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303530:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303532:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303534:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303535:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303547:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303549:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303551:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303554:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303554:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303557:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303558:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303560:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303562:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303563:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303570:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303572:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303574:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303576:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303577:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303579:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303581:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303582:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303586:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303587:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303590:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303591:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303593:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303595:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303597:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303598:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303599:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303601:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303602:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303604:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303606:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303607:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303609:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303611:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303612:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303614:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303615:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303617:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303619:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303620:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303621:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303622:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303624:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303630:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303631:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303633:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303635:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303637:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303641:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303641:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303644:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303645:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303647:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303648:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303649:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303650:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303651:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303653:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303655:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303656:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303658:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303660:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303661:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303664:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303665:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303667:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303668:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303670:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303672:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303673:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303674:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303675:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303677:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303679:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303681:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303683:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303689:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303691:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303695:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303696:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303699:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303700:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303702:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303704:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303705:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303707:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303707:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303710:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303711:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303713:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303715:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303717:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303718:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303721:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303722:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303724:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303725:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303727:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303729:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303730:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303731:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303732:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303734:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303736:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303737:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303739:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303741:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303742:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303745:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303749:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303752:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303753:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303755:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303757:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303758:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303760:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303761:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303763:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303765:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303766:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303768:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303770:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303772:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303776:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303776:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303779:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303779:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303781:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303783:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303784:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303785:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303786:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303788:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303789:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303791:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303793:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303794:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303796:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303799:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303800:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303802:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303803:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303820:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303823:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303825:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303826:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303827:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303830:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303831:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303833:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303835:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303837:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303839:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303843:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303844:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303847:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303848:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303850:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303852:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303853:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303854:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303855:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303857:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303859:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303860:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303861:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303863:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303865:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303867:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303868:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303870:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303870:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303872:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303873:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303875:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303881:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303882:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303884:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303886:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303887:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303889:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303891:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303892:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303896:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303897:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303900:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303901:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303903:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303905:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303906:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303907:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303908:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303909:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303911:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303912:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303914:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303915:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303917:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303919:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303920:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303922:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303923:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303924:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303925:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303927:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303928:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303928:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303930:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303936:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303938:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303939:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303941:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303942:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303946:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303947:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303949:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303950:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303952:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303953:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303955:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303956:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303957:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303959:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303960:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303961:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303963:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303965:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303966:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303969:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.303970:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.303972:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.303973:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.303974:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.303976:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.303978:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.303978:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.303980:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.303981:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303983:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303984:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.303987:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.303993:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303995:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.303999:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304000:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304002:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304003:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304005:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304007:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304008:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304009:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304010:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304012:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304013:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304014:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304016:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304018:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304019:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304021:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304022:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304024:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304025:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304027:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304029:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304030:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304031:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304032:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304034:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304036:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304037:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304039:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304041:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304042:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304045:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304052:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304055:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304056:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304058:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304060:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304062:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304063:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304064:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304066:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304068:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304069:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304071:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304073:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304074:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304079:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304080:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304082:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304083:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304084:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304086:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304088:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304089:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304090:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304092:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304093:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304095:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304097:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304098:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304100:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304103:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304104:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304106:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304107:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304115:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304117:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304119:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304120:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304121:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304123:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304125:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304126:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304128:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304131:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304132:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304138:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304139:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304142:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304143:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304145:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304147:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304149:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304150:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304151:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304152:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304154:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304156:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304157:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304159:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304161:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304163:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304164:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304166:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304167:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304169:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304171:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304172:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304178:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304179:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304182:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304184:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304185:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304187:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304189:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304191:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304195:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304196:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304199:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304200:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304202:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304204:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304205:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304206:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304207:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304209:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304211:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304212:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304214:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304216:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304217:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304219:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304220:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304222:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304223:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304224:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304226:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304227:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304228:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304229:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304231:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304261:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304263:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304266:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304269:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304271:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304276:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304277:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304280:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304281:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304284:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304287:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304288:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304290:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304291:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304294:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304296:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304297:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304299:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304302:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304303:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304307:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304308:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304310:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304311:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304314:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304316:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304318:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304319:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304320:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304322:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304324:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304325:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304327:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304334:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304336:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304340:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304341:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304344:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304345:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304347:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304349:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304350:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304352:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304353:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304355:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304356:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304358:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304360:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304361:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304363:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304365:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304366:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304368:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304369:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304371:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304373:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304375:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304375:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304376:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304378:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304380:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304381:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304383:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304385:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304386:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304389:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304395:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304399:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304400:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304402:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304404:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304406:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304407:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304408:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304411:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304412:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304414:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304416:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304418:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304420:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304424:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304426:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304428:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304429:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304431:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304433:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304434:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304435:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304436:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304438:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304440:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304442:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304443:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304445:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304447:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304450:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304451:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304453:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304454:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304461:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304464:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304465:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304466:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304467:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304469:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304471:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304472:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304474:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304476:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304478:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304482:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304483:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304486:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304487:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304489:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304491:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304493:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304494:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304495:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304497:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304499:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304500:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304502:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304504:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304505:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304508:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304509:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304511:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304512:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304515:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304517:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304518:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304525:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304525:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304528:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304530:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304531:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304533:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304535:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304537:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304542:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304543:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304546:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304547:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304549:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304551:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304552:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304553:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304554:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304556:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304558:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304559:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304561:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304562:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304564:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304567:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304568:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304570:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304571:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304574:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304575:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304577:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304578:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304579:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304581:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304587:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304589:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304591:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304593:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304595:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304600:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304601:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304604:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304605:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304607:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304609:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304611:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304612:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304613:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304615:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304617:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304618:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304620:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304622:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304624:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304627:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304628:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304630:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304631:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304633:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304635:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304637:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304637:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304639:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304641:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304642:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304644:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304645:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304653:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304654:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304659:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304659:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304662:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304663:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304665:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304667:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304668:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304670:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304670:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304673:0:203453:0:(lov_obd.c:1004:lov_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304674:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304675:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551605 : -11 : 0xfffffffffffffff5) 00000080:00000010:3.0:1788880184.304677:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304679:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304680:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551605 : -11 : fffffffffffffff5) 00000080:00000001:3.0:1788880184.304683:0:203453:0:(dir.c:2001:ll_dir_ioctl()) Process entered 00000080:00200080:3.0:1788880184.304684:0:203453:0:(dir.c:2003:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8ca7ba9480c8) cmd=c00866a4 arg=7ffc06e8ad20 00000080:00000001:3.0:1788880184.304687:0:203453:0:(llite_lib.c:3551:ll_iocontrol()) Process entered 00000020:00000001:3.0:1788880184.304687:0:203453:0:(class_obd.c:259:obd_ioctl_getdata()) Process entered 00000020:00000010:3.0:1788880184.304689:0:203453:0:(class_obd.c:286:obd_ioctl_getdata()) kmalloced '(data)': 592 at 000000006dbe0ab3. 00000020:00000001:3.0:1788880184.304691:0:203453:0:(class_obd.c:321:obd_ioctl_getdata()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304693:0:203453:0:(obd_class.h:1201:obd_iocontrol()) Process entered 00020000:00000001:3.0:1788880184.304693:0:203453:0:(lov_obd.c:972:lov_iocontrol()) Process entered 00020000:00000080:3.0:1788880184.304694:0:203453:0:(lov_obd.c:974:lov_iocontrol()) lustre-clilov-ffff8ca787527000: cmd=c00866a4 len=592 karg=ffff8ca79ce46800 uarg=0000000000000000 00020000:00000001:3.0:1788880184.304696:0:203453:0:(lov_obd.c:1000:lov_iocontrol()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000080:00000001:3.0:1788880184.304698:0:203453:0:(obd_class.h:1216:obd_iocontrol()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000080:00000001:3.0:1788880184.304700:0:203453:0:(llite_lib.c:3986:ll_obd_statfs()) Process leaving via out_statfs (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000080:00000010:3.0:1788880184.304701:0:203453:0:(llite_lib.c:3988:ll_obd_statfs()) kvfree 'data': 592 at 000000006dbe0ab3. 00000080:00000001:3.0:1788880184.304703:0:203453:0:(llite_lib.c:3589:ll_iocontrol()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000080:00000001:3.0:1788880184.304705:0:203453:0:(dir.c:2946:ll_dir_ioctl()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000080:00000001:3.0:1788880184.304714:0:203453:0:(dir.c:3016:ll_dir_release()) Process entered 00000080:00000001:3.0:1788880184.304721:0:203453:0:(file.c:475:ll_file_release()) Process entered 00000020:00000001:3.0:1788880184.304724:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 00000000c806c015 00000080:00200008:3.0:1788880184.304726:0:203453:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](00000000f8fa31aa), flags 100000 00000080:00400000:3.0:1788880184.304730:0:203453:0:(statahead.c:1721:ll_deauthorize_statahead()) deauthorize statahead for [0x200000007:0x1:0x0] 00000080:00000010:3.0:1788880184.304732:0:203453:0:(file.c:83:ll_file_data_put()) slab-freed '(lfd)': 288 at 00000000d4d75441. 00000080:00000001:3.0:1788880184.304737:0:203453:0:(file.c:498:ll_file_release()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000001:3.0:1788880184.304739:0:203453:0:(obd_sysfs.c:926:encode_fn_len()) Process filename at 00000000c806c015 00000080:00000008:3.0:1788880184.304740:0:203453:0:(file.c:523:ll_file_release()) COMPLETED file /:[0x200000007:0x1:0x0](00000000f8fa31aa), flags 100000, rc = 0 00000080:00000001:3.0:1788880184.304742:0:203453:0:(file.c:525:ll_file_release()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880184.304744:0:203453:0:(dir.c:3017:ll_dir_release()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880185.022426:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880185.022430:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880185.022433:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880186.046477:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880186.046483:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880186.046487:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788880187.070368:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880187.070372:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880187.070375:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00000001:3.0:1788880187.518302:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000001:3.0:1788880187.518309:0:203401:0:(mgc_request.c:120:config_log_put()) Process leaving 00000100:00000040:3.0:1788880188.030348:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305819095, at 4305819136 00000100:00000010:3.0:1788880188.030366:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001f846689. 00000100:00000040:2.0:1788880188.030507:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305819095, at 4305819136 00000100:00000010:2.0:1788880188.030514:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000dc3cc638. 00000100:00000040:2.0:1788880188.030535:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880188.030540:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880188.030546:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000020:00000040:2.0:1788880188.030550:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880188.030553:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880188.030556:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880188.030560:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880188.030567:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006505de7c. 00000100:00000001:2.0:1788880188.030574:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880188.030584:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d838cab4 x1875776780636288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880188.030593:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880188.030596:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880188.030601:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d838cab4] to pc [ptlrpcd_00_03+3] req@00000000d838cab4 x1875776780636288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880188.030625:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880188.030628:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880188.030632:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880188.030634:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880188.030637:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880188.030639:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880188.030642:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880188.030642:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880188.030644:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880188.030648:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000ea04942e. 00000100:00000001:2.0:1788880188.030651:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880188.030654:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880188.030657:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880188.030662:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d838cab4 x1875776780636288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880188.030675:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880188.030675:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880188.030678:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880188.030685:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880188.030687:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880188.030689:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880188.030691:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880188.030694:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880188.030697:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_00_00+0] req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880188.030702:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780636288:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880188.030711:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880188.030717:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880188.030720:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.030723:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880188.030723:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880188.030727:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:3.0:1788880188.030727:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003deec3bc. 02000000:00000001:3.0:1788880188.030732:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.030734:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880188.030752:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000078d5a15d. 00000020:00000040:2.0:1788880188.030756:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000010:3.0:1788880188.030756:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004100a205. 00000100:00000001:2.0:1788880188.030758:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880188.030761:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880188.030763:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880188.030767:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d24baaa0. 00000100:00000001:2.0:1788880188.030770:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880188.030774:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000078d5a15d x1875776780636544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880188.030780:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880188.030781:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880188.030784:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000078d5a15d] to pc [ptlrpcd_00_01+1] req@0000000078d5a15d x1875776780636544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880188.030791:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880188.030793:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880188.030796:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880188.030799:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 00000020:00000040:2.0:1788880188.030801:0:53635:0:(genops.c:1231:class_import_get()) import 00000000beb50720 refcount=3 obd=lustre-OST0000-osc-ffff8ca787527000 00000100:00000001:2.0:1788880188.030802:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880188.030808:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880188.030810:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880188.030815:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 02000000:00000010:2.0:1788880188.030822:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000200:3.0:1788880188.030823:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780636288, portal 25 00000100:00000001:2.0:1788880188.030827:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880188.030828:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880188.030830:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000100:00000040:2.0:1788880188.030835:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:3.0:1788880188.030838:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d838cab4 x1875776780636288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880204 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880188.030855:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880188.030857:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880188.030858:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880188.030862:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880188.030866:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_02+2] req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880188.030866:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ec90f18d. 00000100:00000200:3.0:1788880188.030871:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780636288, offset 0 00000100:00000001:2.0:1788880188.030882:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880188.030883:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880188.030887:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11156) 00000400:00000200:3.0:1788880188.030898:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880188.030904:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880188.030915:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880188.030920:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880188.030923:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880188.030927:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [37:37:256:4294967295] 192.168.204.111@tcp LPNI seq info [37:37:8:4294967295] 00000400:00000200:3.0:1788880188.030938:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880188.030944:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880188.030949:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000749ed09e. 00000800:00000200:3.0:1788880188.030954:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880188.030962:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880188.030966:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000749ed09e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880188.030978:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880188.030982:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305824207 00000100:00000001:3.0:1788880188.030988:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305819094 00000100:00000001:3.0:1788880188.030991:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305824207 00000100:00000010:3.0:1788880188.030995:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001f846689. 00000100:00000001:3.0:1788880188.031000:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880188.031010:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305824207 00000100:00000010:3.0:1788880188.031013:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000029868be. 00000100:00000001:3.0:1788880188.031016:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880188.031018:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305824207 00000100:00000010:3.0:1788880188.031020:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000016ab1a15. 00000100:00000001:2.0:1788880188.031021:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880188.031023:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880188.031028:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880188.031028:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880188.031032:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880188.031033:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880188.031037:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880188.031037:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880188.031040:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880188.031047:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880188.031049:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880188.031050:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880188.031053:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880188.031054:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880188.031057:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880188.031057:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880188.031061:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880188.031061:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880188.031065:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880188.031067:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880188.031072:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780636672:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880188.031075:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880188.031078:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880188.031080:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880188.031082:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880188.031086:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000014e021ca. 02000000:00000001:2.0:1788880188.031091:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880188.031096:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000400:00000010:2.0:1788880188.031101:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001e963647. 00000100:00000200:2.0:1788880188.031103:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780636672, portal 4 00000100:00000001:2.0:1788880188.031105:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880188.031106:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938435456 : -126823771116160 : ffff8ca78a2c6d80) 00000800:00000200:3.0:1788880188.031112:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:2.0:1788880188.031113:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880204 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:3.0:1788880188.031117:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000749ed09e. 00000400:00000200:3.0:1788880188.031122:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880188.031132:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788880188.031132:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:2.0:1788880188.031138:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880188.031139:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ec90f18d 00000400:00000010:2.0:1788880188.031142:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000006bff322d. 00000400:00000010:3.0:1788880188.031143:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ec90f18d. 00000100:00000200:2.0:1788880188.031148:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875776780636672, offset 0 00000100:00000001:3.0:1788880188.031149:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880188.031155:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780636288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880204 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880188.031159:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880188.031167:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880188.031169:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788880188.031171:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880188.031175:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780636288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880204 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880188.031177:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880188.031184:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880188.031185:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880188.031187:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880188.031188:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880188.031189:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:2.0:1788880188.031192:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:0.0:1788880188.031196:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880188.031198:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880188.031207:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880188.031208:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880188.031211:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880188.031213:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880188.031214:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880188.031218:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780636416:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880188.031222:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880188.031225:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880188.031226:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880188.031229:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880188.031253:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000605d7309. 02000000:00000001:0.0:1788880188.031255:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880188.031260:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050ac3db6. 00000400:00000010:0.0:1788880188.031265:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 00000100:00000200:0.0:1788880188.031268:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780636416, portal 10 00000100:00000001:0.0:1788880188.031270:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880188.031272:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000100:00000040:0.0:1788880188.031277:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880188.031296:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880188.031299:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880188.031301:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 00000100:00000200:0.0:1788880188.031304:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780636416, offset 0 00000400:00000200:0.0:1788880188.031310:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880188.031476:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [38:38:255:4294967295] 192.168.204.111@tcp LPNI seq info [38:38:7:4294967295] 00000400:00000200:2.0:1788880188.031484:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880188.031488:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880188.031491:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008eccda10. 00000800:00000200:2.0:1788880188.031496:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880188.031501:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880188.031504:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008eccda10 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880188.031511:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880188.031515:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: using a timeout of 2538 usecs (3 jiffies) 00000100:00000001:2.0:1788880188.031518:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305819140 00000100:00000001:2.0:1788880188.031520:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305824207 00000100:00000001:2.0:1788880188.031522:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305819140 00000100:00000001:2.0:1788880188.031524:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305824207 00000100:00000001:2.0:1788880188.031526:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305819140 00000100:00000001:2.0:1788880188.031528:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305824207 00000100:00000001:2.0:1788880188.031529:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305819140 00000100:00000001:2.0:1788880188.031531:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305824207 00000100:00000001:2.0:1788880188.031539:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880188.031541:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.031543:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.031544:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.031548:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880188.031549:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880188.031551:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880188.031553:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880188.031555:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.031557:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880188.031560:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880188.031564:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008eccda10. 00000400:00000200:3.0:1788880188.031568:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880188.031572:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880188.031581:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000006bff322d 00000400:00000010:3.0:1788880188.031582:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000006bff322d. 00000100:00000001:3.0:1788880188.031585:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880188.031588:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880204 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880188.031594:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880188.031596:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880204 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880188.031602:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880188.031603:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:0.0:1788880188.032166:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880188.032175:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880188.032179:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880188.032183:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880188.032188:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880188.032191:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [39:39:256:4294967295] 192.168.204.111@tcp LPNI seq info [39:39:8:4294967295] 00000400:00000200:0.0:1788880188.032204:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880188.032225:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880188.032289:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b899fe8d. 00000800:00000200:0.0:1788880188.032305:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880188.032325:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880188.032343:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b899fe8d type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880188.032367:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880188.032375:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 2537271 usecs (2538 jiffies) 00000100:00000001:0.0:1788880188.032381:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305821676 00000100:00000001:0.0:1788880188.032389:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305824207 00000100:00000001:0.0:1788880188.032395:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305821676 00000100:00000001:0.0:1788880188.032400:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305824207 00000100:00000001:0.0:1788880188.032404:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305821676 00000100:00000001:0.0:1788880188.032407:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305824207 00000100:00000001:0.0:1788880188.032410:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305821676 00000100:00000001:0.0:1788880188.032418:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305824207 00000800:00000200:3.0:1788880188.032423:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880188.032427:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b899fe8d. 00000100:00000001:0.0:1788880188.032436:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880188.032438:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880188.032444:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880188.032453:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880188.032453:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880188.032464:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880188.032465:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000400:00000010:3.0:1788880188.032471:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000001:0.0:1788880188.032475:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880188.032478:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880188.032479:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000200:3.0:1788880188.032490:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880188.032493:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880188.032498:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880188.032504:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880188.032512:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880188.032524:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880188.032530:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880188.032541:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880188.032545:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880188.032585:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880188.032589:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:3.0:1788880188.032592:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880188.032596:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880188.032599:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000641230df 00000400:00000010:3.0:1788880188.032601:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000641230df. 00000100:00000001:3.0:1788880188.032604:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880188.032607:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000078d5a15d x1875776780636544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880188.032636:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880188.032638:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000078d5a15d x1875776780636544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880188.032645:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880188.032646:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880188.032935:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.032942:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880188.032944:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.032947:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880188.032952:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880188.032962:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b01080 00000400:00000200:3.0:1788880188.032969:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x21d [1] + 224 00000800:00000001:3.0:1788880188.032974:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.032978:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880188.032980:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880188.032982:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880188.032986:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880188.032989:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780636288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880188.033038:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d838cab4 x1875776780636288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880188.033062:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880188.033068:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.033071:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880188.033073:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.033074:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880188.033078:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880188.033084:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 4 MB=0x6aa0250b01200 00000400:00000200:3.0:1788880188.033152:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.204.111@tcp of length 224/224 into md 0x225 [1] + 224 00000800:00000001:3.0:1788880188.033158:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.033171:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880188.033174:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880188.033180:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880188.033188:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880188.033194:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880188.033216:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880188.033271:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880188.033279:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.033282:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.033287:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880188.033291:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880188.033298:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880188.033301:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880188.033303:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:2.0:1788880188.033311:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001e963647 00000400:00000010:2.0:1788880188.033316:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001e963647. 00000100:00000001:2.0:1788880188.033327:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880188.033340:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880188.033356:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880188.033379:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880188.033382:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.033392:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880188.033397:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880188.033401:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880188.033406:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880188.033412:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880188.033422:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000040ba1a3b time=6 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880188.033431:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003184a8ce time=6 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880188.033443:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880188.033448:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880188.033456:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880188.033464:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880188.033521:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.033531:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880188.033538:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-OST0000-osc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880188.033544:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880188.033549:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880188.033555:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880188.033567:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880188.033571:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880188.033575:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880188.033586:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880188.033633:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780636672:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880188.033640:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.033644:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880188.033648:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000175072aa x1875776780636672/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880188.033657:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880188.033659:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880188.033661:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000014e021ca. 02000000:00000001:2.0:1788880188.033664:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880188.033666:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880188.033672:0:201287:0:(genops.c:1244:class_import_put()) import 00000000beb50720 refcount=2 obd=lustre-OST0000-osc-ffff8ca787527000 00000020:00000001:2.0:1788880188.033677:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880188.033681:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 02000000:00000001:2.0:1788880188.033687:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880188.033689:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880188.033693:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000175072aa. 00000100:00000001:2.0:1788880188.033704:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880188.033708:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.033716:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.033721:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880188.033725:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.033731:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880188.033748:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880188.034007:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.034013:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880188.034015:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.034017:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880188.034022:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880188.034029:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b01180 00000400:00000200:3.0:1788880188.034034:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x235 [1] + 224 00000800:00000001:3.0:1788880188.034036:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.034040:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880188.034041:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880188.034043:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880188.034045:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880188.034047:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000078d5a15d x1875776780636544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880188.034126:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000078d5a15d x1875776780636544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880188.034143:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880188.034148:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.034152:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880188.034154:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.034161:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880188.034204:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880188.034218:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b01100 00000400:00000200:3.0:1788880188.034229:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x22d [1] + 224 00000800:00000001:3.0:1788880188.034266:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.034276:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880188.034279:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880188.034286:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880188.034289:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880188.034295:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880188.034312:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880188.034334:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880188.034338:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.034340:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880188.034411:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880188.034415:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880188.034420:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880188.034422:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880188.034428:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050ac3db6. 00000400:00000200:0.0:1788880188.034433:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000400:00000010:0.0:1788880188.034435:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000001:0.0:1788880188.034438:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880188.034445:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880188.034454:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880188.034461:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880188.034462:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880188.034464:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880188.034466:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880188.034468:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880188.034470:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880188.034472:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880188.034478:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=6 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880188.034481:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=6 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880188.034483:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880188.034485:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880188.034487:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880188.034490:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880188.034493:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880188.034501:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880188.034504:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880188.034505:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880188.034509:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880188.034516:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880188.034518:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880188.034520:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880188.034522:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880188.034529:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780636416:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880188.034534:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880188.034536:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880188.034538:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780636416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880204 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880188.034544:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880188.034546:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880188.034548:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000605d7309. 02000000:00000001:0.0:1788880188.034551:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880188.034553:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880188.034554:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880188.034556:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880188.034557:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000ea04942e. 02000000:00000001:0.0:1788880188.034560:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880188.034561:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880188.034562:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:0.0:1788880188.034565:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880188.034566:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880188.034568:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880188.034572:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880188.034574:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880188.034576:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880188.034578:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.040047:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880188.040056:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880188.040059:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880188.040065:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780636800/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880188.040076:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880188.040079:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880188.040082:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880188.040085:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880188.040087:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880188.040089:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780636800:192.168.204.111@tcp:38: 00000100:00000001:2.0:1788880188.040095:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880188.040101:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880188.040105:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880188.040109:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880188.040117:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000de3147fa. 02000000:00000001:2.0:1788880188.040122:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880188.040130:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000400:00000010:2.0:1788880188.040137:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001e963647. 00000100:00000200:2.0:1788880188.040146:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780636800, portal 10 00000100:00000001:2.0:1788880188.040149:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880188.040151:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000020:00000010:2.0:1788880188.040158:0:201284:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 00000000bb2a2be6. 00000100:00000040:2.0:1788880188.040163:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780636800/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880198 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880188.040170:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880188.040173:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880188.040175:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cc594c. 00000100:00000200:2.0:1788880188.040178:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780636800, offset 0 00000400:00000200:2.0:1788880188.040197:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880188.040206:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880188.040210:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880188.040212:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880188.040215:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880188.040217:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880188.040220:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [41:41:256:4294967295] 192.168.204.111@tcp LPNI seq info [41:41:8:4294967295] 00000400:00000200:2.0:1788880188.040228:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880188.040232:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880188.040268:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e5360691. 00000800:00000200:2.0:1788880188.040273:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880188.040280:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880188.040282:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e5360691 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:2.0:1788880188.040293:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880188.040297:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 211853 usecs (212 jiffies) 00000100:00000001:2.0:1788880188.040301:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305819358 00000100:00000001:2.0:1788880188.040303:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305824207 00000100:00000001:2.0:1788880188.040305:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305819358 00000100:00000001:2.0:1788880188.040306:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305824207 00000100:00000001:2.0:1788880188.040308:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305819358 00000100:00000001:2.0:1788880188.040309:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305824207 00000100:00000001:2.0:1788880188.040310:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305819358 00000100:00000001:2.0:1788880188.040312:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305824207 00000100:00000001:2.0:1788880188.040313:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880188.040315:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.040323:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.040325:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.040329:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880188.040336:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:2.0:1788880188.040338:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880188.040339:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880188.040341:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.040342:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880188.045142:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.045155:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880188.045159:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.045161:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880188.045171:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880188.045187:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b01280 00000400:00000200:3.0:1788880188.045195:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x23d [1] + 0 00000800:00000001:3.0:1788880188.045202:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.045208:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880188.045210:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880188.045214:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880188.045220:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880188.045226:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780636800/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880198 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880188.045266:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@0000000077a1dd13 x1875776780636800/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880198 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880188.045289:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880188.045299:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880188.045302:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.045309:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880188.045315:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880188.045321:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880188.045323:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880188.045325:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:2.0:1788880188.045329:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001e963647 00000400:00000010:2.0:1788880188.045331:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001e963647. 00000100:00000001:2.0:1788880188.045334:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880188.045338:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780636800/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880198 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880188.045346:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780636800/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880198 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880188.045352:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880188.045354:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.045356:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880188.045359:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880188.045361:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880188.045363:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880188.045366:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880188.045374:0:201284:0:(import.c:2149:obd_at_measure()) add 0 to 0000000028cb2d55 time=16 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880188.045377:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=16 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880188.045379:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880188.045381:0:201284:0:(client.c:1493:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:2.0:1788880188.045383:0:201284:0:(client.c:1698:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:2.0:1788880188.045387:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780636800/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880198 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880188.045393:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:2.0:1788880188.045395:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880188.045397:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.045399:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:2.0:1788880188.045407:0:201284:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:2.0:1788880188.045410:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788880188.045414:0:201284:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.111@tcp failed (-19) 00000100:00000001:2.0:1788880188.045416:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:2.0:1788880188.045419:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780636800/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880198 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880188.045426:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780636800:192.168.204.111@tcp:38: 00000100:00000001:2.0:1788880188.045431:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.045433:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880188.045435:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780636800/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880198 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880188.045442:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880188.045444:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880188.045446:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000de3147fa. 02000000:00000001:2.0:1788880188.045449:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880188.045451:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880188.045453:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880188.045455:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880188.045457:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 000000009ef860f0. 02000000:00000001:2.0:1788880188.045460:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880188.045462:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880188.045463:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:2.0:1788880188.045466:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880188.045467:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.045469:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880188.045476:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880188.045477:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880188.045479:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880188.045481:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880188.094526:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880188.094530:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880188.094532:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788880189.118449:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880189.118455:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880189.118458:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880190.143080:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880190.143084:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880190.143087:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788880191.166403:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880191.166407:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880191.166410:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880192.190520:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880192.190524:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880192.190527:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880193.150367:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305824207, at 4305824256 00000100:00000040:2.0:1788880193.150367:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305824207, at 4305824256 00000100:00000010:0.0:1788880193.150384:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c451d2b9. 00000100:00000010:2.0:1788880193.150386:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000029868be. 00000100:00000040:3.0:1788880193.150687:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880193.150692:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880193.150706:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000020:00000040:3.0:1788880193.150710:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880193.150715:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880193.150718:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880193.150722:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880193.150730:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:3.0:1788880193.150737:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880193.150746:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002a42fbe1 x1875776780636928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880193.150756:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880193.150759:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880193.150764:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_03+3] req@000000002a42fbe1 x1875776780636928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880193.150785:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880193.150788:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880193.150790:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880193.150792:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000020:00000040:3.0:1788880193.150808:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880193.150810:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880193.150812:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880193.150814:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880193.150817:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:2.0:1788880193.150820:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880193.150821:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880193.150825:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000028715c64 x1875776780637056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788880193.150832:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:3.0:1788880193.150835:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880193.150836:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880193.150838:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880193.150843:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788880193.150844:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_00+0] req@0000000028715c64 x1875776780637056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880193.150847:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880193.150850:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880193.150853:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880193.150856:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880193.150861:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780636928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880193.150867:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880193.150869:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880193.150872:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880193.150873:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880193.150875:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880193.150876:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880193.150880:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780636928:192.168.204.111@tcp:400: 00000100:00080000:3.0:1788880193.150880:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788880193.150888:0:53635:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00000001:2.0:1788880193.150889:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880193.150892:0:53635:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00000001:0.0:1788880193.150894:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880193.150896:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00080000:3.0:1788880193.150896:0:53635:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:0.0:1788880193.150898:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880193.150899:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880193.150901:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880193.150903:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880193.150904:0:53635:0:(import.c:485:import_select_connection()) Process entered 02000000:00000010:2.0:1788880193.150908:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004d9329ab. 00000100:00000040:0.0:1788880193.150909:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780637056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788880193.150911:0:53635:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 11151 02000000:00000001:2.0:1788880193.150913:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880193.150920:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000100:00000001:0.0:1788880193.150922:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880193.150924:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880193.150927:0:53635:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 02000000:00000001:0.0:1788880193.150928:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:2.0:1788880193.150928:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001e963647. 02000000:00000001:0.0:1788880193.150932:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00001000:3.0:1788880193.150933:0:53635:0:(import.c:2149:obd_at_measure()) add 10 to 0000000040f48a4f time=21 v=5 (1 0 0 0) 02000000:00000001:0.0:1788880193.150936:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880193.150937:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780636928, portal 25 00000100:00001000:3.0:1788880193.150939:0:53635:0:(import.c:2215:obd_at_measure()) AT 0000000040f48a4f change: old=5 new=10 delta=5 (val=10) hist 10 0 0 0 00000100:00000001:2.0:1788880193.150942:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00100000:0.0:1788880193.150943:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780637056:192.168.204.111@tcp:400: 00000100:00080000:3.0:1788880193.150946:0:53635:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: tried all connections, increasing latency to 5s 00000100:00000001:2.0:1788880193.150947:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000100:00000001:0.0:1788880193.150948:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880193.150954:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880193.150955:0:53635:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 02000000:00000001:0.0:1788880193.150957:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880193.150958:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780636928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880209 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880193.150962:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880193.150962:0:53635:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000001:2.0:1788880193.150968:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880193.150968:0:53635:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 02000000:00000010:0.0:1788880193.150974:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000605d7309. 00000100:00000001:3.0:1788880193.150974:0:53635:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000040:2.0:1788880193.150975:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880193.150982:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000023cb0718. 00000020:00000001:3.0:1788880193.150983:0:53635:0:(genops.c:813:class_conn2export()) Process entered 02000000:00000001:0.0:1788880193.150984:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880193.150986:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780636928, offset 0 00000020:00000040:3.0:1788880193.150986:0:53635:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:3.0:1788880193.150996:0:53635:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000400:00000010:0.0:1788880193.150998:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050ac3db6. 00000400:00000200:2.0:1788880193.151000:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000020:00000040:3.0:1788880193.151006:0:53635:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 00000020:00000001:3.0:1788880193.151011:0:53635:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000400:00000010:0.0:1788880193.151012:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 00000400:00000200:2.0:1788880193.151019:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000020:00000001:3.0:1788880193.151021:0:53635:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000100:00000200:0.0:1788880193.151024:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780637056, portal 10 00000100:00000040:3.0:1788880193.151029:0:53635:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:0.0:1788880193.151031:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880193.151037:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000100:00000001:3.0:1788880193.151040:0:53635:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000400:00000200:2.0:1788880193.151045:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880193.151047:0:53635:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000040:0.0:1788880193.151048:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780637056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880209 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880193.151053:0:53635:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000400:00000200:2.0:1788880193.151054:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000020:00000040:3.0:1788880193.151059:0:53635:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 00000400:00000200:2.0:1788880193.151063:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880193.151065:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:2.0:1788880193.151070:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00080000:3.0:1788880193.151072:0:53635:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: import 00000000b98b8a4e using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000100:00000001:3.0:1788880193.151078:0:53635:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880193.151080:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880193.151081:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [42:42:256:4294967295] 192.168.204.111@tcp LPNI seq info [42:42:8:4294967295] 02000000:00000001:3.0:1788880193.151084:0:53635:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 00000400:00000010:0.0:1788880193.151085:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000033f4d237. 00000100:00000200:0.0:1788880193.151089:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780637056, offset 0 02000000:00000001:3.0:1788880193.151092:0:53635:0:(sec.c:1628:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880193.151098:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880193.151098:0:53635:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880193.151102:0:53635:0:(obd_class.h:889:obd_reconnect()) Process entered 00000400:00000200:2.0:1788880193.151104:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000008:00100000:3.0:1788880193.151108:0:53635:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 2097152, lost: 0. 00000800:00000200:2.0:1788880193.151113:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880193.151114:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000008:00000001:3.0:1788880193.151115:0:53635:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880193.151119:0:53635:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880193.151122:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:2.0:1788880193.151127:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5d1e8ed. 00000100:00000010:3.0:1788880193.151127:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000400:00000200:0.0:1788880193.151128:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880193.151134:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000020:00000040:3.0:1788880193.151134:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000800:00000200:2.0:1788880193.151140:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880193.151141:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880193.151141:0:53635:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 02000000:00000001:3.0:1788880193.151145:0:53635:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880193.151149:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [43:43:255:4294967295] 192.168.204.111@tcp LPNI seq info [43:43:7:4294967295] 00000100:00000001:3.0:1788880193.151149:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880193.151153:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000800:00000200:2.0:1788880193.151155:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:3.0:1788880193.151156:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880193.151163:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5d1e8ed type 1, nob 320 niov 1 nkiov 1 02000000:00000010:3.0:1788880193.151164:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 000000006dbe0ab3. 00000400:00000200:0.0:1788880193.151170:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880193.151171:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880193.151178:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00100000:3.0:1788880193.151181:0:53635:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ca4fc672 x1875776780637184/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000010:0.0:1788880193.151185:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 00000100:00000001:2.0:1788880193.151185:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880193.151193:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305829327 00000800:00000200:0.0:1788880193.151196:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880193.151196:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880193.151199:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880193.151200:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b63100a0. 00000800:00000200:0.0:1788880193.151206:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880193.151208:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000040:3.0:1788880193.151209:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_rcv+-1] req@00000000ca4fc672 x1875776780637184/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880193.151211:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880193.151217:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305829327 00000100:00000001:0.0:1788880193.151221:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880193.151224:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000488d1158. 00000100:00000040:0.0:1788880193.151228:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 2307036 usecs (2308 jiffies) 00000100:00000001:2.0:1788880193.151231:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880193.151264:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305826565 00000100:00000001:2.0:1788880193.151265:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305829328 00000100:00000001:0.0:1788880193.151269:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305829327 00000100:00000010:2.0:1788880193.151274:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000059817523. 00000100:00000001:0.0:1788880193.151276:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305826565 00000100:00000001:2.0:1788880193.151280:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880193.151283:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305829327 00000100:00000001:2.0:1788880193.151285:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305829328 00000100:00000001:0.0:1788880193.151287:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305826565 00000100:00000001:2.0:1788880193.151290:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305824207 00000100:00000001:0.0:1788880193.151293:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305829328 00000100:00000001:0.0:1788880193.151297:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305826565 00000100:00000001:3.0:1788880193.151297:0:53635:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880193.151299:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880193.151304:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305829328 00000100:00000001:2.0:1788880193.151304:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880193.151304:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880193.151316:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880193.151316:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880193.151318:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880193.151324:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880193.151325:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880193.151325:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880193.151332:0:53635:0:(genops.c:1231:class_import_get()) import 00000000beb50720 refcount=3 obd=lustre-OST0000-osc-ffff8ca787527000 00000100:00000001:0.0:1788880193.151336:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880193.151345:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880193.151347:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880193.151347:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880193.151349:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880193.151352:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880193.151354:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880193.151355:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880193.151360:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000001:3.0:1788880193.151360:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880193.151368:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880193.151369:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:0.0:1788880193.151374:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880193.151374:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780637184/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880193.151377:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880193.151384:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880193.151398:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID req@00000000e9e280b6 x1875776780637312/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880193.151411:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880193.151432:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880193.151435:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880193.151435:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880193.151438:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880193.151439:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880193.151443:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880193.151445:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880193.151448:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_01+1] req@00000000e9e280b6 x1875776780637312/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880193.151453:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780637184:192.168.204.111@tcp:38: 00000100:00000001:2.0:1788880193.151466:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880193.151470:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880193.151475:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880193.151483:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880193.151488:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000005e22417f. 02000000:00000001:2.0:1788880193.151494:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880193.151497:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000038fe0168. 00000100:00000001:3.0:1788880193.151500:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880193.151509:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000841b943f. 00000100:00000040:3.0:1788880193.151511:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11161) 00000100:00000200:2.0:1788880193.151524:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780637184, portal 10 00000100:00000001:2.0:1788880193.151529:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880193.151531:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000040:2.0:1788880193.151538:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780637184/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880208 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880193.151549:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880193.151553:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880193.151558:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880193.151560:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880193.151562:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880193.151564:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:2.0:1788880193.151564:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c3a5d4ae. 00000100:00000001:3.0:1788880193.151566:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880193.151569:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880193.151570:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780637184, offset 0 00000100:00000001:0.0:1788880193.151573:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880193.151574:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780637312/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880193.151584:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880193.151589:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880193.151592:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880193.151595:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:2.0:1788880193.151597:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880193.151598:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880193.151603:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880193.151607:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:3.0:1788880193.151613:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780637312:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880193.151622:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880193.151626:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880193.151628:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880193.151648:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1788880193.151649:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880193.151651:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880193.151656:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880193.151662:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002f5492f5. 00000400:00000200:2.0:1788880193.151664:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [44:44:255:4294967295] 192.168.204.111@tcp LPNI seq info [44:44:7:4294967295] 02000000:00000001:3.0:1788880193.151668:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880193.151675:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000e0ef94e8. 00000400:00000010:3.0:1788880193.151681:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ec90f18d. 00000100:00000200:3.0:1788880193.151685:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780637312, portal 4 00000100:00000001:3.0:1788880193.151689:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880193.151691:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000400:00000200:2.0:1788880193.151693:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788880193.151701:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780637312/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880209 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880193.151705:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880193.151717:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f9812869. 00000100:00000001:3.0:1788880193.151724:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1788880193.151729:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880193.151744:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880193.151754:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f9812869 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:2.0:1788880193.151760:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880193.151768:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 182543 usecs (183 jiffies) 00000100:00000001:2.0:1788880193.151773:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305824440 00000100:00000001:2.0:1788880193.151782:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305829327 00000100:00000001:2.0:1788880193.151785:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305824440 00000100:00000001:2.0:1788880193.151786:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305829327 00000100:00000001:2.0:1788880193.151789:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305824440 00000100:00000001:2.0:1788880193.151791:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305829328 00000100:00000001:2.0:1788880193.151797:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305824440 00000100:00000001:2.0:1788880193.151801:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305829328 00000100:00000040:3.0:1788880193.151803:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880193.151810:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788880193.151812:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000035d99bc2. 00000100:00000001:2.0:1788880193.151816:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880193.151818:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875776780637312, offset 0 00000100:00000001:2.0:1788880193.151820:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880193.151824:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880193.151828:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880193.151831:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880193.151834:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000400:00000200:3.0:1788880193.151836:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880193.151838:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880193.151842:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880193.151844:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880193.151845:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880193.151848:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880193.151850:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880193.151854:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880193.151858:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880193.151866:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [45:45:255:4294967295] 192.168.204.111@tcp LPNI seq info [45:45:7:4294967295] 00000400:00000200:3.0:1788880193.151886:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880193.151902:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880193.151910:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000055d7d92. 00000800:00000200:3.0:1788880193.151966:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880193.151973:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880193.151976:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000055d7d92 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880193.151987:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880193.151990:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: using a timeout of 2740 usecs (3 jiffies) 00000100:00000001:3.0:1788880193.151992:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305824260 00000100:00000001:3.0:1788880193.152001:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305829327 00000100:00000001:3.0:1788880193.152004:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305824260 00000100:00000001:3.0:1788880193.152006:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305829327 00000100:00000001:3.0:1788880193.152008:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305824260 00000100:00000001:3.0:1788880193.152010:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305829328 00000100:00000001:3.0:1788880193.152013:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305824260 00000100:00000001:3.0:1788880193.152015:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305829328 00000100:00000001:3.0:1788880193.152017:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880193.152019:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880193.152021:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880193.152023:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880193.152028:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880193.152030:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880193.152032:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880193.152033:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880193.152035:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880193.152037:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880193.154791:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880193.154848:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880193.154852:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880193.154855:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880193.154866:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880193.154890:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b01300 00000400:00000200:3.0:1788880193.154899:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x245 [1] + 224 00000800:00000001:3.0:1788880193.154905:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880193.154909:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880193.154912:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880193.154916:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880193.154924:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880193.154930:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780636928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880193.154960:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780636928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880193.154979:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880193.154988:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880193.154992:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880193.154994:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880193.154996:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880193.155000:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880193.155007:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 4 MB=0x6aa0250b01480 00000400:00000200:3.0:1788880193.155013:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.204.111@tcp of length 224/224 into md 0x25d [1] + 224 00000800:00000001:3.0:1788880193.155017:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880193.155020:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880193.155021:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880193.155026:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880193.155026:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880193.155032:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:2.0:1788880193.155033:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880193.155037:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880193.155045:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:2.0:1788880193.155049:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001e963647 00000400:00000010:2.0:1788880193.155051:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001e963647. 00000100:00000001:2.0:1788880193.155055:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880193.155057:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880193.155060:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780636928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880193.155064:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780637312/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880193.155083:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780636928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880193.155086:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780637312/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880193.155102:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880193.155106:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880193.155108:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880193.155112:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880193.155114:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880193.155116:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880193.155118:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880193.155124:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=21 v=5 (5 0 0 0) 00000100:00000001:3.0:1788880193.155126:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:2.0:1788880193.155130:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=21 v=5 (1 0 0 0) 00000800:00000001:3.0:1788880193.155133:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880193.155134:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880193.155137:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880193.155140:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:3.0:1788880193.155140:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880193.155144:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:2.0:1788880193.155145:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880193.155147:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880193.155149:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880193.155155:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000040:2.0:1788880193.155157:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780636928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880193.155166:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b01380 00000100:00000001:2.0:1788880193.155169:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880193.155173:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:3.0:1788880193.155175:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x24d [1] + 224 00000100:00000001:2.0:1788880193.155176:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880193.155183:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:2.0:1788880193.155184:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780636928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880193.155191:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:2.0:1788880193.155194:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780636928:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880193.155201:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880193.155203:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880193.155207:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:3.0:1788880193.155208:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:2.0:1788880193.155212:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780636928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880193.155214:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880193.155219:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780637056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880193.155265:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880193.155269:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880193.155273:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004d9329ab. 00000100:00000040:3.0:1788880193.155274:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780637056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880193.155280:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880193.155284:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880193.155287:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880193.155290:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880193.155290:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:2.0:1788880193.155293:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 00000800:00000001:3.0:1788880193.155297:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:2.0:1788880193.155300:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000800:00000001:3.0:1788880193.155302:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880193.155303:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880193.155307:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:2.0:1788880193.155311:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880193.155313:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880193.155315:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880193.155316:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880193.155320:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880193.155324:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880193.155326:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880193.155327:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:0.0:1788880193.155328:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050ac3db6. 00000100:00000001:2.0:1788880193.155331:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880193.155334:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000100:00000001:2.0:1788880193.155335:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788880193.155337:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000001:2.0:1788880193.155339:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880193.155342:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880193.155347:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780637056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880193.155356:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780637056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880193.155364:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880193.155365:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880193.155368:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880193.155370:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880193.155371:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880193.155373:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880193.155375:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880193.155380:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=11 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880193.155384:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=11 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880193.155388:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880193.155390:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880193.155391:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880193.155394:0:201285:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880193.155402:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880193.155404:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880193.155405:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880193.155409:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880193.155415:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780637056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880193.155429:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880193.155434:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880193.155436:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880193.155443:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780637056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880193.155463:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780637056:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880193.155470:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880193.155475:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880193.155481:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780637056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880209 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880193.155491:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880193.155494:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880193.155498:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000605d7309. 02000000:00000001:0.0:1788880193.155501:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880193.155502:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880193.155504:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880193.155508:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880193.155511:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 02000000:00000001:0.0:1788880193.155515:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880193.155516:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880193.155517:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:0.0:1788880193.155519:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880193.155520:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880193.155522:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880193.155529:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880193.155531:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880193.155536:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880193.155546:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880193.165393:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880193.165398:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880193.165404:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880193.165405:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880193.165408:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000038fe0168. 00000400:00000200:2.0:1788880193.165412:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000841b943f 00000400:00000010:2.0:1788880193.165415:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000841b943f. 00000100:00000001:2.0:1788880193.165418:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880193.165422:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780637184/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880208 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880193.165432:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780637184/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880208 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880193.165439:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880193.165440:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880193.165443:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880193.165446:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880193.165449:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880193.165463:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880193.165466:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880193.165474:0:201284:0:(import.c:2149:obd_at_measure()) add 0 to 0000000028cb2d55 time=21 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880193.165478:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=21 v=10 (10 0 0 0) 00000100:00000001:2.0:1788880193.165481:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880193.165483:0:201284:0:(client.c:1493:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:2.0:1788880193.165486:0:201284:0:(client.c:1698:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:2.0:1788880193.165490:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780637184/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880208 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880193.165497:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:2.0:1788880193.165500:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880193.165502:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880193.165505:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:2.0:1788880193.165506:0:201284:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:2.0:1788880193.165509:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788880193.165514:0:201284:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.111@tcp failed (-19) 00000100:00000001:2.0:1788880193.165516:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:2.0:1788880193.165520:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780637184/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880208 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880193.165528:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780637184:192.168.204.111@tcp:38: 00000100:00000001:2.0:1788880193.165533:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880193.165535:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880193.165538:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780637184/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880208 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880193.165545:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880193.165547:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880193.165551:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000005e22417f. 02000000:00000001:2.0:1788880193.165554:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880193.165556:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880193.165558:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880193.165564:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880193.165566:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 000000006dbe0ab3. 02000000:00000001:2.0:1788880193.165570:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880193.165571:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880193.165573:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:2.0:1788880193.165576:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880193.165577:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880193.165579:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880193.165587:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880193.165588:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880193.165590:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880193.165592:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880193.214609:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880193.214623:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880193.214628:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788880194.238428:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880194.238431:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880194.238433:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788880195.262375:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880195.262379:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880195.262383:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000010:0.0:1788880195.849906:0:203393:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 0000000059963708. 00000100:00000001:0.0:1788880195.849933:0:203393:0:(service.c:2282:ptlrpc_server_handle_req_in()) Process entered 00000100:00100000:0.0:1788880195.849937:0:203393:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875776787994880 02000000:00000001:0.0:1788880195.849940:0:203393:0:(sec.c:2497:sptlrpc_svc_unwrap_request()) Process entered 00000100:00000001:0.0:1788880195.849943:0:203393:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880195.849945:0:203393:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880195.849950:0:203393:0:(sec.c:2546:sptlrpc_svc_unwrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880195.849964:0:203393:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875776787994880 00000020:00000010:3.0:1788880195.849967:0:203394:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 000000002063d487. 00000020:00000001:0.0:1788880195.849968:0:203393:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:0.0:1788880195.849971:0:203393:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000001:0.0:1788880195.849976:0:203393:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880195.849980:0:203393:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=3 00000020:00000001:0.0:1788880195.849983:0:203393:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000100:00000001:3.0:1788880195.849984:0:203394:0:(service.c:2282:ptlrpc_server_handle_req_in()) Process entered 00000020:00000001:0.0:1788880195.849987:0:203393:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000100:00100000:3.0:1788880195.849989:0:203394:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875776787995008 02000000:00000001:3.0:1788880195.849994:0:203394:0:(sec.c:2497:sptlrpc_svc_unwrap_request()) Process entered 00000100:00000001:3.0:1788880195.849996:0:203394:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880195.849998:0:203394:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880195.850003:0:203394:0:(sec.c:2546:sptlrpc_svc_unwrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880195.850007:0:203394:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875776787995008 00000100:00000001:0.0:1788880195.850009:0:203393:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000001:3.0:1788880195.850010:0:203394:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:3.0:1788880195.850015:0:203394:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000010:0.0:1788880195.850018:0:203393:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 00000000a46ef451. 00000020:00000001:3.0:1788880195.850020:0:203394:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880195.850026:0:203394:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=4 00000020:00000001:3.0:1788880195.850028:0:203394:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000020:00000001:3.0:1788880195.850031:0:203394:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000100:00000001:0.0:1788880195.850034:0:203393:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880195.850034:0:203394:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:3.0:1788880195.850040:0:203394:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 000000008a446bab. 00000020:00000010:0.0:1788880195.850041:0:203393:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 00000000a46ef451. 00000100:00000001:3.0:1788880195.850045:0:203394:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880195.850048:0:203394:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 000000008a446bab. 00000020:00000010:0.0:1788880195.850052:0:203393:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000605d7309. 00000020:00000010:0.0:1788880195.850058:0:203393:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 00000000573b9d66. 00000020:00000010:3.0:1788880195.850060:0:203394:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000004095512b. 00000020:00000010:3.0:1788880195.850065:0:203394:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 00000000da8a51e8. 00000020:00000010:0.0:1788880195.850072:0:203393:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 0000000017af7f3d. 00000020:00000010:3.0:1788880195.850074:0:203394:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 000000000bcc11c1. 00000100:00000040:3.0:1788880195.850085:0:203394:0:(service.c:1432:ptlrpc_at_set_timer()) armed ldlm_cbd at +6s 00000100:00000001:3.0:1788880195.850089:0:203394:0:(service.c:2003:ptlrpc_server_request_add()) Process entered 00000100:00000001:3.0:1788880195.850100:0:203394:0:(service.c:1919:ptlrpc_server_hpreq_init()) Process entered 00000100:00000001:0.0:1788880195.850102:0:203393:0:(service.c:2003:ptlrpc_server_request_add()) Process entered 00000100:00000001:3.0:1788880195.850102:0:203394:0:(service.c:1949:ptlrpc_server_hpreq_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880195.850110:0:203393:0:(service.c:1919:ptlrpc_server_hpreq_init()) Process entered 00000100:00000001:0.0:1788880195.850114:0:203393:0:(service.c:1949:ptlrpc_server_hpreq_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.850116:0:203394:0:(service.c:2101:ptlrpc_server_request_add()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.850122:0:203394:0:(service.c:2445:ptlrpc_server_handle_req_in()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880195.850123:0:203393:0:(service.c:2101:ptlrpc_server_request_add()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880195.850138:0:203393:0:(service.c:2445:ptlrpc_server_handle_req_in()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880195.850140:0:203394:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 00000100:00000001:3.0:1788880195.850144:0:203394:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 00000100:00100000:3.0:1788880195.850152:0:203394:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.111@tcp, seq: 0 00000100:00000001:0.0:1788880195.850155:0:203393:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 00000100:00000040:3.0:1788880195.850157:0:203394:0:(service.c:2255:ptlrpc_server_request_get()) RPC GETting export 00000000ea058c72 : new rpc_count 1 00000100:00000001:0.0:1788880195.850160:0:203393:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 00000100:00000001:3.0:1788880195.850163:0:203394:0:(service.c:2257:ptlrpc_server_request_get()) Process leaving (rc=18446617249938058624 : -126823771492992 : ffff8ca78a26ad80) 00000100:00000001:0.0:1788880195.850164:0:203393:0:(service.c:2244:ptlrpc_server_request_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880195.850169:0:203393:0:(service.c:2483:ptlrpc_server_handle_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880195.850174:0:203394:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Interpret req@000000006a9c2e02 x1875776787995008/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:571/0 lens 328/0 e 0 to 0 dl 1788880206 ref 1 fl New:/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880195.850187:0:203394:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:3.0:1788880195.850190:0:203394:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 000000008a446bab. 00000100:00000001:3.0:1788880195.850191:0:203394:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880195.850194:0:203394:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 000000008a446bab. 00000100:00100000:3.0:1788880195.850198:0:203394:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@000000006a9c2e02 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:fa4856d5-51a8-499e-b3f8-b477743a3dec+4:358209:x1875776787995008:12345-192.168.204.111@tcp:104: 00000100:00000200:3.0:1788880195.850201:0:203394:0:(service.c:2563:ptlrpc_server_handle_request()) got req 1875776787995008 00010000:00000001:3.0:1788880195.850203:0:203394:0:(ldlm_lockd.c:2360:ldlm_callback_handler()) Process entered 00010000:00000001:3.0:1788880195.850210:0:203394:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:3.0:1788880195.850212:0:203394:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880195.850213:0:203394:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000081b300c8 refcount=2 00000020:00000001:3.0:1788880195.850214:0:203394:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862489088 : -126823847062528 : ffff8ca785a59400) 00010000:00000001:3.0:1788880195.850216:0:203394:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862489088 : -126823847062528 : ffff8ca785a59400) 00010000:00000001:3.0:1788880195.850228:0:203394:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:3.0:1788880195.850274:0:203394:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880195.850279:0:203394:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7902 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:3.0:1788880195.850292:0:203394:0:(pack_generic.c:296:lustre_pack_reply_v2()) Process entered 02000000:00000001:3.0:1788880195.850295:0:203394:0:(sec.c:2568:sptlrpc_svc_alloc_rs()) Process entered 02000000:00000010:3.0:1788880195.850299:0:203394:0:(sec_null.c:305:null_alloc_rs()) kmalloced '(rs)': 520 at 000000004ef71132. 02000000:00000001:3.0:1788880195.850308:0:203394:0:(sec.c:2605:sptlrpc_svc_alloc_rs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.850311:0:203394:0:(pack_generic.c:331:lustre_pack_reply_v2()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.850317:0:203394:0:(ldlm_lib.c:3317:target_pack_pool_reply()) Process entered 00010000:00000001:3.0:1788880195.850320:0:203394:0:(ldlm_lib.c:3327:target_pack_pool_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880195.851569:0:203394:0:(import.c:2149:obd_at_measure()) add 1 to 0000000013afe31a time=1788880195 v=10 (0 0 0 0) 00000100:00001000:3.0:1788880195.851584:0:203394:0:(import.c:2215:obd_at_measure()) AT 0000000013afe31a change: old=10 new=5 delta=-5 (val=1) hist 1 0 0 0 00000100:00001000:3.0:1788880195.851603:0:203394:0:(niobuf.c:566:ptlrpc_at_set_reply()) @@@ svc ldlm_cbd changed estimate from 10 to 5 req@000000006a9c2e02 x1875776787995008/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:571/0 lens 328/224 e 0 to 0 dl 1788880206 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880195.851613:0:203394:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788880195.851615:0:203394:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 13 to 192.168.204.111@tcp 00000100:00000001:3.0:1788880195.851617:0:203394:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 02000000:00000001:3.0:1788880195.851620:0:203394:0:(sec.c:2619:sptlrpc_svc_wrap_reply()) Process entered 02000000:00000001:3.0:1788880195.851623:0:203394:0:(sec.c:2630:sptlrpc_svc_wrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.851625:0:203394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880195.851628:0:203394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880195.851633:0:203394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cf82a3fa. 00000100:00000200:3.0:1788880195.851638:0:203394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 16, xid 1875776787995008, offset 224 00000400:00000200:3.0:1788880195.851644:0:203394:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880195.851654:0:203394:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880195.851660:0:203394:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [46:46:256:4294967295] 192.168.204.111@tcp LPNI seq info [46:46:8:4294967295] 00000400:00000200:3.0:1788880195.851671:0:203394:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880195.851676:0:203394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880195.851680:0:203394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000800:00000200:3.0:1788880195.851686:0:203394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880195.851693:0:203394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880195.851697:0:203394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880195.851714:0:203394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880195.851717:0:203394:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00010000:00000001:3.0:1788880195.851721:0:203394:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:3.0:1788880195.851725:0:203394:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at 00000000448221d3. 00010000:00000001:3.0:1788880195.851736:0:203394:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:3.0:1788880195.851747:0:203394:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00010000:00000001:3.0:1788880195.851750:0:203394:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.851752:0:203394:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.851754:0:203394:0:(ldlm_lockd.c:2533:ldlm_callback_handler()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880195.851759:0:203394:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006a9c2e02 x1875776787995008/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:571/0 lens 328/224 e 0 to 0 dl 1788880206 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788880195.851768:0:203394:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@000000006a9c2e02 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:fa4856d5-51a8-499e-b3f8-b477743a3dec+4:358209:x1875776787995008:12345-192.168.204.111@tcp:104: Request processed in 1580us (1882us total) trans 0 rc 0/0 00000100:00100000:3.0:1788880195.851779:0:203394:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.111@tcp, seq: 0 00000100:00000040:3.0:1788880195.851782:0:203394:0:(service.c:1113:ptlrpc_server_finish_active_request()) RPC PUTting export 00000000ea058c72 : new rpc_count 0 00000100:00000001:3.0:1788880195.851785:0:203394:0:(service.c:1955:ptlrpc_server_hpreq_fini()) Process entered 00000100:00000001:3.0:1788880195.851787:0:203394:0:(service.c:1966:ptlrpc_server_hpreq_fini()) Process leaving 00000020:00000001:0.0:1788880195.851789:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000010:3.0:1788880195.851792:0:203394:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 00000000da8a51e8. 00000020:00000001:0.0:1788880195.851793:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880195.851798:0:203399:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000020:00000010:3.0:1788880195.851799:0:203394:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 000000000bcc11c1. 00010000:00000001:0.0:1788880195.851802:0:203399:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00000020:00000010:3.0:1788880195.851804:0:203394:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 000000004095512b. 00010000:00000001:0.0:1788880195.851806:0:203399:0:(ldlm_lockd.c:1905:ldlm_handle_bl_callback()) Process entered 00000020:00000040:3.0:1788880195.851810:0:203394:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 3 00010000:00010000:0.0:1788880195.851812:0:203399:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7902 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:3.0:1788880195.851814:0:203394:0:(service.c:2635:ptlrpc_server_handle_request()) Process leaving (rc=1 : 1 : 1) 00010000:00010000:0.0:1788880195.851824:0:203399:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock 0000000081b300c8 already unused, calling callback (00000000387296d6) 00000100:00000001:3.0:1788880195.851827:0:203394:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 10000000:00000001:0.0:1788880195.851828:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 00000100:00000001:3.0:1788880195.851830:0:203394:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 10000000:00010000:0.0:1788880195.851831:0:203399:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0x92f11317f60a7902 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788880195.851837:0:203394:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.111@tcp, seq: 1 00000100:00000040:3.0:1788880195.851843:0:203394:0:(service.c:2255:ptlrpc_server_request_get()) RPC GETting export 00000000ea058c72 : new rpc_count 1 00010000:00000001:0.0:1788880195.851845:0:203399:0:(ldlm_request.c:1770:ldlm_cli_cancel()) Process entered 00000100:00000001:3.0:1788880195.851846:0:203394:0:(service.c:2257:ptlrpc_server_request_get()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00010000:00000001:0.0:1788880195.851848:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880195.851851:0:203399:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000100:00000040:3.0:1788880195.851852:0:203394:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Interpret req@0000000077a1dd13 x1875776787994880/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:571/0 lens 328/0 e 0 to 0 dl 1788880206 ref 1 fl New:/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000020:00000040:0.0:1788880195.851854:0:203399:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 0000000081b300c8 refcount=3 00000020:00000001:0.0:1788880195.851859:0:203399:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862489088 : -126823847062528 : ffff8ca785a59400) 00010000:00000001:0.0:1788880195.851864:0:203399:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862489088 : -126823847062528 : ffff8ca785a59400) 00000100:00000001:3.0:1788880195.851867:0:203394:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00010000:00000001:0.0:1788880195.851868:0:203399:0:(ldlm_request.c:1386:ldlm_cli_cancel_local()) Process entered 00000020:00000010:3.0:1788880195.851871:0:203394:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 000000008a446bab. 00010000:00010000:0.0:1788880195.851872:0:203399:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0x92f11317f60a7902 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:3.0:1788880195.851876:0:203394:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880195.851882:0:203394:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 000000008a446bab. 10000000:00000001:0.0:1788880195.851889:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:0.0:1788880195.851893:0:203399:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0x92f11317f60a7902 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788880195.851898:0:203394:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@0000000077a1dd13 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358209:x1875776787994880:12345-192.168.204.111@tcp:104: 10000000:01000000:0.0:1788880195.851908:0:203399:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00000200:3.0:1788880195.851911:0:203394:0:(service.c:2563:ptlrpc_server_handle_request()) got req 1875776787994880 10000000:00000001:0.0:1788880195.851914:0:203399:0:(mgc_request.c:719:mgc_requeue_add()) Process entered 00010000:00000001:3.0:1788880195.851916:0:203394:0:(ldlm_lockd.c:2360:ldlm_callback_handler()) Process entered 00010000:00000001:3.0:1788880195.851923:0:203394:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 10000000:00000040:0.0:1788880195.851924:0:203399:0:(mgc_request.c:723:mgc_requeue_add()) log lustre-sptlrpc: requeue (r=2 sp=0 st=1) 00000020:00000001:3.0:1788880195.851926:0:203394:0:(lustre_handles.c:129:class_handle2object()) Process entered 10000000:00000040:0.0:1788880195.851928:0:203399:0:(mgc_request.c:726:mgc_requeue_add()) lockh 0x0 00000020:00000040:3.0:1788880195.851930:0:203394:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000000e3a19ac refcount=2 00010000:00000001:0.0:1788880195.851932:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:3.0:1788880195.851934:0:203394:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:0.0:1788880195.851935:0:203399:0:(ldlm_lock.c:617:__ldlm_handle2lock()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.851938:0:203394:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:3.0:1788880195.851943:0:203394:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:3.0:1788880195.851946:0:203394:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788880195.851949:0:203399:0:(mgc_request.c:742:mgc_requeue_add()) Process leaving 00010000:00010000:3.0:1788880195.851949:0:203394:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a8865f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7909 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00000001:0.0:1788880195.851953:0:203399:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:0.0:1788880195.851955:0:203399:0:(mgc_request.c:123:config_log_put()) log lustre-sptlrpc(000000007b8fd88a) refs 2 10000000:00000001:0.0:1788880195.851960:0:203399:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:0.0:1788880195.851961:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880195.851964:0:203399:0:(ldlm_lock.c:2502:ldlm_lock_cancel()) Process entered 00010000:00000001:0.0:1788880195.851980:0:203399:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:0.0:1788880195.851982:0:203399:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:0.0:1788880195.851983:0:203399:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:0.0:1788880195.851985:0:203399:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880195.851987:0:203399:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object 0000000081b300c8 with handle 0x805d548a36a88658 from hash 00010000:00000001:0.0:1788880195.851990:0:203399:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:0.0:1788880195.851992:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880195.851994:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880195.851996:0:203399:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:0.0:1788880195.851998:0:203399:0:(ldlm_lock.c:2538:ldlm_lock_cancel()) Process leaving 00010000:00000001:0.0:1788880195.852000:0:203399:0:(ldlm_request.c:1422:ldlm_cli_cancel_local()) Process leaving (rc=70368744177664 : 70368744177664 : 400000000000) 00010000:00000001:0.0:1788880195.852008:0:203399:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:0.0:1788880195.852010:0:203399:0:(ldlm_request.c:1527:ldlm_cli_cancel_req()) Process entered 00000100:00000010:0.0:1788880195.852034:0:203399:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000937cf312. 00000020:00000040:0.0:1788880195.852038:0:203399:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880195.852042:0:203399:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880195.852045:0:203399:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880195.852049:0:203399:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880195.852056:0:203399:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000a4a3549f. 00000100:00000001:3.0:1788880195.852061:0:203394:0:(pack_generic.c:296:lustre_pack_reply_v2()) Process entered 02000000:00000001:3.0:1788880195.852064:0:203394:0:(sec.c:2568:sptlrpc_svc_alloc_rs()) Process entered 02000000:00000010:3.0:1788880195.852068:0:203394:0:(sec_null.c:305:null_alloc_rs()) kmalloced '(rs)': 520 at 0000000023d951a9. 02000000:00000001:3.0:1788880195.852071:0:203394:0:(sec.c:2605:sptlrpc_svc_alloc_rs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.852073:0:203394:0:(pack_generic.c:331:lustre_pack_reply_v2()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880195.852074:0:203399:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.852078:0:203394:0:(ldlm_lib.c:3317:target_pack_pool_reply()) Process entered 00010000:00000001:0.0:1788880195.852079:0:203399:0:(ldlm_request.c:1461:ldlm_cancel_pack()) Process entered 00010000:00000001:3.0:1788880195.852082:0:203394:0:(ldlm_lib.c:3327:target_pack_pool_reply()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880195.852084:0:203399:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7902 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00001000:3.0:1788880195.852086:0:203394:0:(import.c:2149:obd_at_measure()) add 1 to 0000000013afe31a time=0 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880195.852094:0:203394:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00010000:00010000:0.0:1788880195.852097:0:203399:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00000040:3.0:1788880195.852099:0:203394:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 13 to 192.168.204.111@tcp 00010000:00000001:0.0:1788880195.852100:0:203399:0:(ldlm_request.c:1500:ldlm_cancel_pack()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880195.852104:0:203394:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000001:0.0:1788880195.852106:0:203399:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880195.852108:0:203394:0:(sec.c:2619:sptlrpc_svc_wrap_reply()) Process entered 02000000:00000001:3.0:1788880195.852109:0:203394:0:(sec.c:2630:sptlrpc_svc_wrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.852111:0:203394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880195.852115:0:203399:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880195.852115:0:203394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880195.852119:0:203394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000999bcd04. 00000100:00000200:3.0:1788880195.852123:0:203394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 16, xid 1875776787994880, offset 224 00000100:00000040:0.0:1788880195.852125:0:203399:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000937cf312] to pc [ptlrpcd_00_02+2] req@00000000937cf312 x1875776780637440/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880195.852131:0:203394:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880195.852140:0:203394:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880195.852144:0:203394:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [47:47:256:4294967295] 192.168.204.111@tcp LPNI seq info [47:47:8:4294967295] 00000400:00000200:3.0:1788880195.852161:0:203394:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880195.852165:0:203394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00010000:00000001:0.0:1788880195.852168:0:203399:0:(ldlm_request.c:1595:ldlm_cli_cancel_req()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:3.0:1788880195.852168:0:203394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00010000:00000001:0.0:1788880195.852173:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880195.852177:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000800:00000200:3.0:1788880195.852177:0:203394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00010000:00000001:0.0:1788880195.852179:0:203399:0:(ldlm_request.c:2504:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880195.852183:0:203399:0:(ldlm_request.c:1834:ldlm_cli_cancel()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880195.852184:0:203394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 10000000:00000001:0.0:1788880195.852186:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880195.852189:0:203394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00010000:00010000:0.0:1788880195.852190:0:203399:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7902 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880195.852199:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00000100:00000001:3.0:1788880195.852200:0:203394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880195.852201:0:203399:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.111@tcp lock: 0000000081b300c8/0x805d548a36a88658 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7902 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000040:3.0:1788880195.852207:0:203394:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00010000:00000001:3.0:1788880195.852212:0:203394:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1788880195.852218:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880195.852221:0:203399:0:(ldlm_lockd.c:1935:ldlm_handle_bl_callback()) Process leaving 00010000:00000010:0.0:1788880195.852222:0:203399:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at 00000000448221d3. 00010000:00000010:3.0:1788880195.852225:0:203394:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at 00000000432af089. 00010000:00000001:0.0:1788880195.852227:0:203399:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.852230:0:203394:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:3.0:1788880195.852260:0:203394:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00010000:00000001:3.0:1788880195.852263:0:203394:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.852265:0:203394:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.852266:0:203394:0:(ldlm_lockd.c:2533:ldlm_callback_handler()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880195.852270:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000100:00000040:3.0:1788880195.852271:0:203394:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776787994880/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:571/0 lens 328/224 e 0 to 0 dl 1788880206 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000020:00000001:0.0:1788880195.852273:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880195.852278:0:203399:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00000001:0.0:1788880195.852282:0:203399:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00010000:00000001:0.0:1788880195.852284:0:203399:0:(ldlm_lockd.c:1905:ldlm_handle_bl_callback()) Process entered 00010000:00010000:0.0:1788880195.852286:0:203399:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a8865f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7909 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788880195.852291:0:203394:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@0000000077a1dd13 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358209:x1875776787994880:12345-192.168.204.111@tcp:104: Request processed in 421us (2478us total) trans 0 rc 0/0 00010000:00010000:0.0:1788880195.852299:0:203399:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock 000000000e3a19ac already unused, calling callback (00000000387296d6) 10000000:00000001:0.0:1788880195.852304:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 00000100:00100000:3.0:1788880195.852304:0:203394:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.111@tcp, seq: 1 10000000:00010000:0.0:1788880195.852307:0:203399:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a8865f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x92f11317f60a7909 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000040:3.0:1788880195.852309:0:203394:0:(service.c:1113:ptlrpc_server_finish_active_request()) RPC PUTting export 00000000ea058c72 : new rpc_count 0 00000100:00000001:3.0:1788880195.852313:0:203394:0:(service.c:1955:ptlrpc_server_hpreq_fini()) Process entered 00000100:00000001:3.0:1788880195.852315:0:203394:0:(service.c:1966:ptlrpc_server_hpreq_fini()) Process leaving 00010000:00000001:0.0:1788880195.852318:0:203399:0:(ldlm_request.c:1770:ldlm_cli_cancel()) Process entered 00000020:00000010:3.0:1788880195.852320:0:203394:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 00000000573b9d66. 00000020:00000010:3.0:1788880195.852324:0:203394:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 0000000017af7f3d. 00010000:00000001:0.0:1788880195.852326:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000010:3.0:1788880195.852328:0:203394:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000605d7309. 00000020:00000001:0.0:1788880195.852329:0:203399:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880195.852333:0:203399:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000000e3a19ac refcount=3 00000020:00000040:3.0:1788880195.852334:0:203394:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 2 00000020:00000001:0.0:1788880195.852337:0:203399:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00000100:00000001:3.0:1788880195.852338:0:203394:0:(service.c:2635:ptlrpc_server_handle_request()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880195.852342:0:203399:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:0.0:1788880195.852345:0:203399:0:(ldlm_request.c:1386:ldlm_cli_cancel_local()) Process entered 00010000:00010000:0.0:1788880195.852347:0:203399:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a8865f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x92f11317f60a7909 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00000001:0.0:1788880195.852352:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:0.0:1788880195.852354:0:203399:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a8865f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x92f11317f60a7909 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788880195.852359:0:203399:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:00000001:0.0:1788880195.852362:0:203399:0:(mgc_request.c:719:mgc_requeue_add()) Process entered 10000000:00000040:0.0:1788880195.852363:0:203399:0:(mgc_request.c:723:mgc_requeue_add()) log lustre-client: requeue (r=2 sp=0 st=1) 10000000:00000040:0.0:1788880195.852365:0:203399:0:(mgc_request.c:726:mgc_requeue_add()) lockh 0x0 00000020:00000001:3.0:1788880195.852365:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00010000:00000001:0.0:1788880195.852369:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:3.0:1788880195.852370:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880195.852371:0:203399:0:(ldlm_lock.c:617:__ldlm_handle2lock()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788880195.852377:0:203399:0:(mgc_request.c:742:mgc_requeue_add()) Process leaving 10000000:00000001:0.0:1788880195.852379:0:203399:0:(mgc_request.c:117:config_log_put()) Process entered 00010000:00010000:3.0:1788880195.852380:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00000040:0.0:1788880195.852381:0:203399:0:(mgc_request.c:123:config_log_put()) log lustre-client(00000000c2974351) refs 2 10000000:00000001:0.0:1788880195.852385:0:203399:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:0.0:1788880195.852387:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880195.852389:0:203399:0:(ldlm_lock.c:2502:ldlm_lock_cancel()) Process entered 00010000:00000001:0.0:1788880195.852391:0:203399:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:0.0:1788880195.852392:0:203399:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:0.0:1788880195.852394:0:203399:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:0.0:1788880195.852396:0:203399:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880195.852398:0:203399:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object 000000000e3a19ac with handle 0x805d548a36a8865f from hash 00010000:00000001:0.0:1788880195.852400:0:203399:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:0.0:1788880195.852401:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880195.852402:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880195.852403:0:203399:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:0.0:1788880195.852404:0:203399:0:(ldlm_lock.c:2538:ldlm_lock_cancel()) Process leaving 00010000:00000001:0.0:1788880195.852406:0:203399:0:(ldlm_request.c:1422:ldlm_cli_cancel_local()) Process leaving (rc=70368744177664 : 70368744177664 : 400000000000) 00010000:00000001:0.0:1788880195.852414:0:203399:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:0.0:1788880195.852415:0:203399:0:(ldlm_request.c:1527:ldlm_cli_cancel_req()) Process entered 00000100:00000010:0.0:1788880195.852418:0:203399:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000451fe31a. 00000020:00000040:0.0:1788880195.852420:0:203399:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880195.852423:0:203399:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880195.852424:0:203399:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880195.852426:0:203399:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880195.852440:0:203399:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000007c59c7b9. 00000100:00000001:0.0:1788880195.852446:0:203399:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880195.852449:0:203399:0:(ldlm_request.c:1461:ldlm_cancel_pack()) Process entered 00010000:00010000:0.0:1788880195.852451:0:203399:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a8865f lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7909 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788880195.852456:0:203399:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00000001:0.0:1788880195.852457:0:203399:0:(ldlm_request.c:1500:ldlm_cancel_pack()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880195.852460:0:203399:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880195.852461:0:203399:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880195.852465:0:203399:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000451fe31a] to pc [ptlrpcd_00_03+3] req@00000000451fe31a x1875776780637568/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880195.852474:0:203399:0:(ldlm_request.c:1595:ldlm_cli_cancel_req()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:0.0:1788880195.852476:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788880195.852477:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880195.852478:0:203399:0:(ldlm_request.c:2504:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880195.852480:0:203399:0:(ldlm_request.c:1834:ldlm_cli_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788880195.852482:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880195.852483:0:203399:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a8865f lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7909 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788880195.852488:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00010000:0.0:1788880195.852489:0:203399:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a8865f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7909 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000040:0.0:1788880195.852495:0:203399:0:(ldlm_resource.c:1704:ldlm_resource_putref()) putref res: 000000005bab1010 count: 0 00010000:00000001:0.0:1788880195.852503:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788880195.852509:0:203399:0:(ldlm_lockd.c:1935:ldlm_handle_bl_callback()) Process leaving 00010000:00000010:0.0:1788880195.852510:0:203399:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at 00000000432af089. 00010000:00000001:0.0:1788880195.852513:0:203399:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880195.852514:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880195.852516:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880195.852517:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788880195.852518:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880195.852519:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880195.852521:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788880195.852522:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880195.852523:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880195.852525:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:2.0:1788880195.852606:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880195.852622:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880195.852627:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880195.852642:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000937cf312 x1875776780637440/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880195.852675:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880195.852677:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880195.852681:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880195.852685:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880195.852688:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880195.852695:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780637440:192.168.204.111@tcp:103: 00000100:00000001:2.0:1788880195.852707:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880195.852715:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880195.852718:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880195.852720:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880195.852733:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000790fa4ee. 02000000:00000001:2.0:1788880195.852742:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880195.852749:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000038fe0168. 00000400:00000010:2.0:1788880195.852758:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000841b943f. 00000100:00000200:2.0:1788880195.852770:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780637440, portal 18 00000100:00000001:2.0:1788880195.852779:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880195.852784:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849367680 : -126822860183936 : ffff8ca7c0782680) 00000100:00000040:2.0:1788880195.852794:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000937cf312 x1875776780637440/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880195.853993:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880195.853996:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880195.854000:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001e963647. 00000100:00000200:2.0:1788880195.854004:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 17, xid 1875776780637440, offset 0 00000400:00000200:2.0:1788880195.854009:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880195.854014:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880195.854018:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880195.854020:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880195.854022:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880195.854025:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880195.854027:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [49:49:256:4294967295] 192.168.204.111@tcp LPNI seq info [49:49:8:4294967295] 00000400:00000200:2.0:1788880195.854035:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880195.854039:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880195.854042:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003139f6ba. 00000800:00000200:2.0:1788880195.854046:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880195.854051:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880195.854054:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003139f6ba type 1, nob 424 niov 1 nkiov 1 00000100:00000001:2.0:1788880195.854065:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880195.854068:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305832030 00000100:00000001:2.0:1788880195.854072:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305832029 00000100:00000001:2.0:1788880195.854076:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305832030 00000100:00000001:2.0:1788880195.854078:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305832029 00000100:00000001:2.0:1788880195.854079:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305832030 00000100:00000001:2.0:1788880195.854082:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305832029 00000100:00000001:2.0:1788880195.854084:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305832030 00000100:00000001:2.0:1788880195.854087:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305832029 00000100:00000001:2.0:1788880195.854089:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880195.854091:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880195.854100:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880195.854103:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880195.854108:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880195.854109:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880195.854111:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880195.854113:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880195.854114:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880195.854116:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.856835:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880195.856839:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880195.856847:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880195.856849:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880195.856852:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004b6baab5. 00000400:00000200:3.0:1788880195.856859:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000974f52f4 00000400:00000010:3.0:1788880195.856863:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000974f52f4. 00000100:00000001:3.0:1788880195.856868:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880195.856874:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000451fe31a x1875776780637568/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880195.856881:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000451fe31a x1875776780637568/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880195.856886:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880195.856888:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880195.856892:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880195.856896:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880195.856899:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880195.856901:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880195.856903:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880195.856909:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000081111466 time=1788880195 v=5 (0 0 0 0) 00000100:00001000:3.0:1788880195.856915:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=23 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880195.856921:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880195.856923:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.856925:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880195.856926:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.856931:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880195.856935:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000451fe31a x1875776780637568/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880195.856941:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880195.856945:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880195.856946:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880195.856949:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000451fe31a x1875776780637568/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880195.856958:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780637568:192.168.204.111@tcp:103: 00000100:00000001:3.0:1788880195.856969:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880195.856972:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880195.856978:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000451fe31a x1875776780637568/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880195.856984:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880195.856986:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880195.856990:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009e1ad299. 02000000:00000001:3.0:1788880195.856993:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880195.856995:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880195.856998:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:3.0:1788880195.856998:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880195.857002:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880195.857006:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000007c59c7b9. 00000100:00000001:2.0:1788880195.857008:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880195.857011:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880195.857013:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880195.857015:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000451fe31a. 00000400:00000001:2.0:1788880195.857020:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880195.857020:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880195.857024:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880195.857027:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880195.857028:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880195.857035:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000038fe0168. 00000100:00000001:3.0:1788880195.857038:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880195.857040:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.857044:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880195.857046:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000841b943f 00000100:00000001:3.0:1788880195.857046:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880195.857055:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000841b943f. 00000100:00000001:2.0:1788880195.857062:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880195.857071:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000937cf312 x1875776780637440/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880195.857081:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000937cf312 x1875776780637440/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880195.857088:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880195.857103:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880195.857106:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880195.857108:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880195.857110:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880195.857111:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880195.857114:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880195.857119:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000081111466 time=0 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880195.857123:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=23 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880195.857129:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880195.857130:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880195.857132:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880195.857134:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880195.857137:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880195.857141:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000937cf312 x1875776780637440/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880195.857148:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880195.857175:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880195.857177:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880195.857181:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000937cf312 x1875776780637440/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880195.857188:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780637440:192.168.204.111@tcp:103: 00000100:00000001:2.0:1788880195.857192:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880195.857195:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880195.857197:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000937cf312 x1875776780637440/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880195.857204:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880195.857206:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880195.857208:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000790fa4ee. 02000000:00000001:2.0:1788880195.857212:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880195.857213:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880195.857215:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880195.857217:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880195.857219:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000a4a3549f. 02000000:00000001:2.0:1788880195.857222:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880195.857224:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880195.857225:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000937cf312. 00000100:00000001:2.0:1788880195.857229:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880195.857230:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880195.857232:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880195.857266:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880195.857268:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880195.857270:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880195.857273:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000010:0.1:1788880195.857277:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 0000000081b300c8. 00010000:00000010:0.1:1788880195.861279:0:0:0:(ldlm_resource.c:1555:__ldlm_resource_free()) slab-freed '(res)': 256 at 000000005bab1010. 00010000:00000010:0.1:1788880195.861292:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 000000000e3a19ac. 00000100:00000001:0.0:1788880195.959533:0:203393:0:(service.c:2282:ptlrpc_server_handle_req_in()) Process entered 00000100:00100000:0.0:1788880195.959538:0:203393:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875776788000384 02000000:00000001:0.0:1788880195.959541:0:203393:0:(sec.c:2497:sptlrpc_svc_unwrap_request()) Process entered 00000100:00000001:0.0:1788880195.959543:0:203393:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880195.959545:0:203393:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880195.959548:0:203393:0:(sec.c:2546:sptlrpc_svc_unwrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880195.959551:0:203393:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875776788000384 00000020:00000001:0.0:1788880195.959553:0:203393:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:0.0:1788880195.959554:0:203393:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000001:0.0:1788880195.959556:0:203393:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880195.959559:0:203393:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=3 00000020:00000001:0.0:1788880195.959562:0:203393:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000020:00000001:0.0:1788880195.959563:0:203393:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000100:00000001:0.0:1788880195.959567:0:203393:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:0.0:1788880195.959573:0:203393:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 00000000a46ef451. 00000100:00000001:0.0:1788880195.959575:0:203393:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:0.0:1788880195.959576:0:203393:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 00000000a46ef451. 00000020:00000010:0.0:1788880195.959584:0:203393:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000a4a3549f. 00000020:00000010:0.0:1788880195.959600:0:203393:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 00000000b9cf25e6. 00000020:00000010:0.0:1788880195.959604:0:203393:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 00000000d01614d5. 00000100:00000040:0.0:1788880195.959611:0:203393:0:(service.c:1432:ptlrpc_at_set_timer()) armed ldlm_cbd at +6s 00000100:00000001:0.0:1788880195.959614:0:203393:0:(service.c:2003:ptlrpc_server_request_add()) Process entered 00000100:00000001:0.0:1788880195.959615:0:203393:0:(service.c:1919:ptlrpc_server_hpreq_init()) Process entered 00000100:00000001:0.0:1788880195.959627:0:203393:0:(service.c:1949:ptlrpc_server_hpreq_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880195.959634:0:203393:0:(service.c:2101:ptlrpc_server_request_add()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880195.959648:0:203393:0:(service.c:2445:ptlrpc_server_handle_req_in()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880195.959655:0:203393:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 00000100:00000001:0.0:1788880195.959657:0:203393:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 00000100:00100000:0.0:1788880195.959662:0:203393:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.111@tcp, seq: 2 00000100:00000040:0.0:1788880195.959665:0:203393:0:(service.c:2255:ptlrpc_server_request_get()) RPC GETting export 00000000ea058c72 : new rpc_count 1 00000100:00000001:0.0:1788880195.959666:0:203393:0:(service.c:2257:ptlrpc_server_request_get()) Process leaving (rc=18446617249938052352 : -126823771499264 : ffff8ca78a269500) 00000100:00000040:0.0:1788880195.959670:0:203393:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Interpret req@00000000dc99d6ba x1875776788000384/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:571/0 lens 328/0 e 0 to 0 dl 1788880206 ref 1 fl New:/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880195.959676:0:203393:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:0.0:1788880195.959678:0:203393:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 00000000a46ef451. 00000100:00000001:0.0:1788880195.959680:0:203393:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:0.0:1788880195.959681:0:203393:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 00000000a46ef451. 00000100:00100000:0.0:1788880195.959684:0:203393:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@00000000dc99d6ba pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358231:x1875776788000384:12345-192.168.204.111@tcp:104: 00000100:00000200:0.0:1788880195.959686:0:203393:0:(service.c:2563:ptlrpc_server_handle_request()) got req 1875776788000384 00010000:00000001:0.0:1788880195.959688:0:203393:0:(ldlm_lockd.c:2360:ldlm_callback_handler()) Process entered 00010000:00000001:0.0:1788880195.959694:0:203393:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880195.959695:0:203393:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880195.959697:0:203393:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000001ffdd456 refcount=2 00000020:00000001:0.0:1788880195.959699:0:203393:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000001:0.0:1788880195.959700:0:203393:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000001:0.0:1788880195.959703:0:203393:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:0.0:1788880195.959705:0:203393:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880195.959707:0:203393:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:0.0:1788880195.959715:0:203393:0:(pack_generic.c:296:lustre_pack_reply_v2()) Process entered 02000000:00000001:0.0:1788880195.959717:0:203393:0:(sec.c:2568:sptlrpc_svc_alloc_rs()) Process entered 02000000:00000010:0.0:1788880195.959720:0:203393:0:(sec_null.c:305:null_alloc_rs()) kmalloced '(rs)': 520 at 000000008e52983e. 02000000:00000001:0.0:1788880195.959722:0:203393:0:(sec.c:2605:sptlrpc_svc_alloc_rs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880195.959727:0:203393:0:(pack_generic.c:331:lustre_pack_reply_v2()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880195.959731:0:203393:0:(ldlm_lib.c:3317:target_pack_pool_reply()) Process entered 00010000:00000001:0.0:1788880195.959733:0:203393:0:(ldlm_lib.c:3327:target_pack_pool_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880195.959735:0:203393:0:(import.c:2149:obd_at_measure()) add 1 to 0000000013afe31a time=0 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880195.959739:0:203393:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:0.0:1788880195.959741:0:203393:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 13 to 192.168.204.111@tcp 00000100:00000001:0.0:1788880195.959742:0:203393:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 02000000:00000001:0.0:1788880195.959744:0:203393:0:(sec.c:2619:sptlrpc_svc_wrap_reply()) Process entered 02000000:00000001:0.0:1788880195.959745:0:203393:0:(sec.c:2630:sptlrpc_svc_wrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880195.959747:0:203393:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880195.959749:0:203393:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880195.959752:0:203393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 00000100:00000200:0.0:1788880195.959756:0:203393:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 16, xid 1875776788000384, offset 224 00000400:00000200:0.0:1788880195.959760:0:203393:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880195.959770:0:203393:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880195.959775:0:203393:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [50:50:256:4294967295] 192.168.204.111@tcp LPNI seq info [50:50:8:4294967295] 00000400:00000200:0.0:1788880195.959783:0:203393:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880195.959786:0:203393:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880195.959790:0:203393:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000800:00000200:0.0:1788880195.959794:0:203393:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880195.959799:0:203393:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880195.959802:0:203393:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880195.959814:0:203393:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880195.959816:0:203393:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00010000:00000001:0.0:1788880195.959819:0:203393:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:0.0:1788880195.959822:0:203393:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at 00000000d9225373. 00010000:00000001:0.0:1788880195.959825:0:203393:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:0.0:1788880195.959834:0:203393:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00010000:00000001:0.0:1788880195.959837:0:203393:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880195.959839:0:203393:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880195.959840:0:203393:0:(ldlm_lockd.c:2533:ldlm_callback_handler()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880195.959845:0:203393:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000dc99d6ba x1875776788000384/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:571/0 lens 328/224 e 0 to 0 dl 1788880206 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788880195.959859:0:203393:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@00000000dc99d6ba pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358231:x1875776788000384:12345-192.168.204.111@tcp:104: Request processed in 180us (403us total) trans 0 rc 0/0 00000100:00100000:0.0:1788880195.959867:0:203393:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.111@tcp, seq: 2 00000100:00000040:0.0:1788880195.959870:0:203393:0:(service.c:1113:ptlrpc_server_finish_active_request()) RPC PUTting export 00000000ea058c72 : new rpc_count 0 00000020:00000001:3.0:1788880195.959870:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000100:00000001:0.0:1788880195.959875:0:203393:0:(service.c:1955:ptlrpc_server_hpreq_fini()) Process entered 00000020:00000001:3.0:1788880195.959875:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880195.959877:0:203393:0:(service.c:1966:ptlrpc_server_hpreq_fini()) Process leaving 00010000:00010000:3.0:1788880195.959879:0:203398:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000020:00000010:0.0:1788880195.959882:0:203393:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 00000000b9cf25e6. 00010000:00000001:3.0:1788880195.959883:0:203398:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00000020:00000010:0.0:1788880195.959887:0:203393:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 00000000d01614d5. 00010000:00000001:3.0:1788880195.959887:0:203398:0:(ldlm_lockd.c:1905:ldlm_handle_bl_callback()) Process entered 00010000:00010000:3.0:1788880195.959889:0:203398:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00000010:0.0:1788880195.959892:0:203393:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000a4a3549f. 00000020:00000040:0.0:1788880195.959898:0:203393:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 2 00010000:00010000:3.0:1788880195.959901:0:203398:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock 000000001ffdd456 already unused, calling callback (00000000387296d6) 00000100:00000001:0.0:1788880195.959902:0:203393:0:(service.c:2635:ptlrpc_server_handle_request()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:3.0:1788880195.959906:0:203398:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:3.0:1788880195.959909:0:203398:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:3.0:1788880195.959933:0:203398:0:(ldlm_request.c:1770:ldlm_cli_cancel()) Process entered 00010000:00000001:3.0:1788880195.959936:0:203398:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:3.0:1788880195.959939:0:203398:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880195.959944:0:203398:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000001ffdd456 refcount=3 00000020:00000001:3.0:1788880195.959948:0:203398:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000001:3.0:1788880195.959953:0:203398:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250661946368 : -126823047605248 : ffff8ca7b54c5400) 00010000:00000001:3.0:1788880195.959956:0:203398:0:(ldlm_request.c:1386:ldlm_cli_cancel_local()) Process entered 00010000:00010000:3.0:1788880195.959958:0:203398:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00000001:3.0:1788880195.959970:0:203398:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:3.0:1788880195.959971:0:203398:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788880195.959977:0:203398:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 10000000:00000001:3.0:1788880195.959981:0:203398:0:(mgc_request.c:719:mgc_requeue_add()) Process entered 10000000:00000040:3.0:1788880195.959982:0:203398:0:(mgc_request.c:723:mgc_requeue_add()) log lustre-cliir: requeue (r=2 sp=0 st=3) 10000000:00000040:3.0:1788880195.959984:0:203398:0:(mgc_request.c:726:mgc_requeue_add()) lockh 0x0 00010000:00000001:3.0:1788880195.959985:0:203398:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00010000:00000001:3.0:1788880195.959987:0:203398:0:(ldlm_lock.c:617:__ldlm_handle2lock()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:3.0:1788880195.959996:0:203398:0:(mgc_request.c:742:mgc_requeue_add()) Process leaving 10000000:00000001:3.0:1788880195.959998:0:203398:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:3.0:1788880195.959999:0:203398:0:(mgc_request.c:123:config_log_put()) log lustre-cliir(0000000008e74578) refs 2 10000000:00000001:3.0:1788880195.960001:0:203398:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:3.0:1788880195.960002:0:203398:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.960004:0:203398:0:(ldlm_lock.c:2502:ldlm_lock_cancel()) Process entered 00010000:00000001:3.0:1788880195.960007:0:203398:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:3.0:1788880195.960009:0:203398:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:3.0:1788880195.960011:0:203398:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:3.0:1788880195.960066:0:203398:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880195.960069:0:203398:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object 000000001ffdd456 with handle 0x805d548a36a88689 from hash 00010000:00000001:3.0:1788880195.960071:0:203398:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:3.0:1788880195.960073:0:203398:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:3.0:1788880195.960085:0:203398:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:3.0:1788880195.960086:0:203398:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:3.0:1788880195.960088:0:203398:0:(ldlm_lock.c:2538:ldlm_lock_cancel()) Process leaving 00010000:00000001:3.0:1788880195.960090:0:203398:0:(ldlm_request.c:1422:ldlm_cli_cancel_local()) Process leaving (rc=70368744177664 : 70368744177664 : 400000000000) 00010000:00000001:3.0:1788880195.960093:0:203398:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:3.0:1788880195.960095:0:203398:0:(ldlm_request.c:1527:ldlm_cli_cancel_req()) Process entered 00000100:00000010:3.0:1788880195.960101:0:203398:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788880195.960104:0:203398:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880195.960108:0:203398:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880195.960110:0:203398:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880195.960114:0:203398:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880195.960118:0:203398:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000004095512b. 00000100:00000001:3.0:1788880195.960123:0:203398:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.960125:0:203398:0:(ldlm_request.c:1461:ldlm_cancel_pack()) Process entered 00010000:00010000:3.0:1788880195.960128:0:203398:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788880195.960134:0:203398:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00000001:3.0:1788880195.960135:0:203398:0:(ldlm_request.c:1500:ldlm_cancel_pack()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880195.960138:0:203398:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880195.960140:0:203398:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880195.960144:0:203398:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_00+0] req@00000000aa33757e x1875776780637696/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880195.960174:0:203398:0:(ldlm_request.c:1595:ldlm_cli_cancel_req()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:3.0:1788880195.960178:0:203398:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:3.0:1788880195.960179:0:203398:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:3.0:1788880195.960180:0:203398:0:(ldlm_request.c:2504:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.960182:0:203398:0:(ldlm_request.c:1834:ldlm_cli_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:3.0:1788880195.960184:0:203398:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880195.960186:0:203398:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:3.0:1788880195.960192:0:203398:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00010000:3.0:1788880195.960193:0:203398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.111@tcp lock: 000000001ffdd456/0x805d548a36a88689 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7910 expref: -99 pid: 203392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000040:3.0:1788880195.960208:0:203398:0:(ldlm_resource.c:1704:ldlm_resource_putref()) putref res: 00000000aee22452 count: 0 00010000:00000001:3.0:1788880195.960219:0:203398:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:3.0:1788880195.960221:0:203398:0:(ldlm_lockd.c:1935:ldlm_handle_bl_callback()) Process leaving 00010000:00000010:3.0:1788880195.960224:0:203398:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at 00000000d9225373. 00010000:00000001:3.0:1788880195.960228:0:203398:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880195.960282:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880195.960286:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880195.960290:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880195.960293:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880195.960297:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880195.960300:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880195.960303:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880195.960306:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880195.960308:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:0.0:1788880195.960364:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880195.960370:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880195.960446:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880195.960449:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:3.0:1788880195.960452:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880195.960456:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880195.960459:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000038c5ac58 00000400:00000010:3.0:1788880195.960461:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000038c5ac58. 00000100:00000001:3.0:1788880195.960465:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880195.960468:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780637696/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880195.960477:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880195.960479:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780637696/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880195.960526:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.960528:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880195.963408:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880195.963411:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880195.963416:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880195.963417:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880195.963419:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001ee767d5. 00000400:00000200:3.0:1788880195.963427:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000061cfcb81 00000400:00000010:3.0:1788880195.963428:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000061cfcb81. 00000100:00000001:3.0:1788880195.963430:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880195.963433:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780637696/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880195.963439:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780637696/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880195.963443:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880195.963443:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880195.963445:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880195.963447:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880195.963448:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880195.963449:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880195.963450:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880195.963454:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000081111466 time=0 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880195.963456:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=23 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880195.963458:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880195.963459:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880195.963460:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880195.963461:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.963463:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880195.963466:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780637696/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880195.963469:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880195.963471:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880195.963472:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880195.963474:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780637696/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880195.963478:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780637696:192.168.204.111@tcp:103: 00000100:00000001:3.0:1788880195.963481:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880195.963482:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880195.963483:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780637696/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880211 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880195.963493:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880195.963495:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880195.963497:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008d264e32. 02000000:00000001:3.0:1788880195.963499:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880195.963501:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880195.963502:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880195.963504:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880195.963506:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000004095512b. 02000000:00000001:3.0:1788880195.963509:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880195.963511:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880195.963513:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:3.0:1788880195.963516:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880195.963517:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880195.963519:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880195.963524:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880195.963526:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880195.963528:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880195.963529:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000010:3.1:1788880195.965279:0:0:0:(ldlm_resource.c:1555:__ldlm_resource_free()) slab-freed '(res)': 256 at 00000000aee22452. 00010000:00000010:3.1:1788880195.965290:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 000000001ffdd456. 00000020:00000001:0.0:1788880196.286461:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880196.286468:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880196.286472:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880197.310513:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880197.310522:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880197.310526:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880198.270385:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880198.270394:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880198.270404:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 00000020:00000040:2.0:1788880198.270411:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880198.270415:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880198.270418:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880198.270422:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880198.270444:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006aca3ff7. 00000100:00000001:2.0:1788880198.270451:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880198.270462:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000175072aa x1875776780637824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880198.270470:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880198.270471:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880198.270476:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_01+1] req@00000000175072aa x1875776780637824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880198.270503:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880198.270506:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880198.270509:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880198.270512:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880198.270515:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880198.270517:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880198.270519:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880198.270520:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880198.270523:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000100:00000001:3.0:1788880198.270523:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880198.270526:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880198.270526:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880198.270529:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880198.270530:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000025994245 x1875776780637952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:3.0:1788880198.270534:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780637824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880198.270537:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880198.270539:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880198.270552:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_02+2] req@0000000025994245 x1875776780637952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880198.270553:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880198.270555:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880198.270559:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880198.270562:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880198.270565:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880198.270568:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780637824:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880198.270572:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880198.270577:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880198.270578:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880198.270583:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880198.270588:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:2.0:1788880198.270589:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880198.270592:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:2.0:1788880198.270598:0:200894:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 02000000:00000001:3.0:1788880198.270598:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880198.270600:0:200894:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:2.0:1788880198.270603:0:200894:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 02000000:00000010:3.0:1788880198.270603:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000358be29b. 02000000:00000001:3.0:1788880198.270605:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.270607:0:200894:0:(import.c:485:import_select_connection()) Process entered 00000400:00000010:3.0:1788880198.270611:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000cbf1a475. 00000100:00080000:2.0:1788880198.270615:0:200894:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 11156 00000400:00000010:3.0:1788880198.270617:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e78d7264. 00000400:00000200:2.0:1788880198.270622:0:200894:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000100:00000200:3.0:1788880198.270622:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780637824, portal 25 00000100:00000001:3.0:1788880198.270627:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00001000:2.0:1788880198.270629:0:200894:0:(import.c:2149:obd_at_measure()) add 15 to 0000000040f48a4f time=26 v=10 (10 0 0 0) 00000100:00000001:3.0:1788880198.270629:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938435456 : -126823771116160 : ffff8ca78a2c6d80) 00000100:00001000:2.0:1788880198.270633:0:200894:0:(import.c:2215:obd_at_measure()) AT 0000000040f48a4f change: old=10 new=15 delta=5 (val=15) hist 15 0 0 0 00000100:00000040:3.0:1788880198.270636:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000175072aa x1875776780637824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880214 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788880198.270637:0:200894:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: tried all connections, increasing latency to 10s 00000100:00000001:0.0:1788880198.270642:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880198.270643:0:200894:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:3.0:1788880198.270644:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880198.270653:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.270654:0:200894:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788880198.270659:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880198.270661:0:200894:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000400:00000010:3.0:1788880198.270664:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a165c7f9. 00000100:00000001:2.0:1788880198.270667:0:200894:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000200:3.0:1788880198.270669:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780637824, offset 0 00000020:00000001:2.0:1788880198.270671:0:200894:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880198.270674:0:200894:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:2.0:1788880198.270678:0:200894:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000400:00000200:3.0:1788880198.270682:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000020:00000040:2.0:1788880198.270683:0:200894:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 00000020:00000001:2.0:1788880198.270689:0:200894:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000020:00000001:2.0:1788880198.270691:0:200894:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000400:00000200:3.0:1788880198.270692:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880198.270698:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880198.270705:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:2.0:1788880198.270706:0:200894:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000400:00000200:3.0:1788880198.270711:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880198.270712:0:200894:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880198.270716:0:200894:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000400:00000200:3.0:1788880198.270717:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880198.270719:0:200894:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000040:2.0:1788880198.270722:0:200894:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 00000400:00000200:3.0:1788880198.270723:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [52:52:256:4294967295] 192.168.204.111@tcp LPNI seq info [52:52:8:4294967295] 00000100:00080000:2.0:1788880198.270727:0:200894:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: import 00000000b98b8a4e using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000100:00000001:2.0:1788880198.270731:0:200894:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880198.270734:0:200894:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 00000400:00000200:3.0:1788880198.270736:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788880198.270739:0:200894:0:(sec.c:1628:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880198.270742:0:200894:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880198.270745:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880198.270746:0:200894:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:2.0:1788880198.270750:0:200894:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 2097152, lost: 0. 00000800:00000010:3.0:1788880198.270750:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000008:00000001:2.0:1788880198.270753:0:200894:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.270756:0:200894:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880198.270757:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:2.0:1788880198.270760:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000020:00000040:2.0:1788880198.270762:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000800:00000200:3.0:1788880198.270768:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:2.0:1788880198.270772:0:200894:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 00000800:00000200:3.0:1788880198.270772:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880198.270775:0:200894:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.270780:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880198.270782:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880198.270787:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880198.270790:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880198.270793:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305834447 02000000:00000010:2.0:1788880198.270795:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 000000005e22417f. 00000100:00000001:2.0:1788880198.270804:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880198.270804:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305832137 00000100:00100000:2.0:1788880198.270813:0:200894:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880198.270820:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305834447 00000100:00000001:2.0:1788880198.270825:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880198.270825:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305832137 00000100:00000001:2.0:1788880198.270827:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880198.270829:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305834447 00000100:00000040:2.0:1788880198.270834:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fd6b7ee1] to pc [ptlrpcd_rcv+-1] req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880198.270834:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305832137 00000100:00000001:3.0:1788880198.270837:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305834447 00000100:00000001:3.0:1788880198.270842:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305832137 00000100:00000001:3.0:1788880198.270847:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880198.270849:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880198.270853:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.270855:0:200894:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880198.270858:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880198.270858:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.270865:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880198.270867:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880198.270869:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000010:2.0:1788880198.270870:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000020:00000040:2.0:1788880198.270874:0:200894:0:(genops.c:1231:class_import_get()) import 00000000beb50720 refcount=3 obd=lustre-OST0000-osc-ffff8ca787527000 00000100:00000001:3.0:1788880198.270874:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880198.270878:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880198.270878:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880198.270880:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880198.270880:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880198.270884:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880198.270884:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880198.270889:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 00000100:00000001:2.0:1788880198.270902:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880198.270912:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880198.270912:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880198.270916:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880198.270918:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880198.270923:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880198.270925:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880198.270928:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880198.270933:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_03+3] req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880198.270936:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880198.270938:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880198.270943:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:0.0:1788880198.270944:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 02000000:00000001:3.0:1788880198.270946:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880198.270952:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880198.270956:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880198.270958:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000100:00000040:2.0:1788880198.270960:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11167) 00000100:00100000:3.0:1788880198.270962:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780638080:192.168.204.111@tcp:38: 00000100:00000001:3.0:1788880198.270970:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880198.270973:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880198.270974:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:3.0:1788880198.270977:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880198.270993:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:3.0:1788880198.271000:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880198.271007:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 0000000012905f10. 00000400:00000200:0.0:1788880198.271010:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a165c7f9 00000400:00000010:0.0:1788880198.271018:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a165c7f9. 02000000:00000001:3.0:1788880198.271019:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880198.271035:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880198.271044:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780637824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880214 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880198.271074:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880198.271081:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780637824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880214 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880198.271101:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009877e35e. 00000400:00000010:3.0:1788880198.271114:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000200:3.0:1788880198.271126:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780638080, portal 10 00000100:00000001:3.0:1788880198.271131:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880198.271134:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000100:00000040:3.0:1788880198.271155:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880218 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880198.271169:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880198.271179:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880198.271193:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880198.271199:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880198.271203:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000100:00000200:3.0:1788880198.271207:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780638080, offset 0 00000400:00000200:3.0:1788880198.271214:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880198.271223:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880198.271259:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880198.271264:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880198.271268:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880198.271271:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880198.271275:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [54:54:255:4294967295] 192.168.204.111@tcp LPNI seq info [54:54:7:4294967295] 00000800:00000200:0.0:1788880198.271282:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:3.0:1788880198.271295:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000010:0.0:1788880198.271307:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:0.0:1788880198.271315:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:3.0:1788880198.271319:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880198.271320:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880198.271329:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880198.271331:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000010:3.0:1788880198.271333:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000400:00000200:0.0:1788880198.271346:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000400:00000010:0.0:1788880198.271351:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:0.0:1788880198.271359:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880198.271367:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780637952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880214 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880198.271409:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880198.271410:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880198.271420:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780637952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880214 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880198.271432:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880198.271442:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:3.0:1788880198.271457:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880198.271462:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880198.271467:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000040:3.0:1788880198.271468:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 161938 usecs (162 jiffies) 00000100:00000001:3.0:1788880198.271482:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305829539 00000100:00000001:3.0:1788880198.271487:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305834447 00000100:00000001:3.0:1788880198.271489:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305829539 00000100:00000001:3.0:1788880198.271502:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305834447 00000100:00000001:3.0:1788880198.271507:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305829539 00000100:00000001:3.0:1788880198.271512:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305834447 00000100:00000001:3.0:1788880198.271518:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305829539 00000100:00000001:3.0:1788880198.271524:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305834447 00000800:00000200:0.0:1788880198.271528:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880198.271529:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:0.0:1788880198.271539:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000100:00000001:3.0:1788880198.271539:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880198.271560:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880198.271560:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880198.271574:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880198.271581:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880198.271585:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880198.271589:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000100:00000001:3.0:1788880198.271590:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000400:00000010:0.0:1788880198.271595:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000001:3.0:1788880198.271598:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880198.271603:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880198.271613:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880218 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880198.271642:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880198.271647:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880198.271649:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880198.271656:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880218 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880198.271656:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880198.271693:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880198.271700:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880198.271746:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880198.271756:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:0.0:1788880198.271760:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880198.271764:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880198.271767:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000062e8e6f4 00000400:00000010:0.0:1788880198.271769:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000062e8e6f4. 00000100:00000001:0.0:1788880198.271772:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880198.271775:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880214 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880198.271783:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880198.271784:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880214 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880198.271790:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880198.271791:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880198.272270:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880198.272281:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880198.272284:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880198.272286:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880198.272292:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880198.272303:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b01680 00000400:00000200:3.0:1788880198.272363:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x289 [1] + 224 00000800:00000001:3.0:1788880198.272369:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880198.272374:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880198.272375:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880198.272378:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880198.272384:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880198.272387:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780637824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880198.272394:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000175072aa x1875776780637824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880198.272415:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880198.272423:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880198.272425:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.273301:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880198.273305:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880198.273309:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880198.273310:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880198.273312:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000007389ff7. 00000400:00000200:2.0:1788880198.273316:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d71e0867 00000400:00000010:2.0:1788880198.273317:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d71e0867. 00000100:00000001:2.0:1788880198.273320:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880198.273324:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780637952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880198.273333:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780637952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880198.273340:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880198.273341:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.273343:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880198.273344:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880198.273346:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880198.273347:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880198.273349:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880198.273353:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=16 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880198.273357:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=16 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880198.273359:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880198.273360:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880198.273362:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880198.273366:0:201287:0:(ldlm_request.c:1723:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.273368:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880198.273370:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880198.273372:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880198.273373:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880198.273376:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780637952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880198.273383:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880198.273385:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880198.273392:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880198.273395:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780637952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880198.273401:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780637952:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880198.273405:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.273407:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880198.273409:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780637952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880198.273415:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880198.273416:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880198.273418:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000edcbc44b. 02000000:00000001:2.0:1788880198.273421:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880198.273422:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880198.273424:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880198.273425:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880198.273427:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 02000000:00000001:2.0:1788880198.273429:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880198.273430:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880198.273431:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:2.0:1788880198.273433:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880198.273434:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.273436:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.273438:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880198.273439:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880198.273441:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.273442:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880198.273695:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880198.273702:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880198.273705:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880198.273707:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880198.273713:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880198.273721:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 4 MB=0x6aa0250b01800 00000400:00000200:0.0:1788880198.273727:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.204.111@tcp of length 224/224 into md 0x2a1 [1] + 224 00000800:00000001:0.0:1788880198.273731:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880198.273736:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880198.273738:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880198.273742:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880198.273745:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880198.273749:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880198.273758:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880198.273774:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880198.273780:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880198.273783:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.273811:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880198.273814:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880198.273817:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880198.273818:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880198.273820:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001738de44. 00000400:00000200:2.0:1788880198.273823:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000400:00000010:2.0:1788880198.273825:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:2.0:1788880198.273827:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880198.273831:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880198.273845:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880198.273852:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880198.273854:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.273856:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880198.273858:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880198.273859:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880198.273860:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880198.273862:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880198.273866:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000040ba1a3b time=16 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880198.273869:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003184a8ce time=16 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880198.273871:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880198.273872:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880198.273873:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880198.273876:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880198.273883:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.273885:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880198.273886:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-OST0000-osc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880198.273888:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880198.273889:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880198.273901:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880198.273908:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880198.273910:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880198.273911:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880198.273914:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880198.273921:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780638208:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880198.273925:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.273933:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880198.273936:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780638208/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880214 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880198.273942:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880198.273944:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880198.273945:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007b91b305. 02000000:00000001:2.0:1788880198.273949:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880198.273950:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880198.273951:0:201287:0:(genops.c:1244:class_import_put()) import 00000000beb50720 refcount=2 obd=lustre-OST0000-osc-ffff8ca787527000 00000020:00000001:2.0:1788880198.273953:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880198.273954:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 02000000:00000001:2.0:1788880198.273956:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880198.273957:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880198.273958:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000001:2.0:1788880198.273961:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880198.273962:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.273963:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.273968:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880198.273969:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.273970:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880198.273972:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880198.276722:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880198.276768:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880198.276773:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880198.276775:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880198.276782:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880198.276793:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 416 into portal 10 MB=0x6aa0250b01780 00000400:00000200:3.0:1788880198.276798:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 416/416 into md 0x299 [1] + 0 00000800:00000001:3.0:1788880198.276802:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880198.276806:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880198.276808:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880198.276811:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880198.276814:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880198.276819:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880218 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880198.276827:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=416 offset=0 replen=544 req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880218 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880198.276843:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880198.276847:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880198.276849:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.276888:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880198.276892:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880198.276896:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880198.276898:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880198.276900:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009877e35e. 00000400:00000200:2.0:1788880198.276905:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000400:00000010:2.0:1788880198.276907:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000001:2.0:1788880198.276910:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880198.276914:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880218 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880198.276923:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880218 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880198.276929:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880198.276931:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.276933:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880198.276947:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880198.276948:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880198.276951:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880198.276954:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880198.276962:0:201284:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=26 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880198.276967:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=26 v=15 (15 0 0 0) 00000100:00000001:2.0:1788880198.276969:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880198.276971:0:201284:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880198.276973:0:201284:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880198.276977:0:201284:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880218 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880198.276984:0:201284:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.276988:0:201284:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880198.276990:0:201284:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880198.276992:0:201284:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880198.276993:0:201284:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880198.276997:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880218 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880198.277004:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880198.277006:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880198.277008:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880198.277010:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00080000:2.0:1788880198.277016:0:201284:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0001-mdc-ffff8ca787527000: connect to target with instance 4 00000020:00000001:2.0:1788880198.277019:0:201284:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880198.277021:0:201284:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:2.0:1788880198.277023:0:201284:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880198.277025:0:201284:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 00000020:00000001:2.0:1788880198.277028:0:201284:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000020:00000001:2.0:1788880198.277030:0:201284:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000100:00001000:2.0:1788880198.277033:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=1788880198 v=0 (0 0 0 0) 00000100:00001000:2.0:1788880198.277036:0:201284:0:(import.c:2215:obd_at_measure()) AT 0000000040f48a4f change: old=0 new=5 delta=5 (val=1) hist 1 0 0 0 00000100:00000040:2.0:1788880198.277039:0:201284:0:(obd_cksum.h:77:obd_cksum_types_supported_client()) Crypto hash speed: crc 1651, crc32c 9963, adler 1371 00000020:00000040:2.0:1788880198.277044:0:201284:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 00000100:00000001:2.0:1788880198.277052:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 00000002:00000001:2.0:1788880198.277055:0:201284:0:(mdc_request.c:2879:mdc_import_event()) Process entered 00000008:00000020:2.0:1788880198.277058:0:201284:0:(osc_request.c:1062:osc_init_grant()) lustre-MDT0001-mdc-ffff8ca787527000, setting cl_avail_grant: 2097152 cl_lost_grant: 0. chunk bits: 12 cl_max_extent_pages: 16384 00000002:00000001:2.0:1788880198.277062:0:201284:0:(mdc_request.c:3107:mdc_init_ea_size()) Process entered 00000002:00000001:2.0:1788880198.277064:0:201284:0:(mdc_request.c:3114:mdc_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880198.277066:0:201284:0:(obd_class.h:1243:obd_notify()) Process entered 00800000:00000001:2.0:1788880198.277069:0:201284:0:(lmv_obd.c:188:lmv_notify()) Process entered 00000080:00000001:2.0:1788880198.277073:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:2.0:1788880198.277075:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880198.277077:0:201284:0:(lmv_obd.c:223:lmv_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880198.277079:0:201284:0:(obd_class.h:1256:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880198.277081:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:2.0:1788880198.277082:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880198.277083:0:201284:0:(mdc_request.c:2953:mdc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880198.277085:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00080000:2.0:1788880198.277088:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from CONNECTING to FULL 00000100:00000001:2.0:1788880198.277091:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 00000002:00000001:2.0:1788880198.277092:0:201284:0:(mdc_request.c:2879:mdc_import_event()) Process entered 00000002:00000001:2.0:1788880198.277094:0:201284:0:(obd_class.h:1243:obd_notify()) Process entered 00800000:00000001:2.0:1788880198.277095:0:201284:0:(lmv_obd.c:188:lmv_notify()) Process entered 00800000:00000001:2.0:1788880198.277096:0:201284:0:(lmv_obd.c:82:lmv_set_mdc_active()) Process entered 00800000:00000040:2.0:1788880198.277097:0:201284:0:(lmv_obd.c:85:lmv_set_mdc_active()) Searching in lmv 00000000b999beb9 for uuid lustre-MDT0001_UUID (activate=1) 00800000:00000040:2.0:1788880198.277100:0:201284:0:(lmv_obd.c:91:lmv_set_mdc_active()) Target idx 0 is lustre-MDT0000_UUID conn 0x805d548a36a8869e 00800000:00000040:2.0:1788880198.277103:0:201284:0:(lmv_obd.c:91:lmv_set_mdc_active()) Target idx 1 is lustre-MDT0001_UUID conn 0x805d548a36a886a5 00800000:00000040:2.0:1788880198.277105:0:201284:0:(lmv_obd.c:106:lmv_set_mdc_active()) Found OBD lustre-MDT0001-mdc-ffff8ca787527000=ea27006f-52a8-4007-b69a-dcac8279445a device 4 (000000001b5d8280) type mdc at LMV idx 1 00800000:00000040:2.0:1788880198.277108:0:201284:0:(lmv_obd.c:111:lmv_set_mdc_active()) OBD 000000001b5d8280 already active! 00800000:00000001:2.0:1788880198.277110:0:201284:0:(lmv_obd.c:112:lmv_set_mdc_active()) Process leaving via out_lmv_lock (rc=0 : 0 : 0x0) 00000080:00000001:2.0:1788880198.277112:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:2.0:1788880198.277113:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880198.277115:0:201284:0:(lmv_obd.c:223:lmv_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880198.277116:0:201284:0:(obd_class.h:1256:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880198.277117:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:2.0:1788880198.277119:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880198.277121:0:201284:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=5 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000002:00000001:3.0:1788880198.277359:0:201284:0:(mdc_request.c:2953:mdc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880198.277362:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00000001:3.0:1788880198.277365:0:201284:0:(import.c:1648:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00000001:3.0:1788880198.277368:0:201284:0:(import.c:1754:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880198.277371:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880198.277376:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880218 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000002:00000001:2.0:1788880198.277378:0:203758:0:(mdc_request.c:2638:mdc_kuc_reregister_thread()) Process entered 00000020:00000001:2.0:1788880198.277386:0:203758:0:(kernelcomm.c:1325:libcfs_kkuc_group_foreach()) Process entered 00000100:00100000:3.0:1788880198.277387:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780638080:192.168.204.111@tcp:38: 00000020:00000001:2.0:1788880198.277389:0:203758:0:(kernelcomm.c:1339:libcfs_kkuc_group_foreach()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880198.277392:0:203758:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880198.277392:0:203758:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880198.277395:0:203758:0:(genops.c:1251:class_import_put()) Process leaving 00000002:00000001:2.0:1788880198.277396:0:203758:0:(mdc_request.c:2648:mdc_kuc_reregister_thread()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880198.277402:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880198.277405:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880198.277409:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000fd6b7ee1 x1875776780638080/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880218 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880198.277417:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880198.277423:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880198.277427:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 0000000012905f10. 02000000:00000001:3.0:1788880198.277431:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880198.277432:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880198.277433:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880198.277436:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880198.277438:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 000000005e22417f. 02000000:00000001:3.0:1788880198.277442:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880198.277444:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880198.277446:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000fd6b7ee1. 00000100:00000001:3.0:1788880198.277449:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880198.277450:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880198.277452:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880198.277458:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880198.277459:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880198.277461:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880198.277463:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880198.334412:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880198.334415:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880198.334417:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880199.358413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880199.358421:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880199.358426:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788880200.382425:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880200.382428:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880200.382430:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880201.406384:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880201.406388:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880201.406390:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:3.0:1788880202.366387:0:203394:0:(service.c:1710:ptlrpc_at_check_timed()) Process entered 00000100:00000001:3.0:1788880202.366395:0:203394:0:(service.c:1721:ptlrpc_at_check_timed()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880202.430422:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880202.430427:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880202.430431:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788880203.390324:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305834447, at 4305834496 00000100:00000040:2.0:1788880203.390325:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880203.390326:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305834447, at 4305834496 00000100:00000001:2.0:1788880203.390340:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880203.390349:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000016ab1a15. 00000100:00000010:0.0:1788880203.390353:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b63100a0. 00000100:00000010:2.0:1788880203.390353:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000020:00000040:2.0:1788880203.390361:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880203.390367:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880203.390372:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880203.390378:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880203.390388:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 00000100:00000001:2.0:1788880203.390397:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880203.390410:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880203.390422:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880203.390425:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880203.390431:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_00+0] req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880203.390467:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880203.390469:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880203.390472:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880203.390475:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880203.390477:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880203.390478:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880203.390480:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880203.390481:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880203.390484:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000100:00000001:2.0:1788880203.390486:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880203.390490:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880203.390490:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000025994245 x1875776780638464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880203.390496:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880203.390500:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880203.390508:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:0.0:1788880203.390509:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880203.390512:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880203.390522:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_01+1] req@0000000025994245 x1875776780638464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880203.390528:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880203.390532:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880203.390537:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880203.390540:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880203.390541:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880203.390545:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880203.390546:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.390553:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00100000:0.0:1788880203.390555:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780638336:192.168.204.111@tcp:400: 00000100:00000010:2.0:1788880203.390557:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002084a2b4. 00000020:00000040:2.0:1788880203.390563:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880203.390564:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880203.390570:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880203.390572:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880203.390573:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880203.390576:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880203.390577:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880203.390580:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f47bc5fc. 02000000:00000001:0.0:1788880203.390583:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880203.390585:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880203.390591:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f8a9d8a8. 00000100:00000040:2.0:1788880203.390591:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880203.390596:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.390605:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000400:00000010:0.0:1788880203.390607:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050ac3db6. 00000100:00000001:2.0:1788880203.390608:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880203.390616:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002084a2b4] to pc [ptlrpcd_00_02+2] req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880203.390618:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d6cef011. 00000100:00000200:0.0:1788880203.390629:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780638336, portal 25 00000100:00000001:0.0:1788880203.390634:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880203.390637:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 00000100:00000040:0.0:1788880203.390648:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880219 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880203.390654:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880203.390664:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880203.390673:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880203.390678:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:2.0:1788880203.390679:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000100:00000001:0.0:1788880203.390682:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000020:00000040:2.0:1788880203.390684:0:200894:0:(genops.c:1231:class_import_get()) import 00000000beb50720 refcount=3 obd=lustre-OST0000-osc-ffff8ca787527000 00000100:00000001:3.0:1788880203.390684:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880203.390689:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880203.390691:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880203.390691:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880203.390695:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000010:0.0:1788880203.390697:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 02000000:00000001:2.0:1788880203.390698:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880203.390703:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780638336, offset 0 00000100:00000040:3.0:1788880203.390704:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880203.390710:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000400:00000200:0.0:1788880203.390718:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880203.390724:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880203.390731:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880203.390734:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880203.390735:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880203.390738:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880203.390741:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880203.390745:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880203.390750:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:3.0:1788880203.390752:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.390756:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000400:00000200:0.0:1788880203.390759:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00100000:3.0:1788880203.390760:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780638592:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880203.390761:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880203.390768:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880203.390768:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880203.390770:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_03+3] req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880203.390774:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880203.390776:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880203.390778:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880203.390783:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880203.390785:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [56:56:256:4294967295] 192.168.204.111@tcp LPNI seq info [56:56:8:4294967295] 02000000:00000010:3.0:1788880203.390789:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a4a3549f. 00000100:00000001:2.0:1788880203.390790:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880203.390794:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880203.390796:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11172) 00000400:00000010:3.0:1788880203.390800:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ba5e1826. 00000400:00000010:3.0:1788880203.390807:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000100:00000001:2.0:1788880203.390811:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:3.0:1788880203.390812:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780638592, portal 10 00000100:00000001:2.0:1788880203.390816:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880203.390817:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880203.390818:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880203.390820:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880203.390822:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938429184 : -126823771122432 : ffff8ca78a2c5500) 00000100:00000040:2.0:1788880203.390828:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880203.390833:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880203.390834:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880203.390843:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:0.0:1788880203.390844:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000100:00000001:2.0:1788880203.390854:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1788880203.390857:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880203.390857:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880203.390858:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:2.0:1788880203.390860:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880203.390863:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880203.390864:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880203.390865:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880203.390867:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880203.390868:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880203.390872:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:3.0:1788880203.390873:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 00000100:00100000:2.0:1788880203.390874:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780638720:192.168.204.111@tcp:400: 00000800:00000200:0.0:1788880203.390880:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880203.390880:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [57:57:255:4294967295] 192.168.204.111@tcp LPNI seq info [57:57:7:4294967295] 00000100:00000200:3.0:1788880203.390880:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780638592, offset 0 00000100:00000001:2.0:1788880203.390888:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:0.0:1788880203.390889:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880203.390892:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:3.0:1788880203.390894:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880203.390895:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880203.390899:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880203.390902:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000010:2.0:1788880203.390903:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004080df20. 00000100:00000001:0.0:1788880203.390905:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880203.390907:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880203.390913:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000038fe0168. 00000100:00000001:0.0:1788880203.390914:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305839567 00000800:00000200:1.0:1788880203.390914:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880203.390918:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000841b943f. 00000400:00000200:3.0:1788880203.390918:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880203.390921:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000100:00000010:0.0:1788880203.390922:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009f22a9a0. 00000100:00000200:2.0:1788880203.390923:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780638720, portal 4 00000400:00000200:3.0:1788880203.390927:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880203.390928:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880203.390930:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:1.0:1788880203.390930:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880203.390931:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 00000400:00000200:3.0:1788880203.390935:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880203.390939:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305839567 00000800:00000200:1.0:1788880203.390939:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880203.390941:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1788880203.390943:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880219 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880203.390945:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880203.390948:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880203.390950:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880203.390954:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005b8bebe2. 00000100:00000040:1.0:1788880203.390956:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1952808 usecs (1953 jiffies) 00000400:00000200:3.0:1788880203.390956:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [58:58:254:4294967295] 192.168.204.111@tcp LPNI seq info [58:58:6:4294967295] 00000100:00000001:0.0:1788880203.390962:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880203.390963:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305836449 00000100:00000001:1.0:1788880203.390968:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305839567 00000100:00000001:0.0:1788880203.390969:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305839567 00000100:00000001:1.0:1788880203.390972:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305836449 00000100:00000001:2.0:1788880203.390974:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880203.390978:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305834447 00000100:00000001:1.0:1788880203.390979:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305839567 00000100:00000040:2.0:1788880203.390982:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880203.390984:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305836449 00000100:00000001:0.0:1788880203.390987:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305839567 00000100:00000001:1.0:1788880203.390988:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305839567 00000400:00000010:2.0:1788880203.390988:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001c5b8d0f. 00000100:00000200:2.0:1788880203.390994:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875776780638720, offset 0 00000400:00000200:3.0:1788880203.390994:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880203.390995:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305836449 00000100:00000010:0.0:1788880203.390996:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c0bfe00. 00000400:00000200:2.0:1788880203.391006:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880203.391008:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:3.0:1788880203.391014:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880203.391017:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880203.391023:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305839567 00000400:00000200:2.0:1788880203.391023:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:3.0:1788880203.391023:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000749ed09e. 00000100:00000001:0.0:1788880203.391025:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880203.391029:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880203.391031:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880203.391031:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1788880203.391035:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880203.391037:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880203.391038:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880203.391039:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880203.391045:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880203.391045:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880203.391046:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:3.0:1788880203.391046:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880203.391051:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:3.0:1788880203.391053:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000749ed09e type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880203.391054:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [59:59:253:4294967295] 192.168.204.111@tcp LPNI seq info [59:59:5:4294967295] 00000100:00000001:0.0:1788880203.391058:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880203.391058:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880203.391061:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880203.391063:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880203.391063:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:3.0:1788880203.391067:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 144792 usecs (145 jiffies) 00000100:00000001:0.0:1788880203.391070:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880203.391070:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880203.391073:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880203.391074:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880203.391074:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880203.391075:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305834641 00000100:00000001:0.0:1788880203.391078:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880203.391079:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880203.391082:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305839567 00000100:00000001:0.0:1788880203.391084:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880203.391085:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880203.391089:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305834641 00000100:00000001:1.0:1788880203.391092:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880203.391093:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000007cb951ea. 00000100:00000001:3.0:1788880203.391093:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305839567 00000100:00000001:3.0:1788880203.391099:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305834641 00000800:00000200:2.0:1788880203.391101:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880203.391106:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880203.391109:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000007cb951ea type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880203.391110:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305839567 00000100:00000001:3.0:1788880203.391115:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305834641 00000100:00000001:2.0:1788880203.391116:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880203.391118:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305839567 00000100:00000040:2.0:1788880203.391121:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: using a timeout of 3368 usecs (4 jiffies) 00000100:00000001:3.0:1788880203.391122:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880203.391127:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305834500 00000100:00000001:3.0:1788880203.391128:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.391133:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305839567 00000100:00000001:3.0:1788880203.391134:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.391139:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305834500 00000100:00000001:3.0:1788880203.391140:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.391144:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305839567 00000100:00000001:2.0:1788880203.391148:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305834500 00000100:00000001:3.0:1788880203.391148:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880203.391152:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880203.391153:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305839567 00000800:00000200:0.0:1788880203.391154:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880203.391157:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880203.391158:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305834500 00000800:00000010:0.0:1788880203.391160:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f92e72a2. 00000100:00000001:3.0:1788880203.391160:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880203.391161:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305839567 00000100:00000001:3.0:1788880203.391164:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880203.391168:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880203.391169:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880203.391169:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.391174:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.391177:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.391179:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880203.391180:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880203.391183:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880203.391185:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880203.391188:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000100:00000001:2.0:1788880203.391188:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880203.391190:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880203.391192:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880203.391193:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:2.0:1788880203.391195:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880203.391200:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880203.391206:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880219 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880203.391217:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880203.391220:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880219 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880203.391228:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880203.391230:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880203.391264:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880203.391267:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:0.0:1788880203.391270:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880203.391274:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880203.391276:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e249bdf9 00000400:00000010:0.0:1788880203.391278:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e249bdf9. 00000100:00000001:0.0:1788880203.391280:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880203.391283:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780638464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880203.391290:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880203.391298:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780638464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880203.391304:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880203.391306:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880203.391315:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880203.391317:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000749ed09e. 00000400:00000200:0.0:1788880203.391321:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880203.391325:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880203.391327:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 00000400:00000010:0.0:1788880203.391328:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000100:00000001:0.0:1788880203.391331:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880203.391333:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880203.391340:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880203.391342:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880203.391349:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880203.391350:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880203.391419:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880203.391422:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000007cb951ea. 00000400:00000200:0.0:1788880203.391425:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880203.391428:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880203.391431:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001c5b8d0f 00000400:00000010:0.0:1788880203.391433:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001c5b8d0f. 00000100:00000001:0.0:1788880203.391435:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880203.391438:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880219 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880203.391456:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880203.391458:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880219 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880203.391465:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880203.391467:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880203.395671:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880203.395692:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880203.395698:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880203.395701:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880203.395713:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880203.395741:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b01880 00000400:00000200:0.0:1788880203.395753:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x2a9 [1] + 224 00000800:00000001:0.0:1788880203.395764:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880203.395768:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880203.395771:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880203.395776:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880203.395784:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880203.395792:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880203.395805:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880203.395829:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880203.395839:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880203.395842:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880203.395844:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880203.395847:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880203.395861:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880203.395866:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880203.395868:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b01900 00000100:00000001:2.0:1788880203.395871:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880203.395876:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x2ad [1] + 224 00000800:00000001:0.0:1788880203.395881:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880203.395885:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880203.395885:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:0.0:1788880203.395889:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:2.0:1788880203.395890:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880203.395894:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050ac3db6. 00000400:00000200:0.0:1788880203.395896:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880203.395902:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d6cef011 00000400:00000010:2.0:1788880203.395905:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d6cef011. 00000100:00000001:2.0:1788880203.395909:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880203.395910:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880203.395915:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880203.395918:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780638464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880203.395936:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880203.395940:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000025994245 x1875776780638464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880203.395943:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880203.395944:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.395946:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880203.395949:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880203.395951:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880203.395952:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880203.395954:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880203.395961:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=31 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880203.395967:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=31 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880203.395970:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880203.395971:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880203.395973:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880203.395975:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.395977:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880203.395979:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880203.395985:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:2.0:1788880203.395989:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880203.395991:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880203.395996:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880203.395999:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:0.0:1788880203.396000:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.396003:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880203.396005:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880203.396010:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000040:2.0:1788880203.396010:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880203.396022:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b01980 00000100:00000001:1.0:1788880203.396024:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:2.0:1788880203.396025:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780638336:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880203.396029:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880203.396033:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x2b9 [1] + 224 00000100:00000001:2.0:1788880203.396033:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.396039:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:0.0:1788880203.396041:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880203.396042:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:0.0:1788880203.396046:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880203.396046:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:2.0:1788880203.396046:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780638336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880203.396050:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000831ba52e. 00000400:00000200:0.0:1788880203.396051:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880203.396056:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000200:0.0:1788880203.396060:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:1.0:1788880203.396061:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:2.0:1788880203.396065:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880203.396068:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880203.396068:0:201286:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:2.0:1788880203.396069:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880203.396074:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f8a9d8a8. 00000100:00000200:1.0:1788880203.396076:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780638464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880203.396077:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880203.396079:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880203.396082:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880203.396085:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880203.396089:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880203.396092:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 02000000:00000001:2.0:1788880203.396098:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880203.396100:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880203.396103:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000001:2.0:1788880203.396108:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:0.0:1788880203.396110:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880203.396110:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780638464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880203.396111:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.396116:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.396127:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880203.396130:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880203.396136:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880203.396136:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880203.396140:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880203.396141:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880203.396145:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788880203.396146:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880203.396148:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880203.396151:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:0.0:1788880203.396152:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880203.396153:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880203.396156:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880203.396158:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880203.396161:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:1.0:1788880203.396163:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=21 v=5 (5 0 0 0) 00000800:00000001:0.0:1788880203.396164:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880203.396170:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=21 v=5 (1 0 0 0) 00000400:00000200:0.0:1788880203.396172:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880203.396176:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880203.396179:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880203.396181:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:0.0:1788880203.396183:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 4 MB=0x6aa0250b01a00 00010000:00000001:1.0:1788880203.396189:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880203.396191:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880203.396192:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.204.111@tcp of length 224/224 into md 0x2c1 [1] + 224 00000100:00000001:3.0:1788880203.396195:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880203.396196:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000800:00000001:0.0:1788880203.396199:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788880203.396200:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:3.0:1788880203.396201:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880203.396205:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000001:3.0:1788880203.396205:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880203.396207:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880203.396208:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880203.396211:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:1.0:1788880203.396213:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780638464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880203.396216:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ba5e1826. 00000400:00000200:0.0:1788880203.396221:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880203.396222:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000400:00000010:3.0:1788880203.396226:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:1.0:1788880203.396262:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880203.396262:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880203.396265:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880203.396267:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880203.396270:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880203.396270:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880203.396278:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880203.396288:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780638464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880203.396292:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880203.396307:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880203.396319:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:1.0:1788880203.396321:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780638464:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880203.396323:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880203.396327:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880203.396331:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880203.396332:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880203.396333:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880203.396336:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880203.396337:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880203.396338:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:3.0:1788880203.396340:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880203.396343:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780638464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880203.396346:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:3.0:1788880203.396346:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=31 v=5 (5 0 0 0) 00000800:00000001:0.0:1788880203.396352:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788880203.396354:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=5 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880203.396360:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880203.396361:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880203.396362:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880203.396363:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880203.396364:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880203.396367:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000005842920. 00000100:00000001:2.0:1788880203.396367:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:3.0:1788880203.396367:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:1.0:1788880203.396374:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000001:2.0:1788880203.396375:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00010000:00080000:3.0:1788880203.396375:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788880203.396377:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880203.396380:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000001:2.0:1788880203.396380:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000020:00000001:1.0:1788880203.396384:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880203.396386:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 02000000:00000001:1.0:1788880203.396390:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880203.396392:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00010000:00000001:3.0:1788880203.396392:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880203.396394:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:3.0:1788880203.396397:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:1.0:1788880203.396398:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000400:00000010:2.0:1788880203.396398:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000038fe0168. 00000100:00000040:3.0:1788880203.396400:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880203.396401:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880203.396404:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880203.396404:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000200:2.0:1788880203.396408:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000841b943f 00000100:00000001:3.0:1788880203.396408:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880203.396410:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:2.0:1788880203.396412:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000841b943f. 00000100:00000001:1.0:1788880203.396413:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880203.396415:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880203.396417:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880203.396420:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.396420:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880203.396428:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880203.396429:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880203.396433:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880203.396435:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880203.396453:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880203.396456:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880203.396474:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880203.396478:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1788880203.396481:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780638592:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880203.396482:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880203.396485:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880203.396488:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880203.396490:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.396491:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880203.396493:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:2.0:1788880203.396495:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880203.396498:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002084a2b4 x1875776780638592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:2.0:1788880203.396502:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000040ba1a3b time=21 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880203.396510:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003184a8ce time=21 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880203.396511:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880203.396513:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880203.396515:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000010:3.0:1788880203.396516:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a4a3549f. 00000100:00000001:2.0:1788880203.396518:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880203.396521:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00010000:00000001:2.0:1788880203.396522:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:3.0:1788880203.396524:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880203.396526:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00010000:00080000:2.0:1788880203.396529:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880203.396530:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880203.396532:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f47bc5fc. 02000000:00000001:3.0:1788880203.396537:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880203.396539:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880203.396541:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002084a2b4. 00010000:00000001:2.0:1788880203.396543:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880203.396545:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880203.396546:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880203.396548:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-OST0000-osc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880203.396548:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.396552:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880203.396552:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.396554:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880203.396558:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880203.396559:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880203.396562:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.396564:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880203.396566:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880203.396567:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880203.396569:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880203.396570:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880203.396573:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880203.396584:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780638720:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880203.396588:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.396589:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880203.396591:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780638720/t0(0) o400->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880219 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880203.396596:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880203.396598:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880203.396599:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004080df20. 02000000:00000001:2.0:1788880203.396602:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880203.396603:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880203.396604:0:201288:0:(genops.c:1244:class_import_put()) import 00000000beb50720 refcount=2 obd=lustre-OST0000-osc-ffff8ca787527000 00000020:00000001:2.0:1788880203.396606:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880203.396607:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 02000000:00000001:2.0:1788880203.396609:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880203.396610:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880203.396611:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000100:00000001:2.0:1788880203.396614:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880203.396615:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.396617:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880203.396620:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880203.396622:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880203.396623:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880203.396625:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880203.454381:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880203.454393:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880203.454396:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880203.454400:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880203.454407:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880203.454410:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880203.454411:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880203.454414:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880203.454417:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880203.454419:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880203.454517:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880203.454522:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880203.454525:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880204.478325:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880204.478330:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.478334:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880204.478336:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880204.478338:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880204.478340:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.478342:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880204.478344:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.478345:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.478346:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880204.478395:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880204.478399:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880204.478401:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00000001:1.0:1788880204.926369:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880204.926380:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-cliir (0000000008e74578) refs 2 10000000:00000001:1.0:1788880204.926388:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.926391:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000001:1.0:1788880204.926393:0:203401:0:(mgc_request.c:120:config_log_put()) Process leaving 10000000:00000001:1.0:1788880204.926395:0:203401:0:(mgc_request.c:601:do_requeue()) Process entered 10000000:01000000:1.0:1788880204.926398:0:203401:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:00000001:1.0:1788880204.926400:0:203401:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:1.0:1788880204.926403:0:203401:0:(mgc_request.c:1801:mgc_process_log()) Process log lustre-cliir-ffff8ca787527000 from 12 10000000:00000001:1.0:1788880204.926407:0:203401:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:1.0:1788880204.926410:0:203401:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00000100:00000010:1.0:1788880204.926434:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880204.926441:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880204.926447:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880204.926452:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880204.926459:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880204.926466:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 0000000005842920. 00000100:00000001:1.0:1788880204.926473:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.926478:0:203401:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:1.0:1788880204.926481:0:203401:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:1.0:1788880204.926487:0:203401:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at 00000000035e4197. 00010000:00000001:1.0:1788880204.926491:0:203401:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:1.0:1788880204.926494:0:203401:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 000000000e3a19ac. 00000020:00000001:1.0:1788880204.926498:0:203401:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:1.0:1788880204.926502:0:203401:0:(lustre_handles.c:93:class_handle_hash()) added object 000000000e3a19ac with handle 0x805d548a36a886c1 to hash 00000020:00000001:1.0:1788880204.926503:0:203401:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:1.0:1788880204.926504:0:203401:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:1.0:1788880204.926522:0:203401:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:1.0:1788880204.926526:0:203401:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1788880204.926529:0:203401:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880204.926531:0:203401:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880204.926539:0:203401:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880204.926548:0:203401:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880204.926555:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880204.926557:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880204.926565:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.926568:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249914662080 : -126823794889536 : ffff8ca788c1acc0) 00000100:00000001:1.0:1788880204.926571:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.926572:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000001:1.0:1788880204.926582:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880204.926585:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.926587:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880204.926589:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880204.926600:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.926610:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880204.926613:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.926617:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880204.926620:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880204.926622:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880204.926625:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780638848:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880204.926630:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880204.926637:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880204.926652:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.926655:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880204.926659:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000008eac1e29. 02000000:00000001:1.0:1788880204.926661:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880204.926669:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000831ba52e. 00000400:00000010:1.0:1788880204.926677:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000200:1.0:1788880204.926685:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780638848, portal 25 00000100:00000001:1.0:1788880204.926687:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.926688:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000040:1.0:1788880204.926693:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.926700:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880204.926703:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880204.926705:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000066e8f89d. 00000100:00000200:1.0:1788880204.926707:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780638848, offset 0 00000400:00000200:1.0:1788880204.926713:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880204.926728:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880204.926735:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880204.926739:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880204.926743:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880204.926746:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880204.926750:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [60:60:256:4294967295] 192.168.204.111@tcp LPNI seq info [60:60:8:4294967295] 00000400:00000200:1.0:1788880204.926761:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880204.926768:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880204.926773:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000800:00000200:1.0:1788880204.926781:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880204.926789:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880204.926794:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:1.0:1788880204.926818:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.926823:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305841103 00000100:00000001:1.0:1788880204.926831:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305839567 00000100:00000001:1.0:1788880204.926844:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305841103 00000100:00000001:1.0:1788880204.926849:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305839567 00000100:00000001:1.0:1788880204.926853:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305841103 00000100:00000001:1.0:1788880204.926870:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305839567 00000100:00000001:1.0:1788880204.926873:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305841103 00000100:00000001:1.0:1788880204.926876:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305839567 00000100:00000001:1.0:1788880204.926878:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.926880:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.926882:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880204.926884:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880204.926886:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 0000000029c63527 going to sleep for 11 seconds 00000100:00000001:1.0:1788880204.926890:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880204.926892:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880204.926953:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880204.926965:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:3.0:1788880204.926972:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880204.926980:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880204.926987:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000066e8f89d 00000400:00000010:3.0:1788880204.926989:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000066e8f89d. 00000100:00000001:3.0:1788880204.926995:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880204.927014:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880204.927027:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880204.927030:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880204.927036:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880204.927038:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880204.928655:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.928668:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880204.928672:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.928674:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880204.928684:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880204.928699:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b01a80 00000400:00000200:0.0:1788880204.928708:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0x2c9 [1] + 224 00000800:00000001:0.0:1788880204.928715:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.928723:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880204.928725:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880204.928730:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880204.928735:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880204.928741:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880204.928751:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.928769:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880204.928776:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.928779:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.928808:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880204.928821:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880204.928823:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880204.928826:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000831ba52e. 00000400:00000200:1.0:1788880204.928832:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:1.0:1788880204.928834:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:1.0:1788880204.928838:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880204.928844:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880204.928867:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.928875:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880204.928877:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.928880:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880204.928882:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880204.928885:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880204.928887:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.928890:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880204.928896:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=32 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880204.928899:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=32 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880204.928908:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880204.928910:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.928912:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880204.928913:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.928915:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880204.928919:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.928924:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.928926:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880204.928927:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880204.928930:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880204.928934:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780638848:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880204.928937:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.928939:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.928940:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880204.928942:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.928943:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.928947:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880204.928949:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.928952:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880204.928954:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.928957:0:203401:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:1.0:1788880204.928961:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880204.928963:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880204.928967:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000000e3a19ac refcount=4 00000020:00000001:1.0:1788880204.928969:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:1.0:1788880204.928971:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000040:1.0:1788880204.928977:0:203401:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 000000000e3a19ac, remote cookie: 0x92f11317f60a7abb, flags: 0x0 00010000:00000001:1.0:1788880204.928980:0:203401:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:1.0:1788880204.928985:0:203401:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:1.0:1788880204.928992:0:203401:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:1.0:1788880204.928994:0:203401:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:1.0:1788880204.928995:0:203401:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:1.0:1788880204.928997:0:203401:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x2:0x0].0x0 (00000000035e4197) refcount = 1 00010000:00010000:1.0:1788880204.929001:0:203401:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.929008:0:203401:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:1.0:1788880204.929010:0:203401:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:1.0:1788880204.929011:0:203401:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788880204.929014:0:203401:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:1.0:1788880204.929016:0:203401:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880204.929018:0:203401:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.929024:0:203401:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:1.0:1788880204.929027:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.929028:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880204.929030:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.929031:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880204.929032:0:203401:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.929035:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.929038:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780638848/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.929049:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880204.929052:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880204.929055:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000008eac1e29. 02000000:00000001:1.0:1788880204.929058:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880204.929060:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880204.929062:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880204.929065:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880204.929067:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 0000000005842920. 02000000:00000001:1.0:1788880204.929072:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880204.929074:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880204.929076:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:1.0:1788880204.929079:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880204.929085:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:1.0:1788880204.929087:0:203401:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.929090:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880204.929091:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-cliir (0000000008e74578) refs 3 10000000:00000001:1.0:1788880204.929094:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.929096:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880204.929097:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880204.929098:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000000e3a19ac refcount=3 00000020:00000001:1.0:1788880204.929100:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:1.0:1788880204.929102:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:1.0:1788880204.929104:0:203401:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:1.0:1788880204.929105:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.929106:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880204.929108:0:203401:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.929110:0:203401:0:(mgc_request.c:1519:mgc_process_recover_log()) Process entered 10000000:00000010:1.0:1788880204.929114:0:203401:0:(mgc_request.c:1531:mgc_process_recover_log()) kmalloced '(folios)': 32 at 00000000ff4bb2db. 00000100:00000010:1.0:1788880204.929128:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880204.929131:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880204.929135:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880204.929137:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880204.929140:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880204.929145:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 0000000005842920. 00000100:00000001:1.0:1788880204.929158:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.929164:0:203401:0:(client.c:227:ptlrpc_prep_bulk_imp()) Process entered 00000100:00000010:1.0:1788880204.929167:0:203401:0:(client.c:169:ptlrpc_new_bulk()) kmalloced '(desc)': 984 at 000000008eac1e29. 00000100:00000010:1.0:1788880204.929169:0:203401:0:(client.c:174:ptlrpc_new_bulk()) kmalloced '(desc->bd_vec)': 64 at 00000000083715bf. 00000020:00000040:1.0:1788880204.929171:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880204.929175:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880204.929176:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880204.929179:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.929181:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249914662080 : -126823794889536 : ffff8ca788c1acc0) 00000100:00000001:1.0:1788880204.929182:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.929183:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000001:1.0:1788880204.929186:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880204.929187:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.929191:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880204.929192:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880204.929195:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.929199:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880204.929201:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.929203:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880204.929204:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880204.929205:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880204.929207:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780638976:192.168.204.111@tcp:256: 00000100:00000001:1.0:1788880204.929209:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880204.929212:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880204.929214:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.929216:0:203401:0:(niobuf.c:346:ptlrpc_register_bulk()) Process entered 00000400:00000010:1.0:1788880204.929219:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000831ba52e. 00000400:00000010:1.0:1788880204.929223:0:203401:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 000000004675c888. 00000100:00000200:1.0:1788880204.929228:0:203401:0:(niobuf.c:453:ptlrpc_register_bulk()) Setup 1 bulk put-sink buffers: 4 pages 16384 bytes, mbits x0x6aa0250b01b00-0x6aa0250b01b00, portal 33 00000100:00000001:1.0:1788880204.929231:0:203401:0:(niobuf.c:455:ptlrpc_register_bulk()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.929260:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880204.929264:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000c9a81c5. 02000000:00000001:1.0:1788880204.929266:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880204.929269:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000400:00000010:1.0:1788880204.929272:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000200:1.0:1788880204.929275:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780638976, portal 25 00000100:00000001:1.0:1788880204.929276:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.929277:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000040:1.0:1788880204.929281:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.929286:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880204.929289:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880204.929291:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000100:00000200:1.0:1788880204.929293:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 304 bytes to portal 26, xid 1875776780638976, offset 0 00000400:00000200:1.0:1788880204.929299:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880204.929314:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880204.929319:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880204.929323:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880204.929326:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880204.929329:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880204.929333:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [61:61:256:4294967295] 192.168.204.111@tcp LPNI seq info [61:61:8:4294967295] 00000400:00000200:1.0:1788880204.929342:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880204.929347:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 304 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880204.929352:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000800:00000200:1.0:1788880204.929358:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880204.929369:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880204.929372:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 400 niov 1 nkiov 1 00000100:00000001:1.0:1788880204.929387:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.929403:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305841106 00000100:00000001:1.0:1788880204.929409:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305841103 00000100:00000001:1.0:1788880204.929425:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305841106 00000100:00000001:1.0:1788880204.929430:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305841103 00000100:00000001:1.0:1788880204.929434:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305841106 00000100:00000001:1.0:1788880204.929438:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305841103 00000100:00000001:1.0:1788880204.929441:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305841106 00000100:00000001:1.0:1788880204.929447:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305841103 00000100:00000001:1.0:1788880204.929450:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.929453:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.929455:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880204.929457:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880204.929458:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 0000000029c63527 going to sleep for 11 seconds 00000100:00000001:1.0:1788880204.929461:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880204.929464:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880204.929495:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880204.929500:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:0.0:1788880204.929504:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880204.929517:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880204.929521:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:0.0:1788880204.929524:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:0.0:1788880204.929527:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880204.929531:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.929541:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880204.929544:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.929550:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880204.929551:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880204.930555:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880204.930568:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880204.930572:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880204.930574:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880204.930581:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880204.930593:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 44 into portal 33 MB=0x6aa0250b01b00 00000400:00000200:3.0:1788880204.930600:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 21 from 12345-192.168.204.111@tcp of length 44/44 into md 0x2d1 [4] + 0 00000400:00000010:3.0:1788880204.930604:0:201278:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000831ba52e. 00000400:00000200:3.0:1788880204.930607:0:201278:0:(lib-md.c:42:lnet_md_unlink()) Queueing unlink of md 000000004675c888 00000800:00000001:3.0:1788880204.930611:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880204.930618:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880204.930621:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880204.930624:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880204.930629:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004675c888 00000400:00000010:3.0:1788880204.930631:0:201278:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 000000004675c888. 00000100:00000001:3.0:1788880204.930635:0:201278:0:(events.c:174:client_bulk_callback()) Process entered 00000100:00000200:3.0:1788880204.930636:0:201278:0:(events.c:192:client_bulk_callback()) event type 2, status 0, req 0000000008b25f66 desc 000000008eac1e29 mbits 1875776780638976 00000100:00000001:3.0:1788880204.930647:0:201278:0:(events.c:217:client_bulk_callback()) Process leaving 00000400:00000200:3.0:1788880204.930654:0:201278:0:(lib-md.c:96:lnet_cpt_of_md()) Couldn't resolve first page of md 0000000000000000 with offset 0 00000400:00000200:3.0:1788880204.930658:0:201278:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880204.930663:0:201278:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [62:62:256:4294967295] 192.168.204.111@tcp LPNI seq info [62:62:8:4294967295] 00000400:00000200:3.0:1788880204.930673:0:201278:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) 192.168.204.111@tcp : ACK try# 0 00000800:00000200:3.0:1788880204.930678:0:201278:0:(socklnd_cb.c:931:ksocknal_send()) sending 0 bytes in 4 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880204.930682:0:201278:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 296 at 00000000a4a3549f. 00000100:00000001:1.0:1788880204.930687:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:3.0:1788880204.930698:0:201278:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880204.930699:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880204.930707:0:201278:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880204.930711:0:201278:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000a4a3549f type 0, nob 96 niov 1 nkiov 0 00000800:00000001:0.0:1788880204.930781:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000200:3.0:1788880204.930799:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880204.930804:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 296 at 00000000a4a3549f. 00000400:00000200:3.0:1788880204.930808:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000001:0.0:1788880204.930813:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880204.930814:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: ACK: OK 00000800:00000001:0.0:1788880204.931575:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.931583:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880204.931586:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.931588:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880204.931596:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880204.931609:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 240 into portal 25 MB=0x6aa0250b01b00 00000400:00000200:0.0:1788880204.931616:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 240/240 into md 0x2d5 [1] + 224 00000800:00000001:0.0:1788880204.931621:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.931628:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880204.931630:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880204.931634:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880204.931640:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880204.931644:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880204.931654:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=240 offset=224 replen=240 req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.931672:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880204.931678:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.931681:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.931713:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880204.931721:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880204.931723:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880204.931725:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:1.0:1788880204.931743:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:1.0:1788880204.931745:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:1.0:1788880204.931749:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880204.931755:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880204.931764:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.931771:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880204.931773:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.931776:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880204.931779:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880204.931781:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880204.931783:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.931786:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880204.931792:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=32 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880204.931795:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=32 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880204.931797:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880204.931799:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.931801:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880204.931803:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.931805:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880204.931808:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Bulk req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880204.931821:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Bulk to Interpret req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 2 fl Bulk:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.931828:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880204.931830:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880204.931833:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880204.931854:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780638976:192.168.204.111@tcp:256: 00000100:00000001:1.0:1788880204.931859:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.931862:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.931865:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880204.931866:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.931869:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.931875:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880204.931878:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.931880:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880204.931882:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:1.0:1788880204.931887:0:203401:0:(mgc_request.c:1599:mgc_process_recover_log()) Latest version 12, more 0. 10000000:00000001:1.0:1788880204.931892:0:203401:0:(mgc_request.c:1308:mgc_apply_recover_logs()) Process entered 10000000:00000010:1.0:1788880204.931898:0:203401:0:(mgc_request.c:1331:mgc_apply_recover_logs()) kmalloced '(nidlist)': 640 at 00000000b278c5d9. 10000000:00000010:1.0:1788880204.931902:0:203401:0:(mgc_request.c:1343:mgc_apply_recover_logs()) kmalloced '(obdname)': 128 at 00000000ca48580d. 10000000:00000200:1.0:1788880204.931908:0:203401:0:(mgc_request.c:1241:mgc_build_nidlist()) use 1 from 1 NIDs 10000000:00000040:1.0:1788880204.931911:0:203401:0:(mgc_request.c:1440:mgc_apply_recover_logs()) Looking for OBD lustre-MDT0001-mdc-ffff8ca787527000 00010000:00000001:1.0:1788880204.931919:0:203401:0:(ldlm_lib.c:159:client_import_add_nids_to_conn()) Process entered 00000100:00000001:1.0:1788880204.931922:0:203401:0:(lustre_peer.c:229:class_check_uuid()) Process entered 00000100:00000040:1.0:1788880204.931925:0:203401:0:(lustre_peer.c:232:class_check_uuid()) check if uuid 192.168.204.111@tcp has 192.168.204.111@tcp. 00000100:00000001:1.0:1788880204.931928:0:203401:0:(lustre_peer.c:251:class_check_uuid()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.931930:0:203401:0:(lustre_peer.c:189:class_add_nids_to_uuid()) Process entered 00000100:00000200:1.0:1788880204.931932:0:203401:0:(lustre_peer.c:194:class_add_nids_to_uuid()) Comparing 192.168.204.111@tcp with 192.168.204.111@tcp 00000100:00000200:1.0:1788880204.931934:0:203401:0:(lustre_peer.c:201:class_add_nids_to_uuid()) Updating UUID '192.168.204.111@tcp' 00000400:00000200:1.0:1788880204.931937:0:203401:0:(peer.c:1363:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:1.0:1788880204.931941:0:203401:0:(api-ni.c:3274:LNetNIInit()) refs 1 00000400:00000200:1.0:1788880204.931944:0:203401:0:(peer.c:1377:LNetAddPeer()) Adding NID[0]: 192.168.204.111@tcp 00000400:00000200:1.0:1788880204.931949:0:203401:0:(peer.c:1822:lnet_peer_add()) peer 192.168.204.111@tcp NID flags 0x100001: -17 00000400:00000200:1.0:1788880204.931957:0:203401:0:(peer.c:2890:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:1.0:1788880204.931960:0:203401:0:(peer.c:2899:lnet_discover_peer_locked()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp: 0. pending discovery 00000400:00000200:1.0:1788880204.931964:0:203401:0:(peer.c:1431:LNetAddPeer()) A peer exists for NID 192.168.204.111@tcp 00000100:00000001:1.0:1788880204.931966:0:203401:0:(lustre_peer.c:219:class_add_nids_to_uuid()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.931972:0:203401:0:(ldlm_lib.c:174:client_import_add_nids_to_conn()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:1.0:1788880204.931975:0:203401:0:(mgc_request.c:1486:mgc_apply_recover_logs()) ir apply logs 12/12 for lustre-MDT0001-mdc-ffff8ca787527000 10000000:00000010:1.0:1788880204.931977:0:203401:0:(mgc_request.c:1491:mgc_apply_recover_logs()) kfreed 'obdname': 128 at 00000000ca48580d. 10000000:00000010:1.0:1788880204.931980:0:203401:0:(mgc_request.c:1493:mgc_apply_recover_logs()) kfreed 'nidlist': 640 at 00000000b278c5d9. 10000000:00000001:1.0:1788880204.931983:0:203401:0:(mgc_request.c:1495:mgc_apply_recover_logs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.931985:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.931988:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780638976/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880220 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.931994:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880204.931996:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880204.931998:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000c9a81c5. 02000000:00000001:1.0:1788880204.932002:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880204.932004:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880204.932005:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880204.932008:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880204.932010:0:203401:0:(client.c:312:ptlrpc_free_bulk()) Process entered 00000020:00000001:1.0:1788880204.932012:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880204.932013:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880204.932015:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000010:1.0:1788880204.932018:0:203401:0:(client.c:333:ptlrpc_free_bulk()) kvfree 'desc->bd_vec': 64 at 00000000083715bf. 00000100:00000010:1.0:1788880204.932021:0:203401:0:(client.c:334:ptlrpc_free_bulk()) kfreed 'desc': 984 at 000000008eac1e29. 00000100:00000001:1.0:1788880204.932023:0:203401:0:(client.c:335:ptlrpc_free_bulk()) Process leaving 02000000:00000010:1.0:1788880204.932025:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 0000000005842920. 02000000:00000001:1.0:1788880204.932030:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880204.932031:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880204.932033:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:1.0:1788880204.932036:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880204.932037:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000010:1.0:1788880204.932052:0:203401:0:(mgc_request.c:1651:mgc_process_recover_log()) kvfree 'folios': 32 at 00000000ff4bb2db. 10000000:01000000:1.0:1788880204.932057:0:203401:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'lustre-cliir' succeeded (0). 10000000:00000001:1.0:1788880204.932061:0:203401:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:1.0:1788880204.932063:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880204.932065:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880204.932068:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000000e3a19ac refcount=3 00000020:00000001:1.0:1788880204.932070:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:1.0:1788880204.932073:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:1.0:1788880204.932080:0:203401:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:1.0:1788880204.932084:0:203401:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.932091:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.932093:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:1.0:1788880204.932094:0:203401:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.932101:0:203401:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:1.0:1788880204.932102:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.932103:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:1.0:1788880204.932104:0:203401:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.932106:0:203401:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.932109:0:203401:0:(mgc_request.c:621:do_requeue()) Process leaving 10000000:00000001:1.0:1788880204.932111:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880204.932112:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-client (00000000c2974351) refs 2 10000000:00000001:1.0:1788880204.932114:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.932116:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:1.0:1788880204.932117:0:203401:0:(mgc_request.c:123:config_log_put()) log lustre-cliir(0000000008e74578) refs 3 10000000:00000001:1.0:1788880204.932119:0:203401:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:1.0:1788880204.932121:0:203401:0:(mgc_request.c:601:do_requeue()) Process entered 10000000:01000000:1.0:1788880204.932122:0:203401:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:00000001:1.0:1788880204.932124:0:203401:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:1.0:1788880204.932126:0:203401:0:(mgc_request.c:1801:mgc_process_log()) Process log lustre-client-ffff8ca787527000 from 39 10000000:00000001:1.0:1788880204.932129:0:203401:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:1.0:1788880204.932130:0:203401:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00000100:00000010:1.0:1788880204.932134:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880204.932138:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880204.932142:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880204.932145:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880204.932149:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880204.932153:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 0000000005842920. 00000100:00000001:1.0:1788880204.932159:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.932163:0:203401:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:1.0:1788880204.932166:0:203401:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:1.0:1788880204.932177:0:203401:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at 00000000b9a8b8b1. 00010000:00000001:1.0:1788880204.932182:0:203401:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:1.0:1788880204.932185:0:203401:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 00000000408d8a8f. 00000020:00000001:1.0:1788880204.932190:0:203401:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:1.0:1788880204.932192:0:203401:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000408d8a8f with handle 0x805d548a36a886c8 to hash 00000020:00000001:1.0:1788880204.932194:0:203401:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:1.0:1788880204.932195:0:203401:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880204.932198:0:203401:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880204.932202:0:203401:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1788880204.932205:0:203401:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880204.932208:0:203401:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880204.932214:0:203401:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880204.932223:0:203401:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880204.932228:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880204.932230:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880204.932264:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.932267:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249914662080 : -126823794889536 : ffff8ca788c1acc0) 00000100:00000001:1.0:1788880204.932269:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.932271:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000001:1.0:1788880204.932276:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880204.932278:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.932280:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880204.932281:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880204.932284:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.932292:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880204.932294:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.932301:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880204.932303:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880204.932305:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880204.932308:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780639104:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880204.932322:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880204.932328:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880204.932333:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.932335:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880204.932338:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000008eac1e29. 02000000:00000001:1.0:1788880204.932340:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880204.932344:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000400:00000010:1.0:1788880204.932349:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000200:1.0:1788880204.932354:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780639104, portal 25 00000100:00000001:1.0:1788880204.932356:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.932357:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000040:1.0:1788880204.932360:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.932366:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880204.932369:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880204.932371:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000065ad0e62. 00000100:00000200:1.0:1788880204.932373:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780639104, offset 0 00000400:00000200:1.0:1788880204.932379:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880204.932387:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880204.932399:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880204.932406:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880204.932409:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880204.932412:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880204.932426:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [63:63:256:4294967295] 192.168.204.111@tcp LPNI seq info [63:63:8:4294967295] 00000400:00000200:1.0:1788880204.932435:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880204.932439:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880204.932444:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000800:00000200:1.0:1788880204.932449:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880204.932459:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880204.932461:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 424 niov 1 nkiov 1 00000100:00000001:1.0:1788880204.932477:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.932481:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305841109 00000100:00000001:1.0:1788880204.932488:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305841106 00000100:00000001:1.0:1788880204.932493:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305841109 00000100:00000001:1.0:1788880204.932496:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305841106 00000100:00000001:1.0:1788880204.932499:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305841109 00000100:00000001:1.0:1788880204.932503:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305841106 00000100:00000001:1.0:1788880204.932507:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305841109 00000100:00000001:1.0:1788880204.932511:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305841106 00000100:00000001:1.0:1788880204.932514:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.932517:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.932520:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880204.932521:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880204.932523:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 0000000029c63527 going to sleep for 11 seconds 00000100:00000001:1.0:1788880204.932526:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880204.932528:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880204.932570:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880204.932574:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:0.0:1788880204.932578:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880204.932583:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880204.932586:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000065ad0e62 00000400:00000010:0.0:1788880204.932588:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000065ad0e62. 00000100:00000001:0.0:1788880204.932591:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880204.932595:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.932604:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880204.932606:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.932613:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880204.932615:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880204.933807:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880204.933816:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880204.933819:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880204.933820:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880204.933827:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880204.933839:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b01b80 00000400:00000200:3.0:1788880204.933847:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0x2dd [1] + 224 00000800:00000001:3.0:1788880204.933852:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880204.933857:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880204.933860:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880204.933864:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880204.933869:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880204.933874:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880204.933884:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880204.933905:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880204.933911:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880204.933914:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.933972:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880204.933978:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880204.933981:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880204.933982:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:1.0:1788880204.933986:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:1.0:1788880204.933993:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:1.0:1788880204.933998:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880204.934003:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880204.934014:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.934021:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880204.934023:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.934038:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880204.934040:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880204.934042:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880204.934044:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.934046:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880204.934052:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=32 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880204.934056:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=32 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880204.934059:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880204.934060:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.934062:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880204.934064:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.934169:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880204.934182:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.934191:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.934194:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880204.934196:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880204.934200:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880204.934207:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780639104:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880204.934212:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.934215:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.934217:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880204.934220:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.934222:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.934228:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880204.934230:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.934274:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880204.934276:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.934280:0:203401:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:1.0:1788880204.934283:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880204.934285:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880204.934288:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=4 00000020:00000001:1.0:1788880204.934303:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880204.934306:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000040:1.0:1788880204.934312:0:203401:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 00000000408d8a8f, remote cookie: 0x92f11317f60a7ac2, flags: 0x0 00010000:00000001:1.0:1788880204.934316:0:203401:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:1.0:1788880204.934320:0:203401:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:1.0:1788880204.934323:0:203401:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:1.0:1788880204.934324:0:203401:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:1.0:1788880204.934326:0:203401:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:1.0:1788880204.934327:0:203401:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x0:0x0].0x0 (00000000b9a8b8b1) refcount = 1 00010000:00010000:1.0:1788880204.934332:0:203401:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.934351:0:203401:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:1.0:1788880204.934353:0:203401:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:1.0:1788880204.934354:0:203401:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788880204.934357:0:203401:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:1.0:1788880204.934359:0:203401:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880204.934362:0:203401:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.934368:0:203401:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:1.0:1788880204.934370:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.934371:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880204.934372:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.934373:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880204.934374:0:203401:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.934377:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.934380:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780639104/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.934386:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880204.934388:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880204.934391:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000008eac1e29. 02000000:00000001:1.0:1788880204.934394:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880204.934396:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880204.934398:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880204.934406:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880204.934409:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 0000000005842920. 02000000:00000001:1.0:1788880204.934422:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880204.934424:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880204.934426:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:1.0:1788880204.934429:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880204.934430:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:1.0:1788880204.934432:0:203401:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.934435:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880204.934437:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-client (00000000c2974351) refs 3 10000000:00000001:1.0:1788880204.934439:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.934440:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880204.934442:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880204.934443:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=3 00000020:00000001:1.0:1788880204.934444:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880204.934446:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880204.934448:0:203401:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:1.0:1788880204.934449:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.934450:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880204.934452:0:203401:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.934454:0:203401:0:(mgc_request.c:1668:mgc_process_cfg_log()) Process entered 10000000:00000010:1.0:1788880204.934458:0:203401:0:(mgc_request.c:1682:mgc_process_cfg_log()) kmalloced '(env)': 56 at 00000000083715bf. 00000020:00000010:1.0:1788880204.934461:0:203401:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000005842920. 00000020:00000010:1.0:1788880204.934466:0:203401:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000000c9a81c5. 00000040:00000010:1.0:1788880204.934471:0:203401:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 0000000064b136ca. 00000020:00000010:1.0:1788880204.934476:0:203401:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000a71f1a29. 00000020:00000010:1.0:1788880204.934483:0:203401:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000a21494dd. 80000000:00000010:1.0:1788880204.934487:0:203401:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000cf4d1652. 00040000:00000010:1.0:1788880204.934492:0:203401:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000c9708566. 20000000:00000010:1.0:1788880204.934496:0:203401:0:(mgs_handler.c:1787:mgs_key_init()) kmalloced '(value)': 392 at 000000009cd0104b. 10000000:00000040:1.0:1788880204.934501:0:203401:0:(lustre_log.h:387:llog_ctxt_get()) GETting ctxt 000000006d5d0799 : new refcount 2 00000020:00000001:1.0:1788880204.934504:0:203401:0:(obd_config.c:2120:class_config_parse_llog()) Process entered 00000020:00000040:1.0:1788880204.934505:0:203401:0:(obd_config.c:2122:class_config_parse_llog()) looking up llog lustre-client 00000040:00000001:1.0:1788880204.934508:0:203401:0:(llog.c:1538:llog_open()) Process entered 00000040:00000010:1.0:1788880204.934518:0:203401:0:(llog.c:42:llog_alloc_handle()) kmalloced '(loghandle)': 352 at 00000000aea170a1. 00000040:00000001:1.0:1788880204.934522:0:203401:0:(llog_client.c:70:llog_client_open()) Process entered 00000020:00000040:1.0:1788880204.934533:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788880204.934537:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880204.934540:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880204.934545:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880204.934547:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880204.934551:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880204.934558:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000dc9842d3. 00000100:00000001:1.0:1788880204.934564:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.934570:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880204.934572:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880204.934577:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.934579:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249914662080 : -126823794889536 : ffff8ca788c1acc0) 00000100:00000001:1.0:1788880204.934589:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.934591:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000001:1.0:1788880204.934594:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880204.934597:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.934599:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880204.934600:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880204.934603:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.934610:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880204.934612:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.934615:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880204.934617:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880204.934619:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880204.934622:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780639232:192.168.204.111@tcp:501: 00000100:00000001:1.0:1788880204.934626:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880204.934630:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880204.934633:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.934635:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880204.934639:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000356bd8da. 02000000:00000001:1.0:1788880204.934640:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880204.934646:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000400:00000010:1.0:1788880204.934652:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000200:1.0:1788880204.934663:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780639232, portal 25 00000100:00000001:1.0:1788880204.934665:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.934666:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000040:1.0:1788880204.934671:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.934678:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880204.934681:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880204.934683:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000200:1.0:1788880204.934685:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 512 bytes to portal 26, xid 1875776780639232, offset 0 00000400:00000200:1.0:1788880204.934691:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880204.934701:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880204.934705:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880204.934709:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880204.934713:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880204.934715:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880204.934719:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [64:64:256:4294967295] 192.168.204.111@tcp LPNI seq info [64:64:8:4294967295] 00000400:00000200:1.0:1788880204.934730:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880204.934735:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 512 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880204.934746:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000800:00000200:1.0:1788880204.934754:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880204.934760:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880204.934764:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 608 niov 1 nkiov 1 00000100:00000001:1.0:1788880204.934783:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.934794:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305841111 00000100:00000001:1.0:1788880204.934804:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305841109 00000100:00000001:1.0:1788880204.934809:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305841111 00000100:00000001:1.0:1788880204.934812:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305841109 00000100:00000001:1.0:1788880204.934815:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305841111 00000100:00000001:1.0:1788880204.934818:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305841109 00000100:00000001:1.0:1788880204.934827:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305841111 00000100:00000001:1.0:1788880204.934830:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305841109 00000100:00000001:1.0:1788880204.934832:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.934835:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.934837:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880204.934839:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880204.934842:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 0000000029c63527 going to sleep for 11 seconds 00000100:00000001:1.0:1788880204.934845:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880204.934847:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880204.935064:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880204.935070:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:0.0:1788880204.935075:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880204.935081:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880204.935085:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000400:00000010:0.0:1788880204.935087:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:0.0:1788880204.935092:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880204.935096:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.935106:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880204.935109:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.935116:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880204.935118:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880204.936753:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.936766:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880204.936770:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.936772:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880204.936781:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880204.936795:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 272 into portal 25 MB=0x6aa0250b01c00 00000400:00000200:0.0:1788880204.936814:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 272/272 into md 0x2e5 [1] + 224 00000800:00000001:0.0:1788880204.936822:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.936829:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880204.936832:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880204.936835:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880204.936840:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880204.936845:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880204.936856:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=272 offset=224 replen=272 req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.936879:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880204.936888:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.936891:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.936949:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880204.936959:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880204.936961:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880204.936963:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:1.0:1788880204.936969:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:1.0:1788880204.936971:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:1.0:1788880204.936975:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880204.936983:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880204.936992:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.936999:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880204.937000:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.937003:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880204.937005:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880204.937007:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880204.937009:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.937012:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880204.937018:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=32 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880204.937022:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=32 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880204.937024:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880204.937025:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.937027:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880204.937029:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.937031:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880204.937034:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.937063:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.937066:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880204.937068:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880204.937071:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880204.937078:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780639232:192.168.204.111@tcp:501: 00000100:00000001:1.0:1788880204.937083:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.937085:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.937087:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880204.937089:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.937091:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.937097:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880204.937098:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.937101:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880204.937102:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880204.937106:0:203401:0:(llog_client.c:122:llog_client_open()) Process leaving 00000020:00000001:1.0:1788880204.937109:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880204.937111:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880204.937113:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880204.937114:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.937116:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780639232/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.937122:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880204.937124:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880204.937125:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000356bd8da. 02000000:00000001:1.0:1788880204.937128:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880204.937129:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880204.937129:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880204.937131:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880204.937133:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000dc9842d3. 02000000:00000001:1.0:1788880204.937135:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880204.937140:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880204.937142:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:1.0:1788880204.937145:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880204.937146:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000040:00000001:1.0:1788880204.937148:0:203401:0:(llog.c:1563:llog_open()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880204.937151:0:203401:0:(llog.c:415:llog_init_handle()) Process entered 00000040:00000010:1.0:1788880204.937157:0:203401:0:(llog.c:420:llog_init_handle()) kmalloced '(llh)': 8192 at 00000000807cae5b. 00000040:00000001:1.0:1788880204.937159:0:203401:0:(llog.c:363:llog_read_header()) Process entered 00000040:00000001:1.0:1788880204.937160:0:203401:0:(llog_client.c:269:llog_client_read_header()) Process entered 00000020:00000040:1.0:1788880204.937163:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788880204.937167:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880204.937169:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880204.937171:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880204.937173:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880204.937176:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880204.937179:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000dc9842d3. 00000100:00000001:1.0:1788880204.937183:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.937186:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880204.937187:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880204.937191:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.937192:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249914662080 : -126823794889536 : ffff8ca788c1acc0) 00000100:00000001:1.0:1788880204.937195:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.937196:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000001:1.0:1788880204.937199:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880204.937201:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.937202:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880204.937203:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880204.937206:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.937213:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880204.937215:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.937218:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880204.937220:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880204.937222:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880204.937224:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780639360:192.168.204.111@tcp:503: 00000100:00000001:1.0:1788880204.937265:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880204.937269:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880204.937270:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.937272:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880204.937286:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 16384 at 00000000f43d9f8d. 02000000:00000001:1.0:1788880204.937289:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880204.937293:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000400:00000010:1.0:1788880204.937298:0:203401:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 00000000cc0331d7. 00000100:00000200:1.0:1788880204.937302:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 16384 bytes, xid 1875776780639360, portal 25 00000100:00000001:1.0:1788880204.937304:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.937306:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000040:1.0:1788880204.937310:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.937316:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880204.937319:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880204.937321:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000200:1.0:1788880204.937323:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 272 bytes to portal 26, xid 1875776780639360, offset 0 00000400:00000200:1.0:1788880204.937329:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880204.937355:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880204.937360:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880204.937363:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880204.937367:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880204.937369:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880204.937372:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [65:65:256:4294967295] 192.168.204.111@tcp LPNI seq info [65:65:8:4294967295] 00000400:00000200:1.0:1788880204.937382:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880204.937388:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 272 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880204.937392:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000800:00000200:1.0:1788880204.937397:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880204.937403:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880204.937406:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 368 niov 1 nkiov 1 00000100:00000001:1.0:1788880204.937436:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.937445:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305841114 00000100:00000001:1.0:1788880204.937450:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305841111 00000100:00000001:1.0:1788880204.937455:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305841114 00000100:00000001:1.0:1788880204.937458:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305841111 00000100:00000001:1.0:1788880204.937461:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305841114 00000100:00000001:1.0:1788880204.937464:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305841111 00000100:00000001:1.0:1788880204.937467:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305841114 00000100:00000001:1.0:1788880204.937471:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305841111 00000100:00000001:1.0:1788880204.937474:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.937476:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.937479:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880204.937480:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880204.937483:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 0000000029c63527 going to sleep for 11 seconds 00000100:00000001:1.0:1788880204.937485:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880204.937487:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880204.937568:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880204.937572:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:3.0:1788880204.937576:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880204.937581:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880204.937585:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:3.0:1788880204.937587:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:3.0:1788880204.937596:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880204.937600:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880204.937608:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880204.937611:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880220 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880204.937617:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880204.937619:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880204.938773:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.938786:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880204.938790:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.938792:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880204.938801:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880204.938813:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 8416 into portal 25 MB=0x6aa0250b01c80 00000400:00000200:0.0:1788880204.938821:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 8416/8416 into md 0x2ed [4] + 224 00000800:00000001:0.0:1788880204.938835:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.938848:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880204.938850:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880204.938854:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880204.938858:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880204.938862:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880220 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880204.938869:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=8416 offset=224 replen=8416 req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.938884:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880204.938889:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.938899:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.938937:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880204.938944:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880204.938945:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880204.938947:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:1.0:1788880204.938952:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cc0331d7 00000400:00000010:1.0:1788880204.938953:0:203401:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 00000000cc0331d7. 00000100:00000001:1.0:1788880204.938956:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880204.938962:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880204.938970:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.938976:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880204.938978:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.938980:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880204.938982:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880204.938984:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880204.938985:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.938994:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880204.938999:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=32 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880204.939002:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=32 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880204.939004:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880204.939006:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.939007:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880204.939009:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.939012:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880204.939015:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.939021:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.939023:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880204.939024:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880204.939027:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880220 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880204.939034:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780639360:192.168.204.111@tcp:503: 00000100:00000001:1.0:1788880204.939038:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.939040:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.939042:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880204.939043:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.939045:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880220 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.939050:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880204.939052:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.939054:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880204.939056:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880204.939061:0:203401:0:(llog_client.c:319:llog_client_read_header()) Process leaving 00000100:00000001:1.0:1788880204.939063:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.939065:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780639360/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880220 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.939070:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880204.939072:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880204.939073:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 16384 at 00000000f43d9f8d. 02000000:00000001:1.0:1788880204.939092:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880204.939094:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880204.939095:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880204.939098:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880204.939099:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000dc9842d3. 02000000:00000001:1.0:1788880204.939103:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880204.939104:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880204.939105:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:1.0:1788880204.939108:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880204.939109:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880204.939111:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880204.939112:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880204.939113:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000040:00000001:1.0:1788880204.939115:0:203401:0:(llog.c:403:llog_read_header()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880204.939117:0:203401:0:(llog.c:481:llog_init_handle()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880204.939120:0:203401:0:(llog.c:951:llog_process_or_fork()) Process entered 00000040:00000010:1.0:1788880204.939122:0:203401:0:(llog.c:953:llog_process_or_fork()) kmalloced '(lpi)': 96 at 0000000045cabc93. 00000040:00001000:1.0:1788880204.939124:0:203401:0:(llog.c:968:llog_process_or_fork()) Processing [0xa:0x3:0x0] flags 0x004 startcat -1 startidx -1 first_idx 38 last_idx 0 read_mode 0 00000040:00000001:2.0:1788880204.941040:0:203793:0:(llog.c:552:llog_process_thread()) Process entered 00000020:00000010:2.0:1788880204.941047:0:203793:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000004080df20. 00000020:00000010:2.0:1788880204.941054:0:203793:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000000cc126c4. 00000040:00000010:2.0:1788880204.941058:0:203793:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000a07dd7e3. 00000020:00000010:2.0:1788880204.941063:0:203793:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 000000004322dda6. 00000020:00000010:2.0:1788880204.941069:0:203793:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 00000000ac3b8b67. 00000020:00000010:2.0:1788880204.941074:0:203793:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 0000000088ba1b50. 80000000:00000010:2.0:1788880204.941079:0:203793:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000a3bf26bf. 40000000:00000010:2.0:1788880204.941083:0:203793:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 0000000026dbd873. 00100000:00000010:2.0:1788880204.941097:0:203793:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 0000000071b33f95. 00040000:00000010:2.0:1788880204.941112:0:203793:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 0000000011e1ab42. 00040000:00000010:2.0:1788880204.941117:0:203793:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 00000000214e8c89. 00040000:00000010:2.0:1788880204.941121:0:203793:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 0000000027d4cece. 00000004:00000010:2.0:1788880204.941125:0:203793:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 00000000e43e3cfd. 00000004:00000010:2.0:1788880204.941130:0:203793:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 00000000454d5fcd. 00000004:00000010:2.0:1788880204.941135:0:203793:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 00000000c74d463d. 00002000:00000010:2.0:1788880204.941139:0:203793:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 000000008d40d8dd. 00000004:00000010:2.0:1788880204.941142:0:203793:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 00000000de3147fa. 00000020:00000010:2.0:1788880204.941154:0:203793:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 000000005b8746a9. 00000040:00000010:2.0:1788880204.941165:0:203793:0:(llog.c:578:llog_process_thread()) kmalloced '(buf)': 8192 at 00000000c092b841. 00000040:00001000:2.0:1788880204.941168:0:203793:0:(llog.c:615:llog_process_thread()) index: 39 last_index 64767 00000040:00000001:2.0:1788880204.941170:0:203793:0:(lustre_log.h:475:llog_next_block()) Process entered 00000040:00000001:2.0:1788880204.941172:0:203793:0:(llog_client.c:140:llog_client_next_block()) Process entered 00000020:00000040:2.0:1788880204.941175:0:203793:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:2.0:1788880204.941180:0:203793:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000020:00000040:2.0:1788880204.941183:0:203793:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880204.941186:0:203793:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880204.941189:0:203793:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880204.941192:0:203793:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880204.941197:0:203793:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000003720a685. 00000100:00000001:2.0:1788880204.941202:0:203793:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880204.941206:0:203793:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:2.0:1788880204.941208:0:203793:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:2.0:1788880204.941212:0:203793:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000007d77677. 00000100:00000001:2.0:1788880204.941214:0:203793:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249931840896 : -126823777710720 : ffff8ca789c7cd80) 00000100:00000001:2.0:1788880204.941217:0:203793:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880204.941218:0:203793:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 00000100:00000001:2.0:1788880204.941222:0:203793:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880204.941224:0:203793:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880204.941226:0:203793:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:2.0:1788880204.941228:0:203793:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880204.941252:0:203793:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880204.941261:0:203793:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880204.941267:0:203793:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880204.941271:0:203793:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880204.941274:0:203793:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880204.941276:0:203793:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880204.941279:0:203793:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job llog_process_th:fa4856d5-51a8-499e-b3f8-b477743a3dec:203793:1875776780639488:192.168.204.111@tcp:502: 00000100:00000001:2.0:1788880204.941283:0:203793:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880204.941287:0:203793:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880204.941289:0:203793:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880204.941291:0:203793:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880204.941310:0:203793:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 16384 at 00000000f43d9f8d. 02000000:00000001:2.0:1788880204.941312:0:203793:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880204.941317:0:203793:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000038fe0168. 00000400:00000010:2.0:1788880204.941322:0:203793:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 00000000c21e59e6. 00000100:00000200:2.0:1788880204.941327:0:203793:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 16384 bytes, xid 1875776780639488, portal 25 00000100:00000001:2.0:1788880204.941329:0:203793:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880204.941330:0:203793:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 00000100:00000040:2.0:1788880204.941334:0:203793:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880204.941341:0:203793:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880204.941343:0:203793:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880204.941346:0:203793:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000841b943f. 00000100:00000200:2.0:1788880204.941349:0:203793:0:(niobuf.c:94:ptl_send_buf()) Sending 272 bytes to portal 26, xid 1875776780639488, offset 0 00000400:00000200:2.0:1788880204.941355:0:203793:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880204.941370:0:203793:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880204.941376:0:203793:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880204.941380:0:203793:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880204.941383:0:203793:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880204.941386:0:203793:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880204.941390:0:203793:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [66:66:256:4294967295] 192.168.204.111@tcp LPNI seq info [66:66:8:4294967295] 00000400:00000200:2.0:1788880204.941400:0:203793:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880204.941405:0:203793:0:(socklnd_cb.c:931:ksocknal_send()) sending 272 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880204.941410:0:203793:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 00000800:00000200:2.0:1788880204.941432:0:203793:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880204.941440:0:203793:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880204.941443:0:203793:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 368 niov 1 nkiov 1 00000100:00000001:2.0:1788880204.941463:0:203793:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880204.941468:0:203793:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305841118 00000100:00000001:2.0:1788880204.941475:0:203793:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305841114 00000100:00000001:2.0:1788880204.941480:0:203793:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305841118 00000100:00000001:2.0:1788880204.941482:0:203793:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305841114 00000100:00000001:2.0:1788880204.941485:0:203793:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305841118 00000100:00000001:2.0:1788880204.941487:0:203793:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305841114 00000100:00000001:2.0:1788880204.941490:0:203793:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305841118 00000100:00000001:2.0:1788880204.941494:0:203793:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305841114 00000100:00000001:2.0:1788880204.941497:0:203793:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880204.941500:0:203793:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880204.941502:0:203793:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880204.941504:0:203793:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:2.0:1788880204.941507:0:203793:0:(client.c:2702:ptlrpc_set_wait()) set 0000000007d77677 going to sleep for 11 seconds 00000100:00000001:2.0:1788880204.941516:0:203793:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880204.941518:0:203793:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880204.941586:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880204.941590:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000400:00000200:0.0:1788880204.941595:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880204.941600:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880204.941605:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000841b943f 00000400:00000010:0.0:1788880204.941606:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000841b943f. 00000100:00000001:0.0:1788880204.941609:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880204.941612:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.941618:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880204.941620:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880220 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.941624:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880204.941626:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880204.943312:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880204.943369:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880204.943373:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880204.943376:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880204.943386:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880204.943411:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 8472 into portal 25 MB=0x6aa0250b01d00 00000400:00000200:3.0:1788880204.943430:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 8472/8472 into md 0x2f5 [4] + 224 00000800:00000001:3.0:1788880204.943439:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880204.943466:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880204.943469:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880204.943473:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880204.943478:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880204.943485:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880220 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880204.943493:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=8472 offset=224 replen=8472 req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880204.943510:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880204.943519:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880204.943523:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880204.943558:0:203793:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880204.943566:0:203793:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880204.943568:0:203793:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880204.943570:0:203793:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000038fe0168. 00000400:00000200:2.0:1788880204.943575:0:203793:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c21e59e6 00000400:00000010:2.0:1788880204.943577:0:203793:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 00000000c21e59e6. 00000100:00000001:2.0:1788880204.943581:0:203793:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880204.943587:0:203793:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880204.943597:0:203793:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880204.943609:0:203793:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880204.943621:0:203793:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880204.943624:0:203793:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880204.943626:0:203793:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880204.943628:0:203793:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880204.943631:0:203793:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880204.943633:0:203793:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880204.943639:0:203793:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=32 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880204.943643:0:203793:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=32 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880204.943645:0:203793:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880204.943647:0:203793:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880204.943649:0:203793:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880204.943651:0:203793:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880204.943653:0:203793:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880204.943657:0:203793:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880204.943663:0:203793:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880204.943666:0:203793:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880204.943668:0:203793:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880204.943671:0:203793:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880220 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880204.943678:0:203793:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job llog_process_th:fa4856d5-51a8-499e-b3f8-b477743a3dec:203793:1875776780639488:192.168.204.111@tcp:502: 00000100:00000001:2.0:1788880204.943682:0:203793:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880204.943685:0:203793:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880204.943687:0:203793:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:2.0:1788880204.943688:0:203793:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880204.943691:0:203793:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880220 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880204.943696:0:203793:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880204.943698:0:203793:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000007d77677. 00000100:00000001:2.0:1788880204.943701:0:203793:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:2.0:1788880204.943702:0:203793:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:2.0:1788880204.943710:0:203793:0:(llog_client.c:198:llog_client_next_block()) Process leaving 00000100:00000001:2.0:1788880204.943711:0:203793:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880204.943713:0:203793:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780639488/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880220 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880204.943724:0:203793:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880204.943726:0:203793:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880204.943728:0:203793:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 16384 at 00000000f43d9f8d. 02000000:00000001:2.0:1788880204.943739:0:203793:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880204.943741:0:203793:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880204.943742:0:203793:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880204.943744:0:203793:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880204.943746:0:203793:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000003720a685. 02000000:00000001:2.0:1788880204.943750:0:203793:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880204.943751:0:203793:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880204.943752:0:203793:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000100:00000001:2.0:1788880204.943755:0:203793:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880204.943756:0:203793:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788880204.943758:0:203793:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880204.943768:0:203793:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880204.943770:0:203793:0:(genops.c:1251:class_import_put()) Process leaving 00000040:00000001:2.0:1788880204.943772:0:203793:0:(lustre_log.h:485:llog_next_block()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880204.943774:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2000 type=0x10620000 idx=1 00000040:00001000:2.0:1788880204.943778:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 1 00000040:00001000:2.0:1788880204.943779:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf20e0 type=0x10620000 idx=2 00000040:00001000:2.0:1788880204.943781:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 2 00000040:00001000:2.0:1788880204.943782:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2158 type=0x10620000 idx=3 00000040:00001000:2.0:1788880204.943783:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 3 00000040:00001000:2.0:1788880204.943784:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2200 type=0x10620000 idx=4 00000040:00001000:2.0:1788880204.943785:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 4 00000040:00001000:2.0:1788880204.943786:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf22e0 type=0x10620000 idx=5 00000040:00001000:2.0:1788880204.943788:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 5 00000040:00001000:2.0:1788880204.943789:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf23c0 type=0x10620000 idx=6 00000040:00001000:2.0:1788880204.943790:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 6 00000040:00001000:2.0:1788880204.943792:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2438 type=0x10620000 idx=7 00000040:00001000:2.0:1788880204.943793:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 7 00000040:00001000:2.0:1788880204.943794:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf24e0 type=0x10620000 idx=8 00000040:00001000:2.0:1788880204.943796:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 8 00000040:00001000:2.0:1788880204.943797:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf25c0 type=0x10620000 idx=9 00000040:00001000:2.0:1788880204.943798:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 9 00000040:00001000:2.0:1788880204.943804:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf26a0 type=0x10620000 idx=10 00000040:00001000:2.0:1788880204.943806:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 10 00000040:00001000:2.0:1788880204.943807:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2718 type=0x10620000 idx=11 00000040:00001000:2.0:1788880204.943808:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 11 00000040:00001000:2.0:1788880204.943809:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf27f8 type=0x10620000 idx=12 00000040:00001000:2.0:1788880204.943811:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 12 00000040:00001000:2.0:1788880204.943812:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf28d8 type=0x10620000 idx=13 00000040:00001000:2.0:1788880204.943813:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 13 00000040:00001000:2.0:1788880204.943814:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2930 type=0x10620000 idx=14 00000040:00001000:2.0:1788880204.943815:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 14 00000040:00001000:2.0:1788880204.943816:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf29b0 type=0x10620000 idx=15 00000040:00001000:2.0:1788880204.943818:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 15 00000040:00001000:2.0:1788880204.943819:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2a40 type=0x10620000 idx=16 00000040:00001000:2.0:1788880204.943820:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 16 00000040:00001000:2.0:1788880204.943821:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2ae0 type=0x10620000 idx=17 00000040:00001000:2.0:1788880204.943823:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 17 00000040:00001000:2.0:1788880204.943824:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2bc0 type=0x10620000 idx=18 00000040:00001000:2.0:1788880204.943825:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 18 00000040:00001000:2.0:1788880204.943826:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2ca0 type=0x10620000 idx=19 00000040:00001000:2.0:1788880204.943827:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 19 00000040:00001000:2.0:1788880204.943828:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2cf0 type=0x10620000 idx=20 00000040:00001000:2.0:1788880204.943829:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 20 00000040:00001000:2.0:1788880204.943830:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2dd0 type=0x10620000 idx=21 00000040:00001000:2.0:1788880204.943831:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 21 00000040:00001000:2.0:1788880204.943833:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2eb0 type=0x10620000 idx=22 00000040:00001000:2.0:1788880204.943834:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 22 00000040:00001000:2.0:1788880204.943835:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2f08 type=0x10620000 idx=23 00000040:00001000:2.0:1788880204.943837:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 23 00000040:00001000:2.0:1788880204.943838:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf2f88 type=0x10620000 idx=24 00000040:00001000:2.0:1788880204.943839:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 24 00000040:00001000:2.0:1788880204.943840:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3018 type=0x10620000 idx=25 00000040:00001000:2.0:1788880204.943841:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 25 00000040:00001000:2.0:1788880204.943842:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf30b8 type=0x10620000 idx=26 00000040:00001000:2.0:1788880204.943844:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 26 00000040:00001000:2.0:1788880204.943844:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3198 type=0x10620000 idx=27 00000040:00001000:2.0:1788880204.943850:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 27 00000040:00001000:2.0:1788880204.943851:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3278 type=0x10620000 idx=28 00000040:00001000:2.0:1788880204.943853:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 28 00000040:00001000:2.0:1788880204.943854:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf32c8 type=0x10620000 idx=29 00000040:00001000:2.0:1788880204.943855:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 29 00000040:00001000:2.0:1788880204.943856:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf33a8 type=0x10620000 idx=30 00000040:00001000:2.0:1788880204.943857:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 30 00000040:00001000:2.0:1788880204.943858:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3488 type=0x10620000 idx=31 00000040:00001000:2.0:1788880204.943866:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 31 00000040:00001000:2.0:1788880204.943867:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf34e0 type=0x10620000 idx=32 00000040:00001000:2.0:1788880204.943869:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 32 00000040:00001000:2.0:1788880204.943870:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3560 type=0x10620000 idx=33 00000040:00001000:2.0:1788880204.943871:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 33 00000040:00001000:2.0:1788880204.943872:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf35f0 type=0x10620000 idx=34 00000040:00001000:2.0:1788880204.943874:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 34 00000040:00001000:2.0:1788880204.943875:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3670 type=0x10620000 idx=35 00000040:00001000:2.0:1788880204.943876:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 35 00000040:00001000:2.0:1788880204.943877:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3750 type=0x10620000 idx=36 00000040:00001000:2.0:1788880204.943879:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 36 00000040:00001000:2.0:1788880204.943880:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3830 type=0x10620000 idx=37 00000040:00001000:2.0:1788880204.943881:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 37 00000040:00001000:2.0:1788880204.943882:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3880 type=0x10620000 idx=38 00000040:00001000:2.0:1788880204.943884:0:203793:0:(llog.c:767:llog_process_thread()) skipping lrh_index 38 00000040:00001000:2.0:1788880204.943885:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3960 type=0x10620000 idx=39 00000040:00001000:2.0:1788880204.943887:0:203793:0:(llog.c:784:llog_process_thread()) lrh_index: 39 lrh_len: 224 (1696 remains) 00000040:00001000:2.0:1788880204.943889:0:203793:0:(llog.c:805:llog_process_thread()) index: 39, lh_last_idx: 47 synced_idx: 0 lgh_last_idx: 47 00000020:00000001:2.0:1788880204.943892:0:203793:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000040:00000001:2.0:1788880204.943895:0:203793:0:(llog_swab.c:426:lustre_swab_cfg_marker()) Process entered 00000040:00000001:2.0:1788880204.943896:0:203793:0:(llog_swab.c:469:lustre_swab_cfg_marker()) Process leaving 00000020:01000000:2.0:1788880204.943897:0:203793:0:(obd_config.c:1890:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000001:2.0:1788880204.943903:0:203793:0:(obd_mount.c:953:lustre_check_exclusion()) Process entered 00000020:00000001:2.0:1788880204.943904:0:203793:0:(obd_mount.c:958:lustre_check_exclusion()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880204.943911:0:203793:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 200 at 00000000c54a5c6f. 00000020:00000080:2.0:1788880204.943914:0:203793:0:(obd_config.c:1470:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1788880204.943916:0:203793:0:(obd_config.c:1561:class_process_config()) marker 32 (0x1) lustre-MDT0001 add mdc 00000020:00000001:2.0:1788880204.943922:0:203793:0:(obd_config.c:1562:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880204.943924:0:203793:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 200 at 00000000c54a5c6f. 00000020:00000001:2.0:1788880204.943928:0:203793:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880204.943929:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3a40 type=0x10620000 idx=40 00000040:00001000:2.0:1788880204.943932:0:203793:0:(llog.c:784:llog_process_thread()) lrh_index: 40 lrh_len: 88 (1472 remains) 00000040:00001000:2.0:1788880204.943933:0:203793:0:(llog.c:805:llog_process_thread()) index: 40, lh_last_idx: 47 synced_idx: 0 lgh_last_idx: 47 00000020:00000001:2.0:1788880204.943935:0:203793:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:2.0:1788880204.943937:0:203793:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 64 at 00000000a1e4f73a. 00000020:00000080:2.0:1788880204.943939:0:203793:0:(obd_config.c:1470:class_process_config()) processing cmd: cf005 00000020:00000080:2.0:1788880204.943942:0:203793:0:(obd_config.c:1485:class_process_config()) adding mapping from uuid 192.168.204.111@tcp to nid 0x20000c0a8cc6f (192.168.204.111@tcp) 00000100:00000010:2.0:1788880204.943946:0:203793:0:(lustre_peer.c:90:class_add_uuid()) kmalloced '(data)': 704 at 00000000a5af5020. 00000100:00000040:2.0:1788880204.943951:0:203793:0:(lustre_peer.c:130:class_add_uuid()) found uuid 192.168.204.111@tcp 192.168.204.111@tcp cnt=1 00000400:00000200:2.0:1788880204.943953:0:203793:0:(peer.c:1363:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:2.0:1788880204.943956:0:203793:0:(api-ni.c:3274:LNetNIInit()) refs 1 00000400:00000200:2.0:1788880204.943959:0:203793:0:(peer.c:1377:LNetAddPeer()) Adding NID[0]: 192.168.204.111@tcp 00000400:00000200:2.0:1788880204.943963:0:203793:0:(peer.c:1822:lnet_peer_add()) peer 192.168.204.111@tcp NID flags 0x100001: -17 00000400:00000200:2.0:1788880204.943969:0:203793:0:(peer.c:2890:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:2.0:1788880204.943972:0:203793:0:(peer.c:2899:lnet_discover_peer_locked()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp: 0. pending discovery 00000400:00000200:2.0:1788880204.943975:0:203793:0:(peer.c:1431:LNetAddPeer()) A peer exists for NID 192.168.204.111@tcp 00000100:00000010:2.0:1788880204.943977:0:203793:0:(lustre_peer.c:132:class_add_uuid()) kfreed 'data': 704 at 00000000a5af5020. 00000020:00000001:2.0:1788880204.943979:0:203793:0:(obd_config.c:1500:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:2.0:1788880204.943982:0:203793:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 64 at 00000000a1e4f73a. 00000020:00000001:2.0:1788880204.943984:0:203793:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:2.0:1788880204.943986:0:203793:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca79eaf3a98 type=0x10620000 idx=41 00000040:00001000:2.0:1788880204.943987:0:203793:0:(llog.c:784:llog_process_thread()) lrh_index: 41 lrh_len: 128 (1384 remains) 00000040:00001000:2.0:1788880204.943989:0:203793:0:(llog.c:805:llog_process_thread()) index: 41, lh_last_idx: 47 synced_idx: 0 lgh_last_idx: 47 00000020:00000001:2.0:1788880204.943990:0:203793:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:2.0:1788880204.943993:0:203793:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 39 at 00000000a1e4f73a. 00000020:01000000:2.0:1788880204.943996:0:203793:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf001, instance name: lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000010:2.0:1788880204.943999:0:203793:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 136 at 0000000007d77677. 00000020:00000080:2.0:1788880204.944001:0:203793:0:(obd_config.c:1470:class_process_config()) processing cmd: cf001 00000020:00000001:2.0:1788880204.944002:0:203793:0:(obd_config.c:683:class_attach_name()) Process entered 00000020:00000001:2.0:1788880204.944004:0:203793:0:(genops.c:350:class_newdev()) Process entered 00000020:00000010:2.0:1788880204.944016:0:203793:0:(genops.c:51:obd_device_alloc()) slab-alloced 'obd': 6592 at 00000000afd38811. 00000020:00000080:2.0:1788880204.944021:0:203793:0:(genops.c:413:class_newdev()) Allocate new device lustre-MDT0001-mdc-ffff8ca787527000 (00000000afd38811) 00000020:00000001:2.0:1788880204.944023:0:203793:0:(genops.c:952:__class_new_export()) Process entered 00000020:00000010:2.0:1788880204.944029:0:203793:0:(genops.c:954:__class_new_export()) kmalloced '(export)': 1208 at 00000000b05340da. 00000020:00000001:2.0:1788880204.944032:0:203793:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880204.944034:0:203793:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000b05340da with handle 0x805d548a36a886cf to hash 00000020:00000001:2.0:1788880204.944036:0:203793:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00000020:00000001:2.0:1788880204.944039:0:203793:0:(obd_class.h:1016:obd_init_export()) Process entered 00000020:00000001:2.0:1788880204.944041:0:203793:0:(obd_class.h:1020:obd_init_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880204.944043:0:203793:0:(genops.c:1022:__class_new_export()) Process leaving (rc=18446617249920063488 : -126823789488128 : ffff8ca789141800) 00000020:00000040:2.0:1788880204.944046:0:203793:0:(genops.c:900:class_export_put()) PUTting export 00000000b05340da : new refcount 1 00000020:00000001:2.0:1788880204.944056:0:203793:0:(genops.c:555:class_register_device()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00000020:00000040:2.0:1788880204.944058:0:203793:0:(obd_config.c:1022:class_decref()) Decref lustre-MDT0001-mdc-ffff8ca787527000 (00000000afd38811) now 1 - newdev 00000020:00000040:2.0:1788880204.944061:0:203793:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object 00000000b05340da with handle 0x805d548a36a886cf from hash 00000020:00000040:2.0:1788880204.944064:0:203793:0:(genops.c:900:class_export_put()) PUTting export 00000000b05340da : new refcount 0 00000020:00000080:2.0:1788880204.944065:0:203793:0:(genops.c:906:class_export_put()) final put 00000000b05340da/ea27006f-52a8-4007-b69a-dcac8279445a 00000020:00000001:2.0:1788880204.944068:0:203793:0:(lprocfs_status_server.c:740:lprocfs_exp_cleanup()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880204.944071:0:203793:0:(genops.c:859:class_export_destroy()) Process entered 00000020:00000080:2.0:1788880204.944072:0:203793:0:(genops.c:865:class_export_destroy()) destroying export 00000000b05340da/ea27006f-52a8-4007-b69a-dcac8279445a for lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880204.944074:0:203793:0:(obd_class.h:1025:obd_destroy_export()) Process entered 00000020:00000001:2.0:1788880204.944076:0:203793:0:(obd_class.h:1029:obd_destroy_export()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1788880204.944078:0:203793:0:(genops.c:881:class_export_destroy()) kfreed_rcu 'exp': 1208 at 00000000b05340da. 00000020:00000001:2.0:1788880204.944086:0:203793:0:(genops.c:882:class_export_destroy()) Process leaving 00000020:00000040:2.0:1788880204.944087:0:203793:0:(genops.c:438:class_free_dev()) Release obd device lustre-MDT0001-mdc-ffff8ca787527000 obd_type name = mdc 00000020:01000000:2.0:1788880204.944089:0:203793:0:(genops.c:441:class_free_dev()) finishing cleanup of obd lustre-MDT0001-mdc-ffff8ca787527000 (ea27006f-52a8-4007-b69a-dcac8279445a) 00000020:00000010:2.0:1788880204.944091:0:203793:0:(genops.c:67:obd_device_free()) slab-freed '(obd)': 6592 at 00000000afd38811. 00000020:00000001:2.0:1788880204.944095:0:203793:0:(obd_config.c:715:class_attach_name()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00000020:00000001:2.0:1788880204.944097:0:203793:0:(obd_config.c:1479:class_process_config()) Process leaving via out (rc=18446744073709551599 : -17 : 0xffffffffffffffef) 00000020:00000010:2.0:1788880204.944099:0:203793:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 136 at 0000000007d77677. 00000020:00000010:2.0:1788880204.944102:0:203793:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 39 at 00000000a1e4f73a. 00000020:00020000:2.0:1788880204.944104:0:203793:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.111@tcp: cfg command failed: rc = -17 00000020:00000001:2.0:1788880204.948743:0:203793:0:(obd_config.c:2423:class_config_dump_handler()) Process entered 00000020:00000010:2.0:1788880204.948755:0:203793:0:(obd_config.c:2425:class_config_dump_handler()) kmalloced '(outstr)': 256 at 00000000c54a5c6f. 00000020:00000001:2.0:1788880204.948760:0:203793:0:(obd_config.c:2354:class_config_parse_rec()) Process entered 00000020:00000001:2.0:1788880204.948765:0:203793:0:(obd_config.c:2413:class_config_parse_rec()) Process leaving (rc=62 : 62 : 3e) 00000020:02000400:2.0:1788880204.948767:0:203793:0:(obd_config.c:2431:class_config_dump_handler()) cmd=cf001 0:lustre-MDT0001-mdc 1:mdc 2:lustre-clilmv_UUID 00000020:00000010:2.0:1788880204.952846:0:203793:0:(obd_config.c:2437:class_config_dump_handler()) kfreed 'outstr': 256 at 00000000c54a5c6f. 00000020:00000001:2.0:1788880204.952863:0:203793:0:(obd_config.c:2438:class_config_dump_handler()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880204.952867:0:203793:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00000040:00000001:2.0:1788880204.952871:0:203793:0:(llog.c:843:llog_process_thread()) Process leaving via out (rc=18446744073709551599 : -17 : 0xffffffffffffffef) 00000040:00080000:2.0:1788880204.952875:0:203793:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 41 count 48 00000040:00000010:2.0:1788880204.952878:0:203793:0:(llog.c:899:llog_process_thread()) kvfree 'buf': 8192 at 00000000c092b841. 00000020:00000010:2.0:1788880204.952888:0:203793:0:(lu_object.c:2058:lu_env_remove()) kfreed_rcu 'lei': 40 at 000000005b8746a9. 00000020:00000010:2.0:1788880204.952902:0:203793:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 000000000cc126c4. 00000040:00000010:2.0:1788880204.952907:0:203793:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000a07dd7e3. 00000020:00000010:2.0:1788880204.952910:0:203793:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 000000004322dda6. 00000020:00000010:2.0:1788880204.952919:0:203793:0:(tgt_main.c:741:tgt_key_fini()) slab-freed '(info)': 1288 at 00000000ac3b8b67. 00000020:00000010:2.0:1788880204.952925:0:203793:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 0000000088ba1b50. 80000000:00000010:2.0:1788880204.952930:0:203793:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 00000000a3bf26bf. 40000000:00000010:2.0:1788880204.952935:0:203793:0:(fid_handler.c:479:seq_key_fini()) kfreed 'info': 48 at 0000000026dbd873. 00100000:00000010:2.0:1788880204.952940:0:203793:0:(lfsck_lib.c:43:lfsck_key_fini()) kfreed 'info': 1800 at 0000000071b33f95. 00040000:00000010:2.0:1788880204.952947:0:203793:0:(lquota_lib.c:35:lquota_key_fini()) kfreed 'info': 248 at 0000000011e1ab42. 00040000:00000010:2.0:1788880204.952957:0:203793:0:(qmt_dev.c:418:qmt_key_fini()) kfreed 'info': 1256 at 00000000214e8c89. 00040000:00000010:2.0:1788880204.952961:0:203793:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 0000000027d4cece. 00000004:00000010:2.0:1788880204.952964:0:203793:0:(mdd_device.c:2420:mdd_key_fini()) kfreed 'info': 1872 at 00000000e43e3cfd. 00000004:00000010:2.0:1788880204.952970:0:203793:0:(mdt_handler.c:8495:mdt_key_fini()) kfreed 'info': 2568 at 00000000454d5fcd. 00000004:00000010:2.0:1788880204.952977:0:203793:0:(lod_dev.c:2378:lod_key_fini()) kfreed 'info': 1224 at 00000000c74d463d. 00002000:00000010:2.0:1788880204.952981:0:203793:0:(ofd_dev.c:3002:ofd_key_fini()) kfreed 'info': 816 at 000000008d40d8dd. 00000004:00000010:2.0:1788880204.952984:0:203793:0:(osp_dev.c:1888:osp_key_fini()) kfreed 'info': 616 at 00000000de3147fa. 00000020:00000010:2.0:1788880204.952988:0:203793:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 000000004080df20. 00000040:00000010:1.0:1788880204.953161:0:203401:0:(llog.c:994:llog_process_or_fork()) kfreed 'lpi': 96 at 0000000045cabc93. 00000040:00000001:1.0:1788880204.953169:0:203401:0:(llog.c:995:llog_process_or_fork()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00000020:01000000:1.0:1788880204.953172:0:203401:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 39-41 (rc=-17) 00000040:00000010:1.0:1788880204.953178:0:203401:0:(llog.c:70:llog_free_handle()) kvfree 'loghandle->lgh_hdr': 8192 at 00000000807cae5b. 00000040:00000010:1.0:1788880204.953181:0:203401:0:(llog.c:72:llog_free_handle()) kfreed 'loghandle': 352 at 00000000aea170a1. 00000020:00000001:1.0:1788880204.953184:0:203401:0:(obd_config.c:2151:class_config_parse_llog()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 10000000:00000001:1.0:1788880204.953187:0:203401:0:(mgc_request.c:1714:mgc_process_cfg_log()) Process leaving via out_pop (rc=18446744073709551599 : -17 : 0xffffffffffffffef) 00000020:00000010:1.0:1788880204.953192:0:203401:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 000000000c9a81c5. 00000040:00000010:1.0:1788880204.953196:0:203401:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 0000000064b136ca. 00000020:00000010:1.0:1788880204.953198:0:203401:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 00000000a71f1a29. 00000020:00000010:1.0:1788880204.953203:0:203401:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 00000000a21494dd. 80000000:00000010:1.0:1788880204.953206:0:203401:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 00000000cf4d1652. 00040000:00000010:1.0:1788880204.953208:0:203401:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 00000000c9708566. 20000000:00000010:1.0:1788880204.953211:0:203401:0:(mgs_handler.c:1787:mgs_key_fini()) kfreed 'info': 392 at 000000009cd0104b. 00000020:00000010:1.0:1788880204.953214:0:203401:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000005842920. 10000000:00000010:1.0:1788880204.953217:0:203401:0:(mgc_request.c:1730:mgc_process_cfg_log()) kfreed 'env': 56 at 00000000083715bf. 10000000:01000000:1.0:1788880204.953370:0:203401:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'lustre-client' failed (-17). 10000000:00000001:1.0:1788880204.953373:0:203401:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:1.0:1788880204.953376:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880204.953378:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880204.953380:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=3 00000020:00000001:1.0:1788880204.953382:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880204.953384:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880204.953386:0:203401:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:1.0:1788880204.953389:0:203401:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.953396:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.953397:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:1.0:1788880204.953398:0:203401:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.953403:0:203401:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:1.0:1788880204.953404:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.953405:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:1.0:1788880204.953406:0:203401:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.953408:0:203401:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 10000000:00020000:1.0:1788880204.953410:0:203401:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 10000000:00000001:1.0:1788880204.956946:0:203401:0:(mgc_request.c:621:do_requeue()) Process leaving 10000000:00000001:1.0:1788880204.956951:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880204.956953:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-sptlrpc (000000007b8fd88a) refs 2 10000000:00000001:1.0:1788880204.956957:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.956959:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:1.0:1788880204.956960:0:203401:0:(mgc_request.c:123:config_log_put()) log lustre-client(00000000c2974351) refs 3 10000000:00000001:1.0:1788880204.956963:0:203401:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:1.0:1788880204.956964:0:203401:0:(mgc_request.c:601:do_requeue()) Process entered 10000000:01000000:1.0:1788880204.956966:0:203401:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:00000001:1.0:1788880204.956968:0:203401:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:1.0:1788880204.956969:0:203401:0:(mgc_request.c:1801:mgc_process_log()) Process log lustre-sptlrpc-ffff8ca792d4cd40 from 1 10000000:00000001:1.0:1788880204.956971:0:203401:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:1.0:1788880204.956973:0:203401:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00000010:1.0:1788880204.956981:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880204.956996:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880204.957000:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880204.957002:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880204.957006:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880204.957011:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000009cd0104b. 00000100:00000001:1.0:1788880204.957017:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.957020:0:203401:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:1.0:1788880204.957023:0:203401:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000040:1.0:1788880204.957028:0:203401:0:(ldlm_resource.c:1656:ldlm_resource_getref()) getref res: 00000000b9a8b8b1 count: 2 00010000:00000001:1.0:1788880204.957029:0:203401:0:(ldlm_resource.c:1595:ldlm_resource_get()) Process leaving via found (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788880204.957032:0:203401:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:1.0:1788880204.957034:0:203401:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 00000000d9c80304. 00000020:00000001:1.0:1788880204.957037:0:203401:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:1.0:1788880204.957039:0:203401:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000d9c80304 with handle 0x805d548a36a886d6 to hash 00000020:00000001:1.0:1788880204.957041:0:203401:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:1.0:1788880204.957041:0:203401:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000001:1.0:1788880204.957044:0:203401:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000001:1.0:1788880204.957046:0:203401:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1788880204.957048:0:203401:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880204.957050:0:203401:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880204.957055:0:203401:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880204.957062:0:203401:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880204.957066:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880204.957067:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880204.957073:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.957074:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249914662080 : -126823794889536 : ffff8ca788c1acc0) 00000100:00000001:1.0:1788880204.957076:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.957077:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000001:1.0:1788880204.957085:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880204.957086:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.957088:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880204.957090:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880204.957098:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.957104:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880204.957105:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.957108:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880204.957110:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880204.957111:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880204.957112:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780639616:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880204.957116:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880204.957120:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880204.957122:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.957124:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880204.957126:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000008eac1e29. 02000000:00000001:1.0:1788880204.957128:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880204.957132:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000400:00000010:1.0:1788880204.957137:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:1.0:1788880204.957142:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780639616, portal 25 00000100:00000001:1.0:1788880204.957143:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.957144:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000040:1.0:1788880204.957147:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.957152:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880204.957154:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880204.957155:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000100:00000200:1.0:1788880204.957156:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780639616, offset 0 00000400:00000200:1.0:1788880204.957160:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880204.957172:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880204.957178:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880204.957182:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880204.957191:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880204.957194:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880204.957197:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [67:67:256:4294967295] 192.168.204.111@tcp LPNI seq info [67:67:8:4294967295] 00000400:00000200:1.0:1788880204.957209:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880204.957215:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880204.957220:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000800:00000200:1.0:1788880204.957226:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880204.957270:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880204.957275:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 424 niov 1 nkiov 1 00000100:00000001:1.0:1788880204.957293:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.957298:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305841134 00000100:00000001:1.0:1788880204.957308:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305841118 00000100:00000001:1.0:1788880204.957314:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305841134 00000100:00000001:1.0:1788880204.957319:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305841118 00000100:00000001:1.0:1788880204.957322:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305841134 00000100:00000001:1.0:1788880204.957325:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305841118 00000100:00000001:1.0:1788880204.957328:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305841134 00000100:00000001:1.0:1788880204.957331:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305841118 00000100:00000001:1.0:1788880204.957334:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.957336:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.957338:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880204.957340:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880204.957343:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 0000000029c63527 going to sleep for 11 seconds 00000100:00000001:1.0:1788880204.957346:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880204.957349:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880204.957422:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880204.957427:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:2.0:1788880204.957433:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880204.957438:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880204.957451:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000400:00000010:2.0:1788880204.957453:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00000100:00000001:2.0:1788880204.957458:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880204.957461:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880204.957471:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880204.957474:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880204.957481:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880204.957482:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880204.959566:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.959581:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880204.959592:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.959595:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880204.959606:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880204.959619:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b01d80 00000400:00000200:0.0:1788880204.959628:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0x2fd [1] + 224 00000800:00000001:0.0:1788880204.959634:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.959640:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880204.959643:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880204.959647:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880204.959653:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880204.959659:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880204.959669:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880204.959686:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880204.959708:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880204.959712:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.959739:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880204.959749:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880204.959752:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880204.959754:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:1.0:1788880204.959758:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:1.0:1788880204.959766:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:1.0:1788880204.959770:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880204.959773:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880204.959782:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.959788:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880204.959789:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.959791:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880204.959794:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880204.959796:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880204.959798:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.959801:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880204.959807:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=32 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880204.959811:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=32 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880204.959813:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880204.959815:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.959817:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880204.959819:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.959821:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880204.959825:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.959832:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.959835:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880204.959837:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880204.959840:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880204.959847:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780639616:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880204.959852:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.959854:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.959856:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880204.959862:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.959865:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.959878:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880204.959880:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.959883:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880204.959884:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.959887:0:203401:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:1.0:1788880204.959890:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880204.959892:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880204.959895:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000d9c80304 refcount=4 00000020:00000001:1.0:1788880204.959897:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000001:1.0:1788880204.959900:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000040:1.0:1788880204.959906:0:203401:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 00000000d9c80304, remote cookie: 0x92f11317f60a7ac9, flags: 0x0 00010000:00000001:1.0:1788880204.959909:0:203401:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:1.0:1788880204.959912:0:203401:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:1.0:1788880204.959914:0:203401:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:1.0:1788880204.959915:0:203401:0:(ldlm_lock.c:1006:search_granted_lock()) Process leaving 00010000:00000001:1.0:1788880204.959916:0:203401:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:1.0:1788880204.959918:0:203401:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x0:0x0].0x0 (00000000b9a8b8b1) refcount = 2 00010000:00000040:1.0:1788880204.959921:0:203401:0:(ldlm_resource.c:1879:ldlm_resource_dump()) Granted locks (in reverse order): 00010000:00000040:1.0:1788880204.959923:0:203401:0:(ldlm_resource.c:1882:ldlm_resource_dump()) ### ### ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 1/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880204.959930:0:203401:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.959935:0:203401:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:1.0:1788880204.959937:0:203401:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:1.0:1788880204.959938:0:203401:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788880204.959941:0:203401:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:1.0:1788880204.959943:0:203401:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880204.959945:0:203401:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.959950:0:203401:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:1.0:1788880204.959956:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.959958:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880204.959959:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.959961:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880204.959962:0:203401:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.959964:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.959968:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780639616/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880220 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.959975:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880204.959978:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880204.959980:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000008eac1e29. 02000000:00000001:1.0:1788880204.959983:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880204.959985:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880204.959986:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880204.959988:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880204.959990:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000009cd0104b. 02000000:00000001:1.0:1788880204.959994:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880204.959995:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880204.959996:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:1.0:1788880204.960000:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880204.960001:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:1.0:1788880204.960003:0:203401:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.960005:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880204.960007:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-sptlrpc (000000007b8fd88a) refs 3 10000000:00000001:1.0:1788880204.960009:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880204.960011:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880204.960012:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880204.960013:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000d9c80304 refcount=3 00000020:00000001:1.0:1788880204.960015:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000001:1.0:1788880204.960017:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000001:1.0:1788880204.960019:0:203401:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:1.0:1788880204.960020:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.960021:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880204.960022:0:203401:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.960024:0:203401:0:(mgc_request.c:1668:mgc_process_cfg_log()) Process entered 10000000:00000010:1.0:1788880204.960027:0:203401:0:(mgc_request.c:1682:mgc_process_cfg_log()) kmalloced '(env)': 56 at 00000000083715bf. 00000020:00000010:1.0:1788880204.960031:0:203401:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000009cd0104b. 00000020:00000010:1.0:1788880204.960041:0:203401:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000aea170a1. 00000040:00000010:1.0:1788880204.960044:0:203401:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000dc9842d3. 00000020:00000010:1.0:1788880204.960048:0:203401:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000a71f1a29. 00000020:00000010:1.0:1788880204.960052:0:203401:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000356bd8da. 80000000:00000010:1.0:1788880204.960056:0:203401:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000cf4d1652. 00040000:00000010:1.0:1788880204.960060:0:203401:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000f002f5b3. 20000000:00000010:1.0:1788880204.960064:0:203401:0:(mgs_handler.c:1787:mgs_key_init()) kmalloced '(value)': 392 at 00000000abce1aea. 10000000:00000040:1.0:1788880204.960068:0:203401:0:(lustre_log.h:387:llog_ctxt_get()) GETting ctxt 000000006d5d0799 : new refcount 2 00000020:00000001:1.0:1788880204.960071:0:203401:0:(obd_config.c:2120:class_config_parse_llog()) Process entered 00000020:00000040:1.0:1788880204.960073:0:203401:0:(obd_config.c:2122:class_config_parse_llog()) looking up llog lustre-sptlrpc 00000040:00000001:1.0:1788880204.960076:0:203401:0:(llog.c:1538:llog_open()) Process entered 00000040:00000010:1.0:1788880204.960078:0:203401:0:(llog.c:42:llog_alloc_handle()) kmalloced '(loghandle)': 352 at 00000000cf77ae66. 00000040:00000001:1.0:1788880204.960081:0:203401:0:(llog_client.c:70:llog_client_open()) Process entered 00000020:00000040:1.0:1788880204.960084:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788880204.960088:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880204.960098:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880204.960103:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880204.960105:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880204.960108:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880204.960115:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 0000000030240e3c. 00000100:00000001:1.0:1788880204.960119:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.960124:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880204.960126:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880204.960130:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.960132:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617249914662080 : -126823794889536 : ffff8ca788c1acc0) 00000100:00000001:1.0:1788880204.960134:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.960135:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000001:1.0:1788880204.960139:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880204.960141:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.960142:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880204.960144:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880204.960147:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.960153:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880204.960160:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.960164:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880204.960166:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880204.960169:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880204.960171:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780639744:192.168.204.111@tcp:501: 00000100:00000001:1.0:1788880204.960176:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880204.960184:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880204.960186:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.960188:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880204.960192:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ba022444. 02000000:00000001:1.0:1788880204.960194:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880204.960197:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000400:00000010:1.0:1788880204.960201:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:1.0:1788880204.960206:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780639744, portal 25 00000100:00000001:1.0:1788880204.960207:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880204.960208:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000040:1.0:1788880204.960213:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.960219:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880204.960221:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880204.960223:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000002fa6ff6. 00000100:00000200:1.0:1788880204.960226:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 512 bytes to portal 26, xid 1875776780639744, offset 0 00000400:00000200:1.0:1788880204.960277:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880204.960288:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880204.960294:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880204.960298:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880204.960302:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880204.960306:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880204.960313:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [68:68:256:4294967295] 192.168.204.111@tcp LPNI seq info [68:68:8:4294967295] 00000400:00000200:1.0:1788880204.960322:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880204.960327:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 512 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880204.960332:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000800:00000200:1.0:1788880204.960354:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880204.960361:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880204.960364:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 608 niov 1 nkiov 1 00000100:00000001:1.0:1788880204.960381:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.960384:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305841137 00000100:00000001:1.0:1788880204.960390:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305841134 00000100:00000001:1.0:1788880204.960394:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305841137 00000100:00000001:1.0:1788880204.960405:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305841134 00000100:00000001:1.0:1788880204.960408:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305841137 00000100:00000001:1.0:1788880204.960411:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305841134 00000100:00000001:1.0:1788880204.960422:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305841137 00000100:00000001:1.0:1788880204.960424:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305841134 00000100:00000001:1.0:1788880204.960426:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880204.960429:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880204.960431:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880204.960433:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880204.960434:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 0000000029c63527 going to sleep for 11 seconds 00000100:00000001:1.0:1788880204.960436:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880204.960438:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880204.960641:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880204.960646:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:3.0:1788880204.960691:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880204.960694:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880204.960697:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000002fa6ff6 00000400:00000010:3.0:1788880204.960703:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000002fa6ff6. 00000100:00000001:3.0:1788880204.960705:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880204.960708:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880204.960714:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880204.960715:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880204.960719:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880204.960720:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880204.963641:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880204.963656:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880204.963661:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880204.963663:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880204.963673:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880204.963689:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 272 into portal 25 MB=0x6aa0250b01e00 00000400:00000200:2.0:1788880204.963698:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 272/272 into md 0x305 [1] + 224 00000800:00000001:2.0:1788880204.963704:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880204.963711:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880204.963713:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880204.963718:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880204.963724:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880204.963730:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880204.963741:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=272 offset=224 replen=272 req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880204.963762:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880204.963770:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880204.963773:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.963803:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880204.963810:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880204.963812:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880204.963814:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:1.0:1788880204.963818:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:1.0:1788880204.963820:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:1.0:1788880204.963823:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880204.963828:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880204.963836:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.963848:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880204.963862:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.963865:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880204.963868:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880204.963869:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880204.963871:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880204.963873:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880204.963880:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=32 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880204.963884:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=32 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880204.963886:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:1.0:1788880204.963889:0:203401:0:(client.c:1497:ptlrpc_check_status()) @@@ check status: rc = -2 req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/0 rc 0/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.963896:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000100:00000001:1.0:1788880204.963898:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000100:00000040:1.0:1788880204.963902:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Rpc:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.963909:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551614 : -2 : 0xfffffffffffffffe) 00000100:00000001:1.0:1788880204.963911:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880204.963913:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880204.963916:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Interpret:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880204.963922:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780639744:192.168.204.111@tcp:501: 00000100:00000001:1.0:1788880204.963925:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880204.963927:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000100:00000001:1.0:1788880204.963929:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880204.963930:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.963932:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 2 fl Complete:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.963936:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880204.963937:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000029c63527. 00000100:00000001:1.0:1788880204.963939:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880204.963940:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000040:00000001:1.0:1788880204.963948:0:203401:0:(llog_client.c:114:llog_client_open()) Process leaving via out (rc=18446744073709551614 : -2 : 0xfffffffffffffffe) 00000020:00000001:1.0:1788880204.963951:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880204.963952:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880204.963954:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880204.963956:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880204.963958:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780639744/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880220 ref 1 fl Complete:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880204.963964:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880204.963967:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880204.963969:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ba022444. 02000000:00000001:1.0:1788880204.963972:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880204.963974:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880204.963974:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880204.963976:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880204.963978:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 0000000030240e3c. 02000000:00000001:1.0:1788880204.963981:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880204.963983:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880204.963984:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:1.0:1788880204.963987:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880204.963988:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000040:00001000:1.0:1788880204.963990:0:203401:0:(llog.c:1559:llog_open()) MGC192.168.204.111@tcp: Failed to open llog lustre-sptlrpc: rc -2 00000040:00000010:1.0:1788880204.963994:0:203401:0:(llog.c:72:llog_free_handle()) kfreed 'loghandle': 352 at 00000000cf77ae66. 00000040:00000001:1.0:1788880204.963996:0:203401:0:(llog.c:1563:llog_open()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000020:00000001:1.0:1788880204.964000:0:203401:0:(obd_config.c:2125:class_config_parse_llog()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 10000000:00000001:1.0:1788880204.964003:0:203401:0:(mgc_request.c:1714:mgc_process_cfg_log()) Process leaving via out_pop (rc=18446744073709551614 : -2 : 0xfffffffffffffffe) 00000020:00000010:1.0:1788880204.964009:0:203401:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000aea170a1. 00000040:00000010:1.0:1788880204.964012:0:203401:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000dc9842d3. 00000020:00000010:1.0:1788880204.964015:0:203401:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 00000000a71f1a29. 00000020:00000010:1.0:1788880204.964019:0:203401:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 00000000356bd8da. 80000000:00000010:1.0:1788880204.964021:0:203401:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 00000000cf4d1652. 00040000:00000010:1.0:1788880204.964025:0:203401:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 00000000f002f5b3. 20000000:00000010:1.0:1788880204.964028:0:203401:0:(mgs_handler.c:1787:mgs_key_fini()) kfreed 'info': 392 at 00000000abce1aea. 00000020:00000010:1.0:1788880204.964030:0:203401:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 000000009cd0104b. 10000000:00000010:1.0:1788880204.964032:0:203401:0:(mgc_request.c:1730:mgc_process_cfg_log()) kfreed 'env': 56 at 00000000083715bf. 10000000:01000000:1.0:1788880204.964046:0:203401:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 10000000:00000001:1.0:1788880204.964048:0:203401:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:1.0:1788880204.964050:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880204.964054:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880204.964057:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000d9c80304 refcount=3 00000020:00000001:1.0:1788880204.964059:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000001:1.0:1788880204.964062:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000001:1.0:1788880204.964064:0:203401:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:1.0:1788880204.964067:0:203401:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.964076:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.964078:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:1.0:1788880204.964079:0:203401:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880204.964085:0:203401:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:1.0:1788880204.964086:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880204.964087:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:1.0:1788880204.964088:0:203401:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880204.964090:0:203401:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 10000000:00000001:1.0:1788880204.964093:0:203401:0:(mgc_request.c:621:do_requeue()) Process leaving 10000000:00000001:1.0:1788880204.964096:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:1.0:1788880204.964097:0:203401:0:(mgc_request.c:123:config_log_put()) log lustre-sptlrpc(000000007b8fd88a) refs 3 10000000:00000001:1.0:1788880204.964099:0:203401:0:(mgc_request.c:147:config_log_put()) Process leaving 00010000:00000001:1.0:1788880205.502336:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880205.502343:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880205.502349:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880205.502350:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880205.502383:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880205.502386:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880205.502388:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880206.526326:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880206.526332:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880206.526338:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880206.526339:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880206.526410:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880206.526414:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880206.526417:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880207.550334:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880207.550342:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880207.550374:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880207.550375:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880207.550416:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880207.550419:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880207.550422:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880208.511311:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880208.511319:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880208.511324:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000007f91037. 00000020:00000040:0.0:1788880208.511330:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880208.511335:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880208.511339:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880208.511344:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880208.511363:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f92e72a2. 00000100:00000001:0.0:1788880208.511371:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880208.511381:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880208.511390:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880208.511392:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880208.511399:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000007f91037] to pc [ptlrpcd_00_00+0] req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880208.511428:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880208.511431:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880208.511435:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880208.511438:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000a2475c3c. 00000020:00000040:0.0:1788880208.511441:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880208.511444:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880208.511445:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880208.511447:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880208.511449:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880208.511451:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:0.0:1788880208.511452:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000446e4ede. 00000100:00000001:3.0:1788880208.511454:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880208.511455:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880208.511456:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880208.511461:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880208.511462:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880208.511463:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880208.511485:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880208.511488:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880208.511491:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880208.511495:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880208.511498:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:1.0:1788880208.511499:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880208.511500:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880208.511504:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780639872:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880208.511508:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a2475c3c] to pc [ptlrpcd_00_01+1] req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880208.511509:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880208.511516:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880208.511518:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880208.511520:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880208.511522:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880208.511525:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:1.0:1788880208.511525:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f002f5b3. 02000000:00000001:1.0:1788880208.511529:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880208.511532:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000400:00000010:1.0:1788880208.511535:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000100:00000001:3.0:1788880208.511536:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:0.0:1788880208.511537:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000802a821. 00000100:00000001:3.0:1788880208.511541:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:0.0:1788880208.511542:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000010:1.0:1788880208.511543:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000001:3.0:1788880208.511545:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880208.511548:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880208.511551:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000200:1.0:1788880208.511552:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780639872, portal 25 00000100:00000040:3.0:1788880208.511553:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880208.511555:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880208.511556:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880208.511558:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849371264 : -126822860180352 : ffff8ca7c0783480) 02000000:00000010:0.0:1788880208.511563:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b899fe8d. 00000100:00000040:1.0:1788880208.511566:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880224 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880208.511570:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880208.511571:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880208.511573:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880208.511579:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880208.511580:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:0.0:1788880208.511582:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:3.0:1788880208.511584:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880208.511586:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880208.511589:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880208.511593:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 00000100:00100000:3.0:1788880208.511597:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780640000:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880208.511599:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780639872, offset 0 00000100:00000001:0.0:1788880208.511606:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880208.511607:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880208.511610:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880208.511612:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1788880208.511615:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880208.511615:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880208.511617:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000802a821] to pc [ptlrpcd_00_02+2] req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880208.511621:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880208.511628:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e6a93e02. 02000000:00000001:3.0:1788880208.511632:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880208.511637:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880208.511640:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880208.511643:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880208.511644:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ba5e1826. 00000100:00000001:0.0:1788880208.511645:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000400:00000200:1.0:1788880208.511647:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880208.511649:0:53635:0:(import.c:2066:ptlrpc_disconnect_and_idle_import()) Process entered 00000400:00000010:3.0:1788880208.511652:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000100:00000001:0.0:1788880208.511653:0:53635:0:(import.c:1762:ptlrpc_disconnect_prep_req()) Process entered 00000100:00000010:0.0:1788880208.511657:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b8846eac. 00000100:00000200:3.0:1788880208.511659:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780640000, portal 10 00000020:00000040:0.0:1788880208.511662:0:53635:0:(genops.c:1231:class_import_get()) import 00000000beb50720 refcount=3 obd=lustre-OST0000-osc-ffff8ca787527000 00000400:00000200:1.0:1788880208.511663:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880208.511664:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880208.511668:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849361408 : -126822860190208 : ffff8ca7c0780e00) 00000100:00000001:0.0:1788880208.511669:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880208.511673:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000200:1.0:1788880208.511673:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880208.511679:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880208.511680:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788880208.511684:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880208.511691:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [69:69:256:4294967295] 192.168.204.111@tcp LPNI seq info [69:69:8:4294967295] 02000000:00000010:0.0:1788880208.511693:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000088542ca3. 00000100:00000001:3.0:1788880208.511697:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880208.511701:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880208.511703:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880208.511707:0:53635:0:(import.c:1800:ptlrpc_disconnect_prep_req()) Process leaving (rc=18446617250849364096 : -126822860187520 : ffff8ca7c0781880) 00000400:00000010:3.0:1788880208.511709:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000003539bb5a. 00000400:00000200:1.0:1788880208.511711:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00080000:0.0:1788880208.511715:0:53635:0:(import.c:65:import_set_state_nolock()) 00000000beb50720 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00000200:3.0:1788880208.511715:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780640000, offset 0 00000800:00000200:1.0:1788880208.511722:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00080000:0.0:1788880208.511723:0:53635:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff8ca787527000: disconnect after 24s idle 00000800:00000010:1.0:1788880208.511727:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:0.0:1788880208.511729:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880208.511732:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880208.511737:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:0.0:1788880208.511739:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b8846eac] to pc [ptlrpcd_rcv+-1] req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880208.511742:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880208.511752:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880208.511753:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880208.511760:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880208.511767:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880208.511768:0:53635:0:(import.c:2105:ptlrpc_disconnect_and_idle_import()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880208.511772:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880208.511773:0:53635:0:(pinger.c:123:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880208.511778:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880208.511778:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:0.0:1788880208.511780:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11177) 00000100:00000001:1.0:1788880208.511786:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305844688 00000400:00000200:3.0:1788880208.511794:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880208.511797:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305841137 00000400:00000200:3.0:1788880208.511800:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [70:70:255:4294967295] 192.168.204.111@tcp LPNI seq info [70:70:7:4294967295] 00000100:00000001:1.0:1788880208.511805:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305844688 00000100:00000001:0.0:1788880208.511808:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788880208.511813:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:1.0:1788880208.511814:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305841137 00000100:00000001:0.0:1788880208.511817:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880208.511818:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880208.511821:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305844688 00000100:00000001:1.0:1788880208.511828:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305841137 00000100:00000001:0.0:1788880208.511830:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:3.0:1788880208.511830:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880208.511832:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305844688 00000100:00000001:0.0:1788880208.511834:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880208.511837:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305841137 00000800:00000010:3.0:1788880208.511838:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000055d7d92. 00000100:00000001:0.0:1788880208.511839:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880208.511843:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880208.511843:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880208.511847:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880208.511849:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880208.511849:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880208.511853:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880208.511854:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880208.511856:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880208.511862:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:3.0:1788880208.511862:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880208.511864:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:3.0:1788880208.511869:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000055d7d92 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880208.511870:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880208.511873:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880208.511875:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880208.511878:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880208.511878:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880208.511878:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880208.511882:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880208.511886:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1788880208.511886:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1813823 usecs (1814 jiffies) 02000000:00000001:0.0:1788880208.511890:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880208.511892:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305841431 02000000:00000001:0.0:1788880208.511895:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880208.511897:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305844688 00000100:00100000:0.0:1788880208.511901:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780640128:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880208.511903:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305841431 00000100:00000001:3.0:1788880208.511908:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305844688 00000100:00000001:0.0:1788880208.511910:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880208.511912:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880208.511914:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880208.511919:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880208.511919:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305841431 00000100:00000001:3.0:1788880208.511923:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305844688 00000100:00000001:3.0:1788880208.511926:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305841431 00000800:00000200:1.0:1788880208.511930:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880208.511931:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305844688 02000000:00000010:0.0:1788880208.511932:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e6ca1161. 00000100:00000001:3.0:1788880208.511934:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880208.511936:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880208.511941:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000100:00000001:3.0:1788880208.511941:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880208.511945:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000027964113. 00000400:00000200:1.0:1788880208.511948:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880208.511948:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880208.511951:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000100:00000001:3.0:1788880208.511952:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880208.511957:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780640128, portal 10 00000400:00000200:1.0:1788880208.511959:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880208.511959:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880208.511960:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880208.511963:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849370368 : -126822860181248 : ffff8ca7c0783100) 00000100:00000001:3.0:1788880208.511964:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880208.511968:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880208.511970:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000100:00000040:0.0:1788880208.511971:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880208.511971:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:1.0:1788880208.511974:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000001:3.0:1788880208.511975:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880208.511982:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880208.511983:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880208.511986:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880208.511992:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000200:1.0:1788880208.511993:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880224 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880208.511997:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000001:3.0:1788880208.511997:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880208.512002:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:0.0:1788880208.512004:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780640128, offset 0 00000100:00000001:1.0:1788880208.512005:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880208.512006:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880208.512011:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880224 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880208.512015:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880208.512017:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880208.512027:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880208.512032:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880208.512036:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880208.512038:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:0.0:1788880208.512043:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880208.512043:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880208.512046:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880208.512049:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880208.512051:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788880208.512055:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880208.512056:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880208.512062:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [71:71:254:4294967295] 192.168.204.111@tcp LPNI seq info [71:71:6:4294967295] 02000000:00000001:3.0:1788880208.512062:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880208.512068:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780640256:192.168.204.111@tcp:9: 00000100:00000001:3.0:1788880208.512076:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880208.512081:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:3.0:1788880208.512081:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880208.512085:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880208.512088:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:0.0:1788880208.512091:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000010:3.0:1788880208.512095:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b405d9c4. 00000800:00000010:0.0:1788880208.512097:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000088d6bc1. 02000000:00000001:3.0:1788880208.512099:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880208.512105:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:3.0:1788880208.512106:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c948c598. 00000400:00000010:3.0:1788880208.512113:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000800:00000200:0.0:1788880208.512114:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880208.512118:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000088d6bc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880208.512124:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880208.512126:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780640256, portal 4 00000100:00000040:0.0:1788880208.512127:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 131025 usecs (132 jiffies) 00000100:00000001:3.0:1788880208.512130:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880208.512131:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305839749 00000100:00000001:3.0:1788880208.512133:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364096 : -126822860187520 : ffff8ca7c0781880) 00000100:00000001:0.0:1788880208.512135:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305844688 00000100:00000001:0.0:1788880208.512139:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305839749 00000100:00000040:3.0:1788880208.512146:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880208.512147:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305844688 00000100:00000001:0.0:1788880208.512152:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305839749 00000100:00000001:0.0:1788880208.512156:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305844688 00000100:00000001:0.0:1788880208.512159:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305839749 00000100:00000001:3.0:1788880208.512162:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880208.512163:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305844688 00000800:00000200:1.0:1788880208.512164:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880208.512166:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:1.0:1788880208.512169:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000055d7d92. 00000100:00000040:3.0:1788880208.512170:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880208.512171:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880208.512176:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880208.512177:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:3.0:1788880208.512178:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 00000100:00000001:0.0:1788880208.512179:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880208.512185:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875776780640256, offset 0 00000400:00000200:1.0:1788880208.512186:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880208.512196:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880208.512200:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000003539bb5a 00000400:00000010:1.0:1788880208.512202:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000003539bb5a. 00000100:00000001:1.0:1788880208.512206:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:3.0:1788880208.512206:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:1.0:1788880208.512211:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880208.512212:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880208.512220:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880208.512222:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880208.512227:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880208.512227:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880208.512250:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880208.512251:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880208.512268:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880208.512271:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880208.512323:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880208.512329:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [72:72:254:4294967295] 192.168.204.111@tcp LPNI seq info [72:72:6:4294967295] 00000400:00000200:3.0:1788880208.512339:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880208.512356:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880208.512362:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000800:00000200:3.0:1788880208.512368:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880208.512373:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880208.512375:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880208.512379:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880208.512382:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: using a timeout of 3993 usecs (4 jiffies) 00000100:00000001:3.0:1788880208.512385:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305839622 00000100:00000001:3.0:1788880208.512387:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305844688 00000100:00000001:3.0:1788880208.512389:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305839622 00000800:00000200:1.0:1788880208.512392:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880208.512392:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305844688 00000800:00000010:1.0:1788880208.512397:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000088d6bc1. 00000100:00000001:3.0:1788880208.512397:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305839622 00000400:00000200:1.0:1788880208.512403:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880208.512410:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305844688 00000400:00000200:1.0:1788880208.512411:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880208.512416:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305839622 00000400:00000200:1.0:1788880208.512417:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:1.0:1788880208.512422:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:3.0:1788880208.512422:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305844688 00000100:00000001:3.0:1788880208.512426:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880208.512429:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880208.512435:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880208.512437:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880208.512438:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880208.512444:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880208.512445:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880208.512454:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880208.512457:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880208.512463:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880208.512465:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880208.512481:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880208.512483:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000400:00000200:1.0:1788880208.512487:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880208.512491:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880208.512494:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 00000400:00000010:1.0:1788880208.512496:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 00000100:00000001:1.0:1788880208.512499:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880208.512503:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880208.512511:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880208.512513:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880208.512519:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880208.512521:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880208.512557:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:3.0:1788880208.512559:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880208.512560:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880208.512561:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880208.512563:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880208.513676:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880208.513687:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880208.513690:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880208.513693:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880208.513706:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880208.513839:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b01e80 00000400:00000200:3.0:1788880208.513855:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x30d [1] + 224 00000800:00000001:3.0:1788880208.513867:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880208.513883:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880208.513886:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880208.513891:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880208.513899:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880208.513907:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880208.513920:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880208.513941:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880208.513956:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880208.513963:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880208.514000:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880208.514006:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880208.514013:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880208.514016:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880208.514020:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:1.0:1788880208.514025:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:1.0:1788880208.514027:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:1.0:1788880208.514032:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880208.514037:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880208.514048:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880208.514063:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880208.514065:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880208.514067:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880208.514069:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880208.514071:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880208.514073:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880208.514075:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880208.514081:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=36 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880208.514084:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=36 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880208.514086:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880208.514087:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880208.514088:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880208.514097:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880208.514099:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880208.514107:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880208.514114:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880208.514116:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880208.514119:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880208.514123:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880208.514130:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780639872:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880208.514136:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880208.514139:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880208.514144:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000007f91037 x1875776780639872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880208.514157:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880208.514159:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880208.514163:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f002f5b3. 02000000:00000001:1.0:1788880208.514167:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880208.514169:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880208.514171:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880208.514178:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880208.514181:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f92e72a2. 02000000:00000001:1.0:1788880208.514184:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880208.514186:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880208.514187:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000007f91037. 00000100:00000001:1.0:1788880208.514190:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880208.514192:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880208.514194:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880208.514201:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880208.514203:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880208.514204:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880208.514206:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880208.514330:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880208.514341:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880208.514344:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880208.514366:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880208.514375:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880208.514385:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b01f00 00000400:00000200:2.0:1788880208.514393:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x315 [1] + 224 00000800:00000001:2.0:1788880208.514399:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880208.514404:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880208.514407:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880208.514411:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880208.514416:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880208.514421:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880208.514431:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880208.514445:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880208.514453:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880208.514457:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880208.514460:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880208.514464:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880208.514471:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880208.514479:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b01f80 00000400:00000200:2.0:1788880208.514485:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x31d [1] + 224 00000800:00000001:2.0:1788880208.514490:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880208.514511:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880208.514514:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880208.514520:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880208.514523:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880208.514523:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880208.514530:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880208.514530:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880208.514537:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880208.514543:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:2.0:1788880208.514547:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880208.514548:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ba5e1826. 00000400:00000200:3.0:1788880208.514556:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000400:00000010:3.0:1788880208.514561:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:2.0:1788880208.514589:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880208.514605:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880208.514613:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880208.514614:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880208.514620:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880208.514628:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880208.514631:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880208.514633:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880208.514638:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:3.0:1788880208.514644:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880208.514647:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880208.514649:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:0.0:1788880208.514651:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880208.514666:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000027964113. 00000400:00000200:2.0:1788880208.514672:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 4 MB=0x6aa0250b02000 00000400:00000200:0.0:1788880208.514676:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000400:00000010:0.0:1788880208.514684:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:3.0:1788880208.514688:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880208.514691:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880208.514694:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880208.514697:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.204.111@tcp of length 224/224 into md 0x325 [1] + 224 00000100:00000200:0.0:1788880208.514701:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880208.514702:0:201286:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788880208.514705:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788880208.514706:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880208.514712:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880208.514716:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880208.514720:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880208.514722:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880208.514724:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880208.514730:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00001000:3.0:1788880208.514732:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=26 v=5 (5 0 0 0) 00000400:00000200:2.0:1788880208.514745:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880208.514751:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:3.0:1788880208.514752:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=26 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880208.514754:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880208.514756:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880208.514762:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880208.514765:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880208.514766:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880208.514771:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880208.514772:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880208.514778:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00010000:00000001:3.0:1788880208.514785:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:2.0:1788880208.514789:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880208.514789:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880208.514796:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880208.514804:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:0.0:1788880208.514805:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880208.514807:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880208.514814:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880208.514815:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880208.514816:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880208.514818:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=36 v=5 (5 0 0 0) 00000800:00000001:2.0:1788880208.514825:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880208.514827:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=10 v=5 (1 0 0 0) 00000100:00000040:3.0:1788880208.514828:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880208.514831:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880208.514838:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880208.514844:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880208.514847:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880208.514849:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880208.514851:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880208.514852:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:0.0:1788880208.514854:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880208.514857:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880208.514859:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00010000:00080000:0.0:1788880208.514863:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788880208.514864:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880208.514868:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c948c598. 00000100:00000040:3.0:1788880208.514868:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880208.514874:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00010000:00000001:0.0:1788880208.514882:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880208.514882:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000100:00000001:2.0:1788880208.514896:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880208.514897:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880208.514903:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00100000:3.0:1788880208.514903:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780640000:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880208.514914:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880208.514915:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880208.514918:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880208.514920:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880208.514921:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880208.514930:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000a2475c3c x1875776780640000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880208.514934:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880208.514984:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880208.514986:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880208.514990:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880208.514994:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:3.0:1788880208.514996:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880208.515000:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1788880208.515002:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e6a93e02. 02000000:00000001:3.0:1788880208.515015:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:0.0:1788880208.515016:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880208.515017:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:3.0:1788880208.515020:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880208.515021:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:3.0:1788880208.515025:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880208.515028:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880208.515034:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:3.0:1788880208.515036:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880208.515038:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000010:3.0:1788880208.515040:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000446e4ede. 00000100:00000001:2.0:1788880208.515042:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880208.515052:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880208.515055:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00100000:0.0:1788880208.515057:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780640128:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880208.515058:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00001000:2.0:1788880208.515059:0:201284:0:(import.c:2149:obd_at_measure()) add 5 to 0000000040ba1a3b time=26 v=5 (5 0 0 0) 00000100:00000010:3.0:1788880208.515061:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000a2475c3c. 00000100:00000001:3.0:1788880208.515073:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00001000:2.0:1788880208.515074:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 000000003184a8ce time=26 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880208.515075:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880208.515076:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880208.515083:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880208.515087:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880208.515090:0:201284:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880208.515091:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880208.515093:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:2.0:1788880208.515095:0:201284:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880208.515096:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880208.515100:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000802a821 x1875776780640128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880224 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880208.515106:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00080000:2.0:1788880208.515111:0:201284:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880208.515112:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880208.515131:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00010000:00000001:2.0:1788880208.515133:0:201284:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880208.515136:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880208.515139:0:201284:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000010:0.0:1788880208.515140:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e6ca1161. 00000100:00000040:2.0:1788880208.515142:0:201284:0:(client.c:3108:ptlrpc_free_committed()) lustre-OST0000-osc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880208.515145:0:201284:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000001:0.0:1788880208.515146:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880208.515147:0:201284:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880208.515148:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880208.515151:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:2.0:1788880208.515153:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880208.515154:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880208.515157:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b899fe8d. 02000000:00000001:0.0:1788880208.515164:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880208.515165:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880208.515167:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880208.515169:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000802a821. 00000100:00000001:2.0:1788880208.515169:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880208.515171:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00080000:2.0:1788880208.515175:0:201284:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880208.515178:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00080000:2.0:1788880208.515185:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000beb50720 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00000001:0.0:1788880208.515187:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880208.515191:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880208.515191:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880208.515199:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880208.515202:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880208.515202:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780640256:192.168.204.111@tcp:9: 00000100:00000001:0.0:1788880208.515205:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880208.515206:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880208.515215:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880208.515218:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880208.515221:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b8846eac x1875776780640256/t0(0) o9->lustre-OST0000-osc-ffff8ca787527000@192.168.204.111@tcp:28/4 lens 224/224 e 0 to 0 dl 1788880218 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880208.515226:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880208.515228:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880208.515230:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b405d9c4. 02000000:00000001:2.0:1788880208.515258:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880208.515260:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880208.515262:0:201284:0:(genops.c:1244:class_import_put()) import 00000000beb50720 refcount=2 obd=lustre-OST0000-osc-ffff8ca787527000 00000020:00000001:2.0:1788880208.515265:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880208.515267:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000088542ca3. 02000000:00000001:2.0:1788880208.515271:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880208.515274:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880208.515275:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b8846eac. 00000100:00000001:2.0:1788880208.515278:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880208.515279:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880208.515281:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880208.515288:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880208.515290:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880208.515293:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880208.515295:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880208.574396:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880208.574407:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880208.574416:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880208.574417:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880208.574486:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880208.574507:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880208.574514:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880209.598395:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880209.598406:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880209.598414:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880209.598415:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880209.598460:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880209.598465:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880209.598469:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880210.622368:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880210.622376:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880210.622382:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880210.622383:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880210.622482:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880210.622485:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880210.622488:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880211.646383:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880211.646389:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880211.646399:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880211.646400:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880211.646747:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880211.646751:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880211.646754:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880212.670388:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880212.670399:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880212.670408:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880212.670410:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880212.670492:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880212.670499:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880212.670503:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00000001:1.0:1788880213.118384:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000001:1.0:1788880213.118392:0:203401:0:(mgc_request.c:120:config_log_put()) Process leaving 00000100:00000040:2.0:1788880213.630333:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305844688, at 4305844736 00000100:00000040:0.0:1788880213.630335:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305844688, at 4305844736 00000100:00000010:2.0:1788880213.630353:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000059817523. 00000100:00000010:0.0:1788880213.630357:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009f22a9a0. 00000100:00000040:2.0:1788880213.630384:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880213.630390:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000040:3.0:1788880213.630390:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305844688, at 4305844736 00000100:00000040:1.0:1788880213.630393:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305844688, at 4305844736 00000100:00000010:2.0:1788880213.630396:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000100:00000010:3.0:1788880213.630398:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c0bfe00. 00000020:00000040:2.0:1788880213.630401:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880213.630411:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000010:1.0:1788880213.630416:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005b8bebe2. 02000000:00000001:2.0:1788880213.630416:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880213.630425:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880213.630436:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:2.0:1788880213.630446:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880213.630467:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880213.630492:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880213.630495:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880213.630502:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_03+3] req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880213.630531:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880213.630534:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880213.630538:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880213.630541:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000020:00000040:2.0:1788880213.630545:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880213.630547:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880213.630548:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880213.630550:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880213.630553:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c54a5c6f. 00000100:00000001:2.0:1788880213.630556:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880213.630558:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880213.630559:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880213.630565:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880213.630567:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880213.630569:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880213.630572:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_00+0] req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880213.630575:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880213.630584:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880213.630588:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880213.630593:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880213.630597:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880213.630598:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880213.630600:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880213.630605:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880213.630607:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880213.630612:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880213.630617:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880213.630620:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880213.630623:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788880213.630624:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780640384:192.168.204.111@tcp:400: 00000100:00000010:2.0:1788880213.630625:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 00000100:00000001:1.0:1788880213.630630:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880213.630631:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880213.630633:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880213.630637:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880213.630639:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880213.630641:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880213.630642:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880213.630643:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000010:2.0:1788880213.630645:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 02000000:00000001:0.0:1788880213.630647:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880213.630654:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880213.630659:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880213.630661:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880213.630665:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:0.0:1788880213.630668:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a2f4800f. 00000100:00000001:2.0:1788880213.630674:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788880213.630675:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880213.630676:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880213.630682:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_01+1] req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880213.630687:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880213.630696:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000027964113. 00000100:00000001:2.0:1788880213.630699:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880213.630701:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:0.0:1788880213.630706:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000100:00000040:2.0:1788880213.630710:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000200:0.0:1788880213.630721:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780640384, portal 25 00000100:00000040:2.0:1788880213.630721:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11182) 00000100:00000001:1.0:1788880213.630729:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880213.630733:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880213.630736:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1788880213.630740:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880213.630742:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 02000000:00000001:1.0:1788880213.630746:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880213.630750:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880213.630752:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880229 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880213.630752:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880213.630757:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880213.630762:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00100000:1.0:1788880213.630769:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780640512:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880213.630770:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880213.630772:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880213.630780:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880213.630784:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880213.630792:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a09430ba. 00000100:00000001:1.0:1788880213.630793:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000200:0.0:1788880213.630801:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780640384, offset 0 00000100:00000001:3.0:1788880213.630803:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:1.0:1788880213.630805:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880213.630807:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880213.630809:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880213.630812:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788880213.630813:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880213.630817:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880213.630818:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:3.0:1788880213.630823:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880213.630830:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000123d40c9. 00000100:00100000:3.0:1788880213.630835:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780640640:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880213.630837:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788880213.630838:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880213.630852:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880213.630853:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:1.0:1788880213.630854:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 02000000:00000001:3.0:1788880213.630857:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880213.630860:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880213.630863:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880213.630865:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:1.0:1788880213.630866:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 02000000:00000010:3.0:1788880213.630871:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000005842920. 00000400:00000200:0.0:1788880213.630872:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880213.630875:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880213.630877:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780640512, portal 10 00000400:00000200:0.0:1788880213.630879:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:3.0:1788880213.630883:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ba5e1826. 00000100:00000001:1.0:1788880213.630888:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000010:3.0:1788880213.630891:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000100:00000001:1.0:1788880213.630893:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 00000400:00000200:0.0:1788880213.630894:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [73:73:256:4294967295] 192.168.204.111@tcp LPNI seq info [73:73:8:4294967295] 00000100:00000200:3.0:1788880213.630898:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780640640, portal 10 00000100:00000001:3.0:1788880213.630905:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880213.630908:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000100:00000040:1.0:1788880213.630915:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880213.630923:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880213.630929:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880213.630960:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880213.630970:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b899fe8d. 00000100:00000001:3.0:1788880213.630982:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880213.630983:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880213.630988:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:3.0:1788880213.630993:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880213.630999:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880213.631002:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b6d0d6b5. 00000400:00000010:1.0:1788880213.631004:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 00000800:00000200:0.0:1788880213.631005:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:3.0:1788880213.631008:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780640640, offset 0 00000100:00000200:1.0:1788880213.631013:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780640512, offset 0 00000800:00000200:0.0:1788880213.631014:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b899fe8d type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880213.631027:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880213.631039:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880213.631040:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880213.631041:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880213.631049:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880213.631053:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305849807 00000100:00000010:0.0:1788880213.631061:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009f22a9a0. 00000400:00000200:3.0:1788880213.631062:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880213.631067:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880213.631068:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:3.0:1788880213.631071:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880213.631075:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305849807 00000400:00000200:3.0:1788880213.631077:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [74:74:255:4294967295] 192.168.204.111@tcp LPNI seq info [74:74:7:4294967295] 00000100:00000010:0.0:1788880213.631080:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000029f028ab. 00000100:00000001:0.0:1788880213.631086:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:3.0:1788880213.631090:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880213.631101:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305849807 00000800:00000200:3.0:1788880213.631102:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880213.631108:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ffa8a4fd. 00000400:00000200:1.0:1788880213.631108:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:3.0:1788880213.631110:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000100:00000001:0.0:1788880213.631116:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880213.631123:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305849807 00000800:00000200:3.0:1788880213.631125:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880213.631128:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:0.0:1788880213.631129:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000a4f85af. 00000100:00000001:0.0:1788880213.631136:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788880213.631138:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880213.631142:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880213.631144:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:3.0:1788880213.631145:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880213.631148:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880213.631153:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b899fe8d. 00000800:00000200:3.0:1788880213.631153:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880213.631154:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880213.631160:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880213.631161:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880213.631162:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880213.631163:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880213.631176:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:3.0:1788880213.631176:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 119471 usecs (120 jiffies) 00000100:00000001:0.0:1788880213.631178:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880213.631184:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880213.631185:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880213.631188:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305844856 00000100:00000001:0.0:1788880213.631191:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880213.631198:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880213.631199:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305849807 00000400:00000200:1.0:1788880213.631200:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [75:75:254:4294967295] 192.168.204.111@tcp LPNI seq info [75:75:6:4294967295] 00000100:00000001:0.0:1788880213.631202:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880213.631204:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305844856 00000100:00000001:0.0:1788880213.631207:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880213.631228:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880213.631279:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305849807 00000100:00000001:3.0:1788880213.631282:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305844857 00000100:00000001:3.0:1788880213.631285:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305849807 00000100:00000001:3.0:1788880213.631288:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305844857 00000400:00000200:2.0:1788880213.631289:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a09430ba 00000800:00000200:1.0:1788880213.631296:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880213.631296:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305849807 00000400:00000010:2.0:1788880213.631297:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a09430ba. 00000100:00000001:3.0:1788880213.631305:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880213.631308:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880213.631309:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880213.631310:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000200:2.0:1788880213.631314:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880229 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880213.631314:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880213.631321:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880213.631330:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:1.0:1788880213.631331:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880213.631334:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880213.631342:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880213.631344:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880213.631345:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880213.631346:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880213.631351:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:2.0:1788880213.631355:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880229 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880213.631357:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880213.631361:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880213.631373:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880213.631381:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1693167 usecs (1694 jiffies) 00000100:00000001:2.0:1788880213.631381:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880213.631388:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880213.631390:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305846431 00000100:00000001:1.0:1788880213.631394:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305849807 00000100:00000001:1.0:1788880213.631397:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305846431 00000100:00000001:1.0:1788880213.631399:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305849807 00000100:00000001:1.0:1788880213.631401:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305846431 00000100:00000001:1.0:1788880213.631403:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305849807 00000100:00000001:1.0:1788880213.631405:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305846431 00000100:00000001:1.0:1788880213.631407:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305849807 00000100:00000001:1.0:1788880213.631410:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880213.631412:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880213.631414:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880213.631416:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000100:00000001:1.0:1788880213.631420:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880213.631423:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880213.631425:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880213.631431:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880213.631433:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b6d0d6b5 00000100:00000001:1.0:1788880213.631435:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:2.0:1788880213.631435:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b6d0d6b5. 00000100:00000001:1.0:1788880213.631438:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880213.631447:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880213.631448:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880213.631453:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880213.631454:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880213.631458:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880213.631467:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880213.631479:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880213.631483:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880213.631490:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880213.631492:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880213.631527:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880213.631530:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:2.0:1788880213.631534:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880213.631538:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880213.631540:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000400:00000010:2.0:1788880213.631542:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000001:2.0:1788880213.631544:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880213.631547:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880213.631553:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880213.631555:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880213.631558:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880213.631559:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880213.633033:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880213.633048:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880213.633052:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880213.633055:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880213.633118:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880213.633133:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02100 00000400:00000200:3.0:1788880213.633141:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x335 [1] + 224 00000800:00000001:3.0:1788880213.633147:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880213.633152:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880213.633154:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880213.633157:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880213.633162:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880213.633166:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880213.633176:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880213.633205:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880213.633213:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880213.633217:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880213.633219:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880213.633220:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880213.633224:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880213.633230:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b02080 00000100:00000001:1.0:1788880213.633274:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880213.633278:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880213.633282:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x32d [1] + 224 00000400:00000001:1.0:1788880213.633287:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788880213.633287:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880213.633291:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:3.0:1788880213.633296:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880213.633300:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:1.0:1788880213.633302:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:3.0:1788880213.633307:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880213.633308:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:1.0:1788880213.633311:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:1.0:1788880213.633315:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880213.633319:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880213.633319:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880213.633326:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880213.633338:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880213.633345:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880213.633348:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880213.633352:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880213.633356:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880213.633359:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:3.0:1788880213.633360:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880213.633363:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880213.633368:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880213.633381:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:1.0:1788880213.633383:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=31 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880213.633389:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=31 v=5 (1 0 0 0) 00000800:00000001:3.0:1788880213.633390:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880213.633394:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880213.633397:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880213.633398:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880213.633401:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:3.0:1788880213.633402:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:1.0:1788880213.633406:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880213.633407:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880213.633408:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880213.633412:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880213.633413:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880213.633417:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000200:3.0:1788880213.633417:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:0.0:1788880213.633421:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880213.633422:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000001:0.0:1788880213.633425:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880213.633426:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880213.633429:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02180 00000400:00000010:0.0:1788880213.633430:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000027964113. 00000400:00000200:0.0:1788880213.633435:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000400:00000010:0.0:1788880213.633439:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:0.0:1788880213.633442:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:1.0:1788880213.633445:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880213.633447:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x339 [1] + 224 00000100:00000200:0.0:1788880213.633450:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880213.633458:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880213.633466:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880213.633468:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880213.633472:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880213.633474:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:0.0:1788880213.633477:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880213.633479:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880213.633481:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880213.633489:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880213.633491:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880213.633492:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880213.633494:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880213.633496:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880213.633498:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880213.633501:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880213.633504:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880213.633507:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880213.633512:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880213.633517:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780640512:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880213.633521:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880213.633524:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=41 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880213.633531:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=41 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880213.633535:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880213.633539:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880213.633541:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880213.633542:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880213.633546:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:1.0:1788880213.633548:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780640512/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880213.633549:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880213.633555:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880213.633559:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880213.633562:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880213.633565:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000800:00000001:3.0:1788880213.633567:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000010:1.0:1788880213.633569:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000123d40c9. 00000800:00000001:3.0:1788880213.633572:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880213.633575:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880213.633576:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880213.633580:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880213.633584:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880213.633588:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880213.633589:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000010:1.0:1788880213.633593:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c54a5c6f. 00000100:00000001:0.0:1788880213.633594:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880213.633597:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880213.633598:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880213.633602:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:0.0:1788880213.633603:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880213.633605:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000001:2.0:1788880213.633607:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880213.633610:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880213.633612:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880213.633614:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880213.633619:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880213.633620:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00100000:0.0:1788880213.633624:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780640384:192.168.204.111@tcp:400: 00000400:00000001:2.0:1788880213.633625:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880213.633629:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:2.0:1788880213.633629:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ba5e1826. 00000100:00000001:0.0:1788880213.633631:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880213.633633:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880213.633636:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000100:00000001:0.0:1788880213.633637:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880213.633638:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:2.0:1788880213.633642:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:1.0:1788880213.633643:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880213.633644:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780640384/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880213.633650:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880213.633656:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:2.0:1788880213.633659:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880213.633660:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880213.633663:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a2f4800f. 02000000:00000001:0.0:1788880213.633669:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880213.633672:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880213.633675:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000200:2.0:1788880213.633676:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880213.633679:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880213.633681:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 02000000:00000001:0.0:1788880213.633683:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880213.633685:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880213.633686:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000100:00000001:0.0:1788880213.633691:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880213.633692:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880213.633695:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880213.633698:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880213.633701:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880213.633705:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788880213.633706:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788880213.633707:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880213.633709:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880213.633710:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880213.633711:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880213.633714:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880213.633714:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880213.633719:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880213.633724:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=41 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880213.633728:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=15 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880213.633730:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880213.633731:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880213.633732:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880213.633735:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880213.633753:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880213.633755:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880213.633757:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880213.633759:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880213.633760:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880213.633767:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880213.633773:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880213.633775:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880213.633777:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880213.633779:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880213.633787:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780640640:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880213.633792:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880213.633793:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880213.633795:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780640640/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880229 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880213.633813:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880213.633814:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880213.633816:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000005842920. 02000000:00000001:2.0:1788880213.633821:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880213.633823:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880213.633824:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880213.633826:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880213.633827:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 02000000:00000001:2.0:1788880213.633830:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880213.633831:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880213.633832:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 00000100:00000001:2.0:1788880213.633834:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880213.633836:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880213.633837:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880213.633841:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880213.633842:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880213.633844:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880213.633846:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880213.694341:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880213.694352:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880213.694355:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880213.694358:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880213.694364:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880213.694367:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880213.694369:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880213.694371:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880213.694375:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880213.694377:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880213.694419:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880213.694424:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880213.694428:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880214.718391:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880214.718399:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880214.718404:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880214.718407:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880214.718429:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880214.718431:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880214.718434:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880214.718435:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880214.718437:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880214.718438:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880214.718505:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880214.718513:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880214.718524:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000010:3.0:1788880214.846325:0:201282:0:(lib-lnet.h:450:lnet_rspt_free()) free 'rspt': 64 at 000000003270ddf2. 00000400:00000010:3.0:1788880214.846337:0:201282:0:(lib-lnet.h:450:lnet_rspt_free()) free 'rspt': 64 at 00000000295db731. 00010000:00000001:1.0:1788880215.742299:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880215.742306:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880215.742311:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880215.742312:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880215.742358:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880215.742362:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880215.742364:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880216.766363:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880216.766369:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880216.766373:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880216.766373:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880216.766430:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880216.766433:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880216.766435:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880217.790405:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880217.790412:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880217.790418:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880217.790420:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880217.790484:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880217.790491:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880217.790495:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880218.750336:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305849807, at 4305849856 00000100:00000040:0.0:1788880218.750337:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305849807, at 4305849856 00000100:00000040:1.0:1788880218.750337:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305849807, at 4305849856 00000100:00000040:3.0:1788880218.750337:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305849807, at 4305849856 00000100:00000010:2.0:1788880218.750351:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ffa8a4fd. 00000100:00000010:1.0:1788880218.750354:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000029f028ab. 00000100:00000010:0.0:1788880218.750356:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009f22a9a0. 00000100:00000010:3.0:1788880218.750359:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000a4f85af. 00000100:00000040:2.0:1788880218.750378:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880218.750382:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880218.750388:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 00000020:00000040:2.0:1788880218.750392:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880218.750395:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880218.750397:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880218.750401:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880218.750407:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 00000100:00000001:2.0:1788880218.750413:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880218.750421:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880218.750428:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880218.750430:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880218.750450:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_02+2] req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880218.750466:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880218.750468:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880218.750470:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880218.750471:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000020:00000040:2.0:1788880218.750473:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880218.750475:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880218.750476:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880218.750477:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880218.750479:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 00000100:00000001:2.0:1788880218.750482:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880218.750485:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880218.750490:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880218.750491:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880218.750492:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880218.750493:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_03+3] req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880218.750494:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788880218.750500:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:0.0:1788880218.750504:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1788880218.750505:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:2.0:1788880218.750506:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880218.750509:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880218.750511:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880218.750512:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880218.750516:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880218.750517:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880218.750520:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:2.0:1788880218.750522:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000884e74b8. 00000100:00000001:0.0:1788880218.750523:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880218.750525:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880218.750527:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:2.0:1788880218.750527:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880218.750528:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880218.750533:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880218.750534:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880218.750535:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880218.750537:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880218.750537:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880218.750541:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880218.750541:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:2.0:1788880218.750548:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 00000100:00000040:3.0:1788880218.750550:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880218.750552:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880218.750553:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880218.750555:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880218.750560:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880218.750563:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880218.750567:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880218.750573:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:0.0:1788880218.750574:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880218.750577:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880218.750581:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780640768:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880218.750581:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880218.750584:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880218.750591:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880218.750593:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00100000:3.0:1788880218.750597:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780640896:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880218.750603:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880218.750607:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880218.750608:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880218.750608:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880218.750612:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880218.750612:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880218.750612:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880218.750619:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:3.0:1788880218.750619:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880218.750623:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000884e74b8] to pc [ptlrpcd_00_00+0] req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880218.750625:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000482afcf7. 02000000:00000010:3.0:1788880218.750626:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d653e4f6. 02000000:00000001:0.0:1788880218.750630:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880218.750631:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880218.750639:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000027964113. 00000400:00000010:3.0:1788880218.750639:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007e36d068. 00000400:00000010:0.0:1788880218.750649:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000400:00000010:3.0:1788880218.750649:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000100:00000200:0.0:1788880218.750656:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780640768, portal 25 00000100:00000200:3.0:1788880218.750656:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780640896, portal 10 00000100:00000001:2.0:1788880218.750657:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880218.750662:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880218.750662:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:2.0:1788880218.750663:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880218.750666:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000100:00000001:3.0:1788880218.750666:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 00000100:00000040:2.0:1788880218.750672:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788880218.750676:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880234 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880218.750678:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880218.750678:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880218.750679:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11187) 00000100:00000001:1.0:1788880218.750684:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880218.750688:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880218.750692:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880218.750696:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880218.750697:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880218.750705:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880218.750706:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880218.750713:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880218.750715:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880218.750718:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000400:00000010:3.0:1788880218.750720:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b6d0d6b5. 00000100:00000001:1.0:1788880218.750721:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000200:0.0:1788880218.750724:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780640768, offset 0 00000100:00000001:1.0:1788880218.750725:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880218.750726:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780640896, offset 0 02000000:00000001:1.0:1788880218.750732:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788880218.750736:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880218.750737:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880218.750742:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880218.750746:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788880218.750748:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780641024:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880218.750753:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880218.750756:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880218.750760:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788880218.750760:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880218.750765:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880218.750768:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880218.750773:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000010:1.0:1788880218.750776:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005309c375. 00000400:00000200:0.0:1788880218.750778:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:1.0:1788880218.750781:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880218.750784:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:1.0:1788880218.750788:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000400:00000200:0.0:1788880218.750790:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [76:76:256:4294967295] 192.168.204.111@tcp LPNI seq info [76:76:8:4294967295] 00000400:00000010:1.0:1788880218.750794:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:1.0:1788880218.750798:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780641024, portal 10 00000100:00000001:1.0:1788880218.750802:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880218.750804:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938423808 : -126823771127808 : ffff8ca78a2c4000) 00000400:00000200:0.0:1788880218.750807:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:1.0:1788880218.750815:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880218.750825:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880218.750825:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880218.750830:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000010:0.0:1788880218.750833:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b899fe8d. 00000400:00000200:3.0:1788880218.750837:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788880218.750838:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880218.750843:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:1.0:1788880218.750843:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000030b8c48c. 00000400:00000200:3.0:1788880218.750844:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:1.0:1788880218.750850:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780641024, offset 0 00000400:00000200:3.0:1788880218.750851:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788880218.750856:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880218.750858:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880218.750861:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880218.750864:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b899fe8d type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880218.750866:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [77:77:255:4294967295] 192.168.204.111@tcp LPNI seq info [77:77:7:4294967295] 00000400:00000200:3.0:1788880218.750877:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880218.750878:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880218.750885:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305854927 00000800:00000200:3.0:1788880218.750886:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880218.750889:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:0.0:1788880218.750891:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009f22a9a0. 00000800:00000010:3.0:1788880218.750893:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000400:00000200:1.0:1788880218.750895:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880218.750901:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880218.750903:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:3.0:1788880218.750904:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880218.750908:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880218.750910:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305854927 00000400:00000200:1.0:1788880218.750915:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788880218.750915:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:0.0:1788880218.750917:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000975d5882. 00000800:00000200:3.0:1788880218.750921:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880218.750922:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [78:78:254:4294967295] 192.168.204.111@tcp LPNI seq info [78:78:6:4294967295] 00000100:00000001:0.0:1788880218.750923:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880218.750929:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880218.750930:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305854927 00000100:00000010:0.0:1788880218.750936:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001a2ff840. 00000100:00000040:3.0:1788880218.750936:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1587561 usecs (1588 jiffies) 00000400:00000200:1.0:1788880218.750937:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880218.750942:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880218.750943:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305851444 00000800:00000200:1.0:1788880218.750948:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880218.750949:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305854927 00000100:00000001:3.0:1788880218.750949:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305854927 00000800:00000010:1.0:1788880218.750953:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:3.0:1788880218.750955:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305851444 00000800:00000200:1.0:1788880218.750959:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:0.0:1788880218.750960:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009ac8d70b. 00000100:00000001:3.0:1788880218.750961:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305854927 00000100:00000001:3.0:1788880218.750966:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305851444 00000100:00000001:0.0:1788880218.750967:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880218.750969:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305854927 00000100:00000001:0.0:1788880218.750972:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880218.750975:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880218.750976:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305851444 00000100:00000001:0.0:1788880218.750977:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880218.750983:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880218.750983:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305854927 00000100:00000001:0.0:1788880218.750984:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880218.750984:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880218.750988:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880218.750990:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880218.750991:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b899fe8d. 00000100:00000001:1.0:1788880218.750992:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880218.750994:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880218.750999:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 109828 usecs (110 jiffies) 00000100:00000001:3.0:1788880218.750999:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880218.751001:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880218.751004:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880218.751009:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305849966 00000400:00000200:2.0:1788880218.751010:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880218.751014:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305854927 00000100:00000001:1.0:1788880218.751017:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305849966 00000400:00000200:2.0:1788880218.751018:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:2.0:1788880218.751022:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:1.0:1788880218.751027:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305854927 00000100:00000001:1.0:1788880218.751030:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305849966 00000100:00000001:2.0:1788880218.751030:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880218.751033:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305854927 00000100:00000001:1.0:1788880218.751035:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305849966 00000100:00000001:1.0:1788880218.751037:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305854927 00000100:00000200:2.0:1788880218.751037:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880234 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880218.751041:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880218.751045:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880218.751049:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880218.751053:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880218.751057:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880218.751058:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880218.751061:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880218.751064:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880218.751064:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880234 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880218.751066:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880218.751069:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880218.751072:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880218.751079:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880218.751081:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880218.751095:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880218.751097:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000400:00000200:2.0:1788880218.751100:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880218.751102:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880218.751105:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b6d0d6b5 00000400:00000010:2.0:1788880218.751106:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b6d0d6b5. 00000100:00000001:2.0:1788880218.751114:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880218.751117:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880218.751123:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880218.751125:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880218.751131:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880218.751133:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880218.751186:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880218.751188:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:2.0:1788880218.751190:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880218.751193:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880218.751195:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000030b8c48c 00000400:00000010:2.0:1788880218.751196:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000030b8c48c. 00000100:00000001:2.0:1788880218.751198:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880218.751199:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880218.751204:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880218.751205:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880218.751209:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880218.751209:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880218.752358:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880218.752367:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880218.752370:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880218.752372:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880218.752378:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880218.752388:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b02200 00000400:00000200:2.0:1788880218.752395:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x345 [1] + 224 00000800:00000001:2.0:1788880218.752402:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880218.752405:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880218.752407:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880218.752417:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880218.752422:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880218.752425:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880218.752435:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880218.752450:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880218.752457:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880218.752461:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880218.752463:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880218.752466:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880218.752470:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880218.752474:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880218.752477:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880218.752478:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02280 00000400:00000001:0.0:1788880218.752482:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880218.752484:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880218.752486:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000027964113. 00000400:00000200:0.0:1788880218.752489:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000400:00000010:0.0:1788880218.752490:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:0.0:1788880218.752492:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880218.752496:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880218.752497:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x349 [1] + 224 00000800:00000001:2.0:1788880218.752507:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880218.752509:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880218.752514:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880218.752519:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880218.752523:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880218.752524:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880218.752528:0:201288:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:2.0:1788880218.752528:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:0.0:1788880218.752531:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880218.752534:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880218.752536:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880218.752537:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880218.752538:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880218.752543:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880218.752549:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=46 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880218.752555:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=46 v=5 (1 0 0 0) 00000100:00000040:2.0:1788880218.752557:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880218.752558:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880218.752560:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880218.752563:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880218.752566:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880218.752570:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880218.752574:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880218.752577:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880218.752580:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880218.752583:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880218.752585:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880218.752585:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1788880218.752590:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880218.752590:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:0.0:1788880218.752598:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780640768:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880218.752601:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880218.752602:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880218.752604:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780640768/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880218.752608:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880218.752610:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880218.752611:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000482afcf7. 02000000:00000001:0.0:1788880218.752614:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880218.752615:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880218.752617:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880218.752619:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880218.752620:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 02000000:00000001:0.0:1788880218.752623:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880218.752624:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880218.752625:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 00000100:00000001:0.0:1788880218.752627:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880218.752628:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880218.752629:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880218.752636:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880218.752636:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880218.752638:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880218.752640:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880218.752743:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880218.752750:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880218.752753:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880218.752755:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880218.752761:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880218.752772:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02300 00000100:00000001:3.0:1788880218.752776:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880218.752778:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x355 [1] + 224 00000100:00000001:3.0:1788880218.752785:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:1.0:1788880218.752787:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1788880218.752797:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:1.0:1788880218.752799:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880218.752802:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:1.0:1788880218.752804:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880218.752808:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:3.0:1788880218.752819:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007e36d068. 00000400:00000200:3.0:1788880218.752826:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000400:00000010:3.0:1788880218.752828:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:3.0:1788880218.752832:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880218.752834:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880218.752846:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880218.752847:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880218.752895:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880218.752900:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880218.752929:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880218.752933:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880218.752935:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880218.752940:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880218.752943:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880218.752945:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880218.752952:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880218.752964:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=36 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880218.752970:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=36 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880218.752973:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880218.752974:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880218.752978:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880218.752979:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:3.0:1788880218.752982:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880218.752987:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000800:00000001:1.0:1788880218.752989:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:3.0:1788880218.752990:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880218.752995:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000800:00000001:1.0:1788880218.752996:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880218.752997:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880218.753008:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880218.753014:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880218.753017:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880218.753019:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880218.753022:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880218.753027:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880218.753027:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780640896:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880218.753033:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880218.753037:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880218.753042:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880218.753043:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880218.753045:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000100:00000001:3.0:1788880218.753049:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788880218.753053:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000100:00000040:3.0:1788880218.753056:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780640896/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880218.753066:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:3.0:1788880218.753070:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880218.753073:0:201285:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1788880218.753075:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880218.753079:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d653e4f6. 00000100:00000200:2.0:1788880218.753082:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880218.753085:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880218.753090:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880218.753092:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880218.753100:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880218.753104:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 02000000:00000001:3.0:1788880218.753110:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000200:2.0:1788880218.753111:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880218.753114:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880218.753116:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000100:00000001:3.0:1788880218.753122:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880218.753126:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880218.753132:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880218.753138:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880218.753141:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880218.753143:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880218.753144:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880218.753163:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880218.753167:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880218.753169:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880218.753173:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880218.753175:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880218.753176:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880218.753179:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880218.753184:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=46 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880218.753188:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=20 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880218.753191:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880218.753192:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880218.753194:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880218.753197:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880218.753205:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880218.753207:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880218.753208:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880218.753210:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880218.753212:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880218.753215:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880218.753221:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880218.753223:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880218.753225:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880218.753228:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880218.753273:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780641024:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880218.753277:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880218.753279:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880218.753281:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000884e74b8 x1875776780641024/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880234 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880218.753287:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880218.753289:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880218.753291:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005309c375. 02000000:00000001:2.0:1788880218.753294:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880218.753296:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880218.753297:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880218.753299:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880218.753301:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 02000000:00000001:2.0:1788880218.753303:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880218.753305:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880218.753306:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000884e74b8. 00000100:00000001:2.0:1788880218.753309:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880218.753310:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880218.753312:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880218.753315:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880218.753317:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880218.753319:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880218.753321:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880218.814358:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880218.814367:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880218.814373:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880218.814375:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880218.814431:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880218.814436:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880218.814439:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880219.838350:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880219.838357:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880219.838363:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880219.838364:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880219.838416:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880219.838420:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880219.838423:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880220.862380:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880220.862391:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880220.862400:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880220.862402:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880220.862471:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880220.862476:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880220.862480:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880221.886379:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880221.886384:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880221.886388:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880221.886389:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880221.886450:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880221.886455:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880221.886458:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880222.910368:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880222.910375:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880222.910381:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880222.910382:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880222.910447:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880222.910451:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880222.910453:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880223.870379:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305854927, at 4305854976 00000100:00000040:1.0:1788880223.870379:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305854927, at 4305854976 00000100:00000040:3.0:1788880223.870379:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305854927, at 4305854976 00000100:00000010:3.0:1788880223.870399:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009ac8d70b. 00000100:00000010:1.0:1788880223.870401:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000975d5882. 00000100:00000010:0.0:1788880223.870405:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009f22a9a0. 00000100:00000040:3.0:1788880223.870418:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880223.870424:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880223.870433:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788880223.870438:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880223.870444:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880223.870447:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880223.870452:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880223.870460:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5b428a0. 00000100:00000001:3.0:1788880223.870467:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880223.870478:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:2.0:1788880223.870486:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305854927, at 4305854976 00000100:00000010:2.0:1788880223.870504:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001a2ff840. 00000100:00000001:3.0:1788880223.870515:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880223.870521:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880223.870529:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_01+1] req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880223.870560:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880223.870563:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880223.870566:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1788880223.870567:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:3.0:1788880223.870570:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880223.870575:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880223.870576:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880223.870580:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880223.870582:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880223.870582:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880223.870587:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880223.870590:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000055d7d92. 00000100:00000001:3.0:1788880223.870593:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880223.870599:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:2.0:1788880223.870600:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880223.870616:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880223.870620:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880223.870628:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_02+2] req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880223.870643:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880223.870651:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880223.870660:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880223.870665:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880223.870679:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880223.870686:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880223.870691:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880223.870691:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880223.870694:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000100:00000001:1.0:1788880223.870696:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880223.870699:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000020:00000040:3.0:1788880223.870699:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00100000:1.0:1788880223.870700:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:3.0:1788880223.870707:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880223.870709:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880223.870709:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880223.870710:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880223.870715:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880223.870720:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880223.870725:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880223.870731:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00100000:2.0:1788880223.870731:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780641152:192.168.204.111@tcp:400: 02000000:00000010:3.0:1788880223.870734:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000252f4482. 00000100:00000001:1.0:1788880223.870735:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880223.870737:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880223.870739:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880223.870740:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880223.870742:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880223.870742:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:1.0:1788880223.870750:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880223.870750:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880223.870752:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880223.870753:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880223.870760:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ee42a892. 02000000:00000001:2.0:1788880223.870764:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880223.870766:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:2.0:1788880223.870772:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000038fe0168. 00000400:00000010:2.0:1788880223.870781:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c56cc487. 00000100:00000001:1.0:1788880223.870789:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000200:2.0:1788880223.870793:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780641152, portal 25 00000100:00000001:3.0:1788880223.870795:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880223.870798:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880223.870799:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880223.870801:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880223.870807:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000040:3.0:1788880223.870812:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_03+3] req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880223.870814:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880223.870818:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880223.870820:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880239 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880223.870828:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880223.870840:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780641280:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880223.870859:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880223.870865:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880223.870869:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880223.870879:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880223.870881:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:1.0:1788880223.870883:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000852b7c70. 02000000:00000001:1.0:1788880223.870887:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880223.870887:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880223.870889:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880223.870893:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008f790e4a. 00000400:00000010:1.0:1788880223.870894:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b43de041. 00000100:00000040:3.0:1788880223.870895:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:2.0:1788880223.870898:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780641152, offset 0 00000400:00000010:1.0:1788880223.870900:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000040:3.0:1788880223.870908:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880223.870910:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880223.870911:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880223.870916:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11192) 00000100:00100000:0.0:1788880223.870919:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000200:1.0:1788880223.870921:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780641280, portal 10 00000100:00000001:0.0:1788880223.870926:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880223.870927:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788880223.870929:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880223.870931:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000400:00000200:2.0:1788880223.870940:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880223.870941:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788880223.870946:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880223.870947:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880223.870947:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880223.870954:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880223.870956:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880223.870961:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880223.870962:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880223.870969:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880223.870970:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [79:79:256:4294967295] 192.168.204.111@tcp LPNI seq info [79:79:8:4294967295] 00000100:00000001:1.0:1788880223.870975:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880223.870980:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880223.870982:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880223.870989:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000ae22117. 00000100:00000200:1.0:1788880223.870999:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780641280, offset 0 00000100:00000001:0.0:1788880223.871016:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1788880223.871017:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880223.871021:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880223.871025:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880223.871030:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880223.871041:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880223.871046:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880223.871046:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:2.0:1788880223.871049:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00100000:0.0:1788880223.871062:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780641408:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880223.871067:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:2.0:1788880223.871072:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000045330cf1. 00000100:00000001:0.0:1788880223.871076:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788880223.871080:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880223.871084:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1788880223.871086:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1788880223.871088:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880223.871093:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880223.871094:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788880223.871096:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880223.871103:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [80:80:255:4294967295] 192.168.204.111@tcp LPNI seq info [80:80:7:4294967295] 02000000:00000010:0.0:1788880223.871104:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a2f4800f. 02000000:00000001:0.0:1788880223.871112:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880223.871114:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:0.0:1788880223.871123:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000027964113. 00000800:00000200:2.0:1788880223.871123:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000045330cf1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880223.871124:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:0.0:1788880223.871135:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000800:00000200:1.0:1788880223.871141:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:0.0:1788880223.871143:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780641408, portal 10 00000100:00000001:0.0:1788880223.871148:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000010:1.0:1788880223.871148:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000100:00000001:0.0:1788880223.871153:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000100:00000001:2.0:1788880223.871153:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880223.871160:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880223.871167:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305860047 00000800:00000200:1.0:1788880223.871168:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:0.0:1788880223.871174:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880223.871176:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000010:2.0:1788880223.871177:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000059817523. 00000100:00000001:1.0:1788880223.871190:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880223.871190:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:2.0:1788880223.871199:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305860047 00000100:00000040:1.0:1788880223.871201:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1494355 usecs (1495 jiffies) 00000100:00000001:0.0:1788880223.871202:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:2.0:1788880223.871205:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000780e164. 00000100:00000001:1.0:1788880223.871210:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305856471 00000100:00000001:2.0:1788880223.871210:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:3.0:1788880223.871217:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:0.0:1788880223.871218:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880223.871221:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305860047 00000100:00000001:2.0:1788880223.871221:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305860047 00000400:00000010:0.0:1788880223.871226:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000115221d8. 00000800:00000010:3.0:1788880223.871227:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000045330cf1. 00000100:00000001:1.0:1788880223.871230:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305856471 00000100:00000010:2.0:1788880223.871231:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001e508d7a. 00000100:00000001:1.0:1788880223.871262:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305860047 00000100:00000001:2.0:1788880223.871263:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:3.0:1788880223.871265:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:0.0:1788880223.871269:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780641408, offset 0 00000100:00000001:1.0:1788880223.871269:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305856472 00000100:00000001:2.0:1788880223.871273:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305860048 00000400:00000200:3.0:1788880223.871279:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880223.871281:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305860047 00000100:00000010:2.0:1788880223.871284:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003053b935. 00000100:00000001:1.0:1788880223.871286:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305856472 00000400:00000200:0.0:1788880223.871292:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880223.871293:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880223.871300:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880223.871301:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305860048 00000100:00000001:2.0:1788880223.871303:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880223.871306:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880223.871308:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880223.871308:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008f790e4a 00000400:00000010:3.0:1788880223.871313:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008f790e4a. 00000400:00000200:0.0:1788880223.871314:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880223.871314:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880223.871316:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880223.871326:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880223.871329:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880223.871329:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:0.0:1788880223.871331:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880223.871334:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880223.871336:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880223.871341:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:3.0:1788880223.871346:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880239 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880223.871349:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880223.871350:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880223.871352:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880223.871356:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880223.871359:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880223.871362:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880223.871364:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880223.871368:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880223.871373:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880223.871377:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [81:81:254:4294967295] 192.168.204.111@tcp LPNI seq info [81:81:6:4294967295] 00000100:00000001:1.0:1788880223.871384:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880223.871397:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1788880223.871402:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788880223.871407:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880239 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880223.871412:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880223.871419:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000088542ca3. 00000800:00000200:0.0:1788880223.871426:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880223.871427:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880223.871431:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880223.871433:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880223.871443:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000088542ca3 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880223.871448:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880223.871452:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 101623 usecs (102 jiffies) 00000100:00000001:0.0:1788880223.871455:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305855079 00000100:00000001:0.0:1788880223.871456:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305860047 00000100:00000001:0.0:1788880223.871458:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305855079 00000100:00000001:0.0:1788880223.871459:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305860047 00000100:00000001:0.0:1788880223.871461:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305855079 00000100:00000001:0.0:1788880223.871463:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305860047 00000100:00000001:0.0:1788880223.871464:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305855079 00000100:00000001:0.0:1788880223.871465:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305860048 00000100:00000001:0.0:1788880223.871467:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880223.871469:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880223.871471:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880223.871474:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880223.871503:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880223.871508:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:3.0:1788880223.871512:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880223.871517:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880223.871520:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000ae22117 00000400:00000010:3.0:1788880223.871521:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000ae22117. 00000100:00000001:3.0:1788880223.871531:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880223.871534:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880223.871542:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880223.871544:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880223.871550:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880223.871551:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880223.871564:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880223.871567:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000088542ca3. 00000400:00000200:3.0:1788880223.871570:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880223.871573:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880223.871576:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000115221d8 00000400:00000010:3.0:1788880223.871578:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000115221d8. 00000100:00000001:3.0:1788880223.871580:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880223.871583:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880223.871590:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880223.871591:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880223.871597:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880223.871598:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880223.873262:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880223.873273:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880223.873276:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880223.873277:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880223.873284:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880223.873293:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b02380 00000400:00000200:2.0:1788880223.873299:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x35d [1] + 224 00000800:00000001:2.0:1788880223.873310:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880223.873313:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880223.873315:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880223.873319:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880223.873323:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880223.873327:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880223.873333:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880223.873350:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880223.873356:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880223.873358:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880223.873359:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880223.873360:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880223.873362:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880223.873366:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02400 00000400:00000200:2.0:1788880223.873369:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x365 [1] + 224 00000800:00000001:2.0:1788880223.873373:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880223.873374:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880223.873376:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880223.873382:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880223.873385:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880223.873387:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880223.873392:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880223.873404:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880223.873412:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880223.873414:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880223.873415:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880223.873420:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880223.873421:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880223.873421:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880223.873431:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880223.873433:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:0.0:1788880223.873437:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880223.873442:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000038fe0168. 00000400:00000200:2.0:1788880223.873443:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02480 00000400:00000200:0.0:1788880223.873454:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c56cc487 00000400:00000010:0.0:1788880223.873464:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c56cc487. 00000100:00000001:1.0:1788880223.873465:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880223.873471:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880223.873473:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880223.873475:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x36d [1] + 224 00000800:00000001:2.0:1788880223.873479:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880223.873480:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:0.0:1788880223.873484:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880223.873485:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:2.0:1788880223.873485:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880223.873492:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b43de041. 00000400:00000200:2.0:1788880223.873492:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880223.873497:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880223.873509:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:1.0:1788880223.873514:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:1.0:1788880223.873520:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880223.873521:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880223.873522:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880223.873533:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880223.873539:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880223.873568:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:2.0:1788880223.873573:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880223.873574:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880223.873583:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000200:1.0:1788880223.873584:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880223.873595:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880223.873597:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788880223.873598:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880223.873600:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880223.873603:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880223.873603:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880223.873604:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788880223.873612:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880223.873613:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788880223.873613:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880223.873617:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880223.873620:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880223.873628:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880223.873629:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880223.873643:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=51 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880223.873644:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=41 v=5 (5 0 0 0) 00000100:00000001:3.0:1788880223.873649:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880223.873654:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:0.0:1788880223.873664:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=51 v=5 (1 0 0 0) 00000100:00001000:1.0:1788880223.873665:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=41 v=5 (1 0 0 0) 00000400:00000001:3.0:1788880223.873665:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880223.873670:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880223.873676:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000027964113. 00000100:00000001:0.0:1788880223.873684:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:3.0:1788880223.873686:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000100:00000001:0.0:1788880223.873688:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880223.873691:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:1.0:1788880223.873692:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00010000:00000001:0.0:1788880223.873695:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880223.873697:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880223.873698:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:0.0:1788880223.873699:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880223.873702:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880223.873706:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880223.873706:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880223.873708:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880223.873716:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880223.873718:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880223.873720:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880223.873727:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880223.873730:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880223.873743:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880223.873744:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880223.873750:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880223.873758:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880223.873763:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880223.873779:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880223.873811:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880223.873818:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880223.873820:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:0.0:1788880223.873828:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780641152:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880223.873829:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880223.873830:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880223.873840:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880223.873840:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880223.873842:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880223.873845:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880223.873848:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880223.873849:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880223.873853:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780641152/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880223.873854:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880223.873857:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880223.873874:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=51 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880223.873889:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=25 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880223.873902:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880223.873909:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880223.873911:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00100000:1.0:1788880223.873912:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780641280:192.168.204.111@tcp:400: 00010000:00000001:3.0:1788880223.873913:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:0.0:1788880223.873916:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880223.873923:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ee42a892. 00010000:00080000:3.0:1788880223.873923:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880223.873925:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880223.873931:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:0.0:1788880223.873935:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:1.0:1788880223.873940:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780641280/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880223.873942:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00010000:00000001:3.0:1788880223.873943:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880223.873944:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880223.873951:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880223.873954:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880223.873958:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880223.873961:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880223.873962:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000020:00000001:0.0:1788880223.873965:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:1.0:1788880223.873967:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880223.873971:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5b428a0. 00000100:00000040:3.0:1788880223.873971:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880223.873972:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000852b7c70. 02000000:00000001:1.0:1788880223.873982:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:0.0:1788880223.873983:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000020:00000001:1.0:1788880223.873986:0:201288:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000001:0.0:1788880223.873987:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880223.873991:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000020:00000040:1.0:1788880223.873992:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880223.873996:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880223.873998:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000055d7d92. 00000100:00000001:3.0:1788880223.874007:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880223.874011:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880223.874011:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880223.874013:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880223.874015:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880223.874016:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000010:1.0:1788880223.874018:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:0.0:1788880223.874021:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880223.874021:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880223.874026:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880223.874029:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880223.874030:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880223.874040:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880223.874045:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880223.874050:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880223.874050:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780641408:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880223.874051:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880223.874065:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880223.874074:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880223.874075:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880223.874076:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880223.874081:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880223.874081:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880223.874082:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880223.874090:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780641408/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880239 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880223.874121:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880223.874123:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880223.874126:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a2f4800f. 02000000:00000001:3.0:1788880223.874129:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880223.874130:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880223.874131:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880223.874134:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880223.874135:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000252f4482. 02000000:00000001:3.0:1788880223.874138:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880223.874139:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880223.874140:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:3.0:1788880223.874143:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880223.874144:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880223.874146:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880223.874150:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880223.874151:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880223.874153:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880223.874155:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880223.934347:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880223.934354:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880223.934356:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880223.934358:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880223.934362:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880223.934363:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880223.934365:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880223.934366:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880223.934369:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880223.934369:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880223.934426:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880223.934433:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880223.934438:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880224.958385:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880224.958394:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880224.958404:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880224.958408:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880224.958411:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880224.958415:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880224.958419:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880224.958422:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880224.958449:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880224.958452:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880224.958548:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880224.958555:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880224.958558:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880225.982382:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880225.982389:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880225.982395:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880225.982396:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880225.982446:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880225.982450:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880225.982454:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880227.006383:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880227.006390:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880227.006396:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880227.006397:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880227.006463:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880227.006467:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880227.006470:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880228.030364:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880228.030371:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880228.030379:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880228.030380:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880228.030442:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880228.030448:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880228.030451:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880228.990388:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305860047, at 4305860096 00000100:00000040:0.0:1788880228.990390:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305860047, at 4305860096 00000100:00000040:1.0:1788880228.990390:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305860047, at 4305860096 00000100:00000040:3.0:1788880228.990400:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305860048, at 4305860096 00000100:00000010:2.0:1788880228.990410:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001e508d7a. 00000100:00000010:0.0:1788880228.990413:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000059817523. 00000100:00000010:1.0:1788880228.990418:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000780e164. 00000100:00000010:3.0:1788880228.990420:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003053b935. 00000100:00000040:2.0:1788880228.990471:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880228.990478:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880228.990488:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000884e74b8. 00000020:00000040:2.0:1788880228.990494:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880228.990498:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880228.990502:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880228.990507:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880228.990516:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e53a8f34. 00000100:00000001:2.0:1788880228.990524:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880228.990537:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880228.990548:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880228.990551:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880228.990559:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000884e74b8] to pc [ptlrpcd_00_00+0] req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880228.990619:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880228.990621:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880228.990625:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880228.990629:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cc13c26b. 00000020:00000040:2.0:1788880228.990632:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880228.990635:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880228.990638:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880228.990642:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880228.990645:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880228.990648:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 00000100:00000001:3.0:1788880228.990651:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880228.990654:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880228.990655:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880228.990666:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880228.990669:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880228.990686:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880228.990688:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880228.990691:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880228.990692:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880228.990700:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880228.990703:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880228.990703:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880228.990710:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880228.990714:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880228.990721:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780641536:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880228.990723:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cc13c26b] to pc [ptlrpcd_00_01+1] req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880228.990726:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880228.990733:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880228.990748:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880228.990748:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880228.990751:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880228.990759:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007d30b075. 00000100:00000040:2.0:1788880228.990760:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880228.990765:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880228.990769:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880228.990775:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880228.990776:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000400:00000010:3.0:1788880228.990776:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007e36d068. 00000100:00000001:0.0:1788880228.990779:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880228.990788:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880228.990788:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000100:00000010:2.0:1788880228.990791:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000629edf55. 00000100:00000200:3.0:1788880228.990797:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780641536, portal 25 00000100:00000001:3.0:1788880228.990805:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000020:00000040:2.0:1788880228.990806:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880228.990808:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938423808 : -126823771127808 : ffff8ca78a2c4000) 00000100:00000001:0.0:1788880228.990810:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880228.990818:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880228.990819:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:3.0:1788880228.990822:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880244 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880228.990825:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880228.990827:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880228.990833:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880228.990834:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880228.990836:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880228.990838:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880228.990845:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:2.0:1788880228.990848:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 00000400:00000010:3.0:1788880228.990849:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000100:00100000:0.0:1788880228.990852:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780641664:192.168.204.111@tcp:400: 00000100:00000200:3.0:1788880228.990859:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780641536, offset 0 00000100:00000001:0.0:1788880228.990861:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880228.990864:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880228.990868:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880228.990868:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880228.990871:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880228.990876:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880228.990882:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000010b36294. 00000100:00000040:2.0:1788880228.990882:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880228.990891:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880228.990896:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:0.0:1788880228.990905:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000400:00000200:3.0:1788880228.990909:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:0.0:1788880228.990917:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000400:00000200:3.0:1788880228.990920:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880228.990922:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000200:0.0:1788880228.990926:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780641664, portal 10 00000400:00000200:3.0:1788880228.990926:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880228.990930:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788880228.990933:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880228.990938:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938433664 : -126823771117952 : ffff8ca78a2c6680) 00000100:00000001:2.0:1788880228.990939:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880228.990948:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [82:82:256:4294967295] 192.168.204.111@tcp LPNI seq info [82:82:8:4294967295] 00000100:00000040:0.0:1788880228.990967:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880228.990968:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000629edf55] to pc [ptlrpcd_00_02+2] req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880228.990993:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880228.991009:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880228.991025:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880228.991025:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880228.991030:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000fbbca8a. 00000100:00000200:0.0:1788880228.991033:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780641664, offset 0 00000800:00000010:3.0:1788880228.991034:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000400:00000200:0.0:1788880228.991041:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880228.991044:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880228.991047:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880228.991050:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:2.0:1788880228.991058:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000200:3.0:1788880228.991059:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880228.991060:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880228.991072:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880228.991074:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880228.991076:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880228.991080:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880228.991082:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880228.991085:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880228.991088:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:0.0:1788880228.991089:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880228.991097:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880228.991103:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11197) 00000400:00000200:0.0:1788880228.991104:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [83:83:255:4294967295] 192.168.204.111@tcp LPNI seq info [83:83:7:4294967295] 00000100:00000040:1.0:1788880228.991105:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880228.991105:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305865167 00000100:00000010:3.0:1788880228.991115:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fbefa178. 00000100:00000001:2.0:1788880228.991122:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880228.991123:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:0.0:1788880228.991126:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880228.991130:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880228.991130:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880228.991132:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:0.0:1788880228.991137:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880228.991141:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000010:0.0:1788880228.991143:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 02000000:00000001:1.0:1788880228.991146:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880228.991146:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305865167 00000800:00000200:0.0:1788880228.991147:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:1.0:1788880228.991153:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880228.991154:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000016ab1a15. 00000800:00000200:0.0:1788880228.991159:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00100000:1.0:1788880228.991160:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780641792:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880228.991161:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:0.0:1788880228.991162:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880228.991167:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880228.991169:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305865167 02000000:00000001:1.0:1788880228.991172:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880228.991173:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880228.991174:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000945234cf. 02000000:00000001:1.0:1788880228.991175:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880228.991179:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880228.991180:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1411540 usecs (1412 jiffies) 00000100:00000001:3.0:1788880228.991180:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 02000000:00000010:1.0:1788880228.991184:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002b77e71a. 00000100:00000001:0.0:1788880228.991186:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305861508 02000000:00000001:1.0:1788880228.991190:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880228.991190:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305865167 00000100:00000001:0.0:1788880228.991191:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305865167 00000800:00000200:2.0:1788880228.991194:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:3.0:1788880228.991196:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a5ad6403. 00000100:00000001:0.0:1788880228.991197:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305861508 00000400:00000010:1.0:1788880228.991198:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b43de041. 00000800:00000010:2.0:1788880228.991201:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000100:00000001:3.0:1788880228.991202:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880228.991205:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305865167 00000400:00000010:1.0:1788880228.991208:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000001:3.0:1788880228.991211:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880228.991213:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305861508 00000400:00000200:2.0:1788880228.991214:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880228.991216:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880228.991222:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780641792, portal 10 00000100:00000001:0.0:1788880228.991226:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305865167 00000100:00000001:3.0:1788880228.991227:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880228.991230:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880228.991255:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880228.991258:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305861509 00000100:00000001:1.0:1788880228.991258:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938432768 : -126823771118848 : ffff8ca78a2c6300) 00000100:00000001:0.0:1788880228.991262:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305865167 00000100:00000001:3.0:1788880228.991264:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880228.991265:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880228.991268:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880228.991270:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880228.991273:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880228.991274:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880228.991276:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880228.991276:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880228.991284:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880228.991284:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880228.991292:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880228.991293:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880228.991295:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00000100:00000001:3.0:1788880228.991295:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880228.991296:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880228.991308:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880228.991309:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:2.0:1788880228.991309:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000100:00000001:0.0:1788880228.991315:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880228.991319:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880228.991320:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880228.991323:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880228.991324:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000487fae38. 00000100:00000001:2.0:1788880228.991326:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880228.991330:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780641792, offset 0 00000100:00000200:2.0:1788880228.991337:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880244 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880228.991341:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880228.991353:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880228.991355:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:1.0:1788880228.991358:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:2.0:1788880228.991366:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880244 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880228.991367:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880228.991373:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880228.991382:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880228.991390:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [84:84:254:4294967295] 192.168.204.111@tcp LPNI seq info [84:84:6:4294967295] 00000100:00000001:2.0:1788880228.991401:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880228.991405:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:1.0:1788880228.991413:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880228.991419:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880228.991424:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000800:00000200:1.0:1788880228.991428:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880228.991438:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880228.991442:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880228.991445:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880228.991450:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 94628 usecs (95 jiffies) 00000100:00000001:1.0:1788880228.991453:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305860192 00000100:00000001:1.0:1788880228.991455:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305865167 00000100:00000001:1.0:1788880228.991457:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305860192 00000100:00000001:1.0:1788880228.991459:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305865167 00000100:00000001:1.0:1788880228.991461:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305860192 00000100:00000001:1.0:1788880228.991469:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305865167 00000100:00000001:1.0:1788880228.991471:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305860192 00000100:00000001:1.0:1788880228.991473:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305865167 00000100:00000001:1.0:1788880228.991474:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880228.991476:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880228.991479:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880228.991481:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880228.991485:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880228.991487:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:2.0:1788880228.991489:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880228.991490:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880228.991493:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000010:2.0:1788880228.991494:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000100:00000001:1.0:1788880228.991496:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880228.991503:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880228.991504:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880228.991513:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880228.991516:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000fbbca8a 00000400:00000010:2.0:1788880228.991519:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000fbbca8a. 00000100:00000001:2.0:1788880228.991522:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880228.991526:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880228.991547:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880228.991550:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880228.991554:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880228.991555:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880228.991567:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880228.991569:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:2.0:1788880228.991571:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880228.991573:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880228.991574:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000487fae38 00000400:00000010:2.0:1788880228.991575:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000487fae38. 00000100:00000001:2.0:1788880228.991577:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880228.991579:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880228.991583:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880228.991584:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880228.991587:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880228.991588:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880228.993031:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880228.993146:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880228.993150:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880228.993153:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880228.993162:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880228.993176:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b02500 00000400:00000200:3.0:1788880228.993304:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x375 [1] + 224 00000800:00000001:3.0:1788880228.993313:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880228.993318:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880228.993321:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880228.993325:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880228.993333:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880228.993339:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880228.993396:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880228.993416:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880228.993427:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880228.993431:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880228.993433:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880228.993436:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880228.993441:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880228.993450:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880228.993450:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02580 00000100:00000001:1.0:1788880228.993455:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880228.993460:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x37d [1] + 224 00000800:00000001:3.0:1788880228.993465:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880228.993468:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788880228.993470:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880228.993472:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880228.993474:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007e36d068. 00000400:00000200:3.0:1788880228.993474:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880228.993480:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000400:00000200:3.0:1788880228.993483:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:1.0:1788880228.993484:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:1.0:1788880228.993489:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880228.993491:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880228.993494:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880228.993499:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880228.993507:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880228.993522:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880228.993523:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880228.993525:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880228.993527:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880228.993529:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880228.993531:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880228.993533:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880228.993537:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880228.993546:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=56 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880228.993553:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=56 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880228.993556:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880228.993563:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880228.993569:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880228.993570:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880228.993575:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:3.0:1788880228.993575:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880228.993580:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880228.993580:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880228.993584:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880228.993585:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880228.993592:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880228.993596:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880228.993604:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880228.993608:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:3.0:1788880228.993608:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02600 00000100:00000001:1.0:1788880228.993611:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880228.993616:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880228.993620:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x385 [1] + 224 00000100:00100000:1.0:1788880228.993627:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780641536:192.168.204.111@tcp:400: 00000800:00000001:3.0:1788880228.993627:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880228.993634:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880228.993635:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880228.993638:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880228.993641:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000884e74b8 x1875776780641536/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880228.993646:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880228.993655:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000200:3.0:1788880228.993655:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:1.0:1788880228.993660:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880228.993661:0:201280:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000010:1.0:1788880228.993664:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007d30b075. 00000100:00000200:3.0:1788880228.993666:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880228.993671:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880228.993675:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880228.993678:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000040:3.0:1788880228.993679:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788880228.993683:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880228.993687:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e53a8f34. 02000000:00000001:1.0:1788880228.993693:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880228.993696:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880228.993698:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000884e74b8. 00000100:00000001:1.0:1788880228.993702:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880228.993703:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880228.993705:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880228.993707:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880228.993710:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880228.993714:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880228.993714:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880228.993719:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880228.993719:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880228.993721:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880228.993750:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880228.993754:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880228.993758:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880228.993759:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880228.993761:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b43de041. 00000400:00000200:2.0:1788880228.993765:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:2.0:1788880228.993767:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:2.0:1788880228.993772:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880228.993776:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880228.993777:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880228.993781:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880228.993785:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880228.993787:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880228.993790:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000100:00000200:2.0:1788880228.993792:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880228.993794:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000010:0.0:1788880228.993797:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880228.993803:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880228.993805:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:0.0:1788880228.993809:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880228.993813:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880228.993817:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880228.993819:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880228.993821:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880228.993824:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880228.993827:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880228.993827:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880228.993833:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=56 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880228.993838:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=30 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880228.993840:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880228.993841:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880228.993842:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880228.993846:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880228.993867:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:2.0:1788880228.993867:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880228.993870:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880228.993870:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880228.993873:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880228.993874:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880228.993876:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880228.993877:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880228.993878:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880228.993879:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880228.993880:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880228.993884:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880228.993884:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880228.993890:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=46 v=5 (5 0 0 0) 00000100:00000001:2.0:1788880228.993891:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880228.993896:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00001000:0.0:1788880228.993897:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=46 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880228.993898:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880228.993903:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880228.993905:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880228.993905:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880228.993909:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880228.993912:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880228.993917:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00100000:2.0:1788880228.993917:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780641792:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880228.993919:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880228.993922:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880228.993924:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880228.993924:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880228.993928:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880228.993931:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880228.993932:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000629edf55 x1875776780641792/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880228.993949:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880228.993950:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880228.993965:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880228.993969:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000010:2.0:1788880228.993970:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002b77e71a. 00000100:00000001:0.0:1788880228.993971:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880228.993974:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880228.993976:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:0.0:1788880228.993977:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:2.0:1788880228.993978:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880228.993982:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880228.993984:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 02000000:00000001:2.0:1788880228.993988:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880228.993990:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00100000:0.0:1788880228.993991:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780641664:192.168.204.111@tcp:400: 00000100:00000010:2.0:1788880228.993992:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000629edf55. 00000100:00000001:2.0:1788880228.993997:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880228.993998:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880228.993999:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880228.994001:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880228.994002:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880228.994005:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cc13c26b x1875776780641664/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880244 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880228.994007:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880228.994009:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880228.994013:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880228.994014:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880228.994015:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880228.994016:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880228.994024:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000010b36294. 02000000:00000001:0.0:1788880228.994028:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880228.994030:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880228.994031:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880228.994033:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880228.994035:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 02000000:00000001:0.0:1788880228.994037:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880228.994039:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880228.994040:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cc13c26b. 00000100:00000001:0.0:1788880228.994043:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880228.994044:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880228.994046:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880228.994050:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880228.994051:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880228.994052:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880228.994054:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880229.054378:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880229.054388:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880229.054395:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880229.054396:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880229.054464:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880229.054469:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880229.054472:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880230.078811:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880230.078823:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880230.078832:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880230.078833:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880230.078961:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880230.078965:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880230.078969:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880231.102391:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880231.102402:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880231.102411:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880231.102412:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880231.102524:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880231.102528:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880231.102532:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880232.126366:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880232.126377:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880232.126386:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880232.126387:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880232.126457:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880232.126462:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880232.126465:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880233.150420:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880233.150426:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880233.150430:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880233.150431:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880233.150481:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880233.150486:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880233.150489:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880234.110334:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305865167, at 4305865216 00000100:00000040:1.0:1788880234.110334:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305865167, at 4305865216 00000100:00000040:2.0:1788880234.110347:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305865167, at 4305865216 00000100:00000010:1.0:1788880234.110352:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000016ab1a15. 00000100:00000010:0.0:1788880234.110354:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fbefa178. 00000100:00000010:2.0:1788880234.110356:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000945234cf. 00000100:00000040:3.0:1788880234.110373:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305865167, at 4305865216 00000100:00000040:2.0:1788880234.110376:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:3.0:1788880234.110379:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a5ad6403. 00000100:00000001:2.0:1788880234.110382:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880234.110388:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000629edf55. 00000020:00000040:2.0:1788880234.110392:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880234.110397:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880234.110400:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880234.110404:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880234.110410:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 00000100:00000001:2.0:1788880234.110418:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880234.110426:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880234.110433:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880234.110435:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880234.110439:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000629edf55] to pc [ptlrpcd_00_03+3] req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880234.110460:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880234.110474:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880234.110478:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880234.110479:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:2.0:1788880234.110480:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000005a8bb1de. 00000100:00000001:1.0:1788880234.110482:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:2.0:1788880234.110482:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880234.110484:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880234.110484:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880234.110486:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880234.110487:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880234.110489:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880234.110490:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 00000100:00000001:2.0:1788880234.110494:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880234.110496:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880234.110497:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880234.110498:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880234.110502:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880234.110503:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880234.110504:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880234.110505:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880234.110507:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880234.110507:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a8bb1de] to pc [ptlrpcd_00_00+0] req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880234.110510:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780641920:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880234.110514:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880234.110520:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880234.110528:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880234.110528:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880234.110530:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880234.110530:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880234.110533:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000010:1.0:1788880234.110534:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000790fa4ee. 02000000:00000001:1.0:1788880234.110535:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880234.110535:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000910cb22c. 00000020:00000040:2.0:1788880234.110538:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880234.110540:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:1.0:1788880234.110541:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000020fda082. 02000000:00000001:2.0:1788880234.110541:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880234.110543:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880234.110545:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:0.0:1788880234.110547:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880234.110547:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880234.110548:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000001:0.0:1788880234.110550:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880234.110552:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880234.110553:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000200:1.0:1788880234.110555:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780641920, portal 25 00000100:00000040:0.0:1788880234.110558:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880234.110559:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880234.110561:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938432768 : -126823771118848 : ffff8ca78a2c6300) 00000100:00000001:2.0:1788880234.110562:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880234.110563:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880234.110568:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880250 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880234.110568:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000910cb22c] to pc [ptlrpcd_00_01+1] req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880234.110579:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880234.110584:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880234.110590:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880234.110592:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880234.110592:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880234.110597:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:1.0:1788880234.110598:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880234.110600:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788880234.110602:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000487fae38. 02000000:00000001:0.0:1788880234.110605:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880234.110606:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780641920, offset 0 00000100:00000001:2.0:1788880234.110609:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880234.110610:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780642048:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880234.110612:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880234.110612:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880234.110615:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880234.110617:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880234.110618:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880234.110619:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:0.0:1788880234.110620:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880234.110621:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880234.110624:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11202) 02000000:00000001:0.0:1788880234.110625:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880234.110630:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880234.110631:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880234.110634:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000010:0.0:1788880234.110637:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000482afcf7. 02000000:00000001:0.0:1788880234.110642:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880234.110644:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880234.110650:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:0.0:1788880234.110651:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000400:00000200:1.0:1788880234.110652:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880234.110654:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880234.110658:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000400:00000200:1.0:1788880234.110659:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:0.0:1788880234.110663:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780642048, portal 10 02000000:00000001:3.0:1788880234.110663:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880234.110666:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880234.110667:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880234.110668:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880234.110671:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938427392 : -126823771124224 : ffff8ca78a2c4e00) 02000000:00000001:3.0:1788880234.110673:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880234.110674:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [85:85:256:4294967295] 192.168.204.111@tcp LPNI seq info [85:85:8:4294967295] 00000100:00100000:3.0:1788880234.110681:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780642176:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880234.110683:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880234.110690:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788880234.110694:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:3.0:1788880234.110696:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880234.110699:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880234.110699:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880234.110703:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880234.110706:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880234.110706:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000010:3.0:1788880234.110707:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000046cece22. 00000400:00000010:0.0:1788880234.110709:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002aa6ba25. 02000000:00000001:3.0:1788880234.110711:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880234.110712:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000200:0.0:1788880234.110713:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780642048, offset 0 00000400:00000010:3.0:1788880234.110716:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000977bf705. 00000800:00000200:1.0:1788880234.110722:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:3.0:1788880234.110722:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 00000400:00000200:0.0:1788880234.110726:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880234.110731:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:3.0:1788880234.110733:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780642176, portal 10 00000400:00000200:0.0:1788880234.110735:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880234.110735:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880234.110736:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880234.110738:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938437248 : -126823771114368 : ffff8ca78a2c7480) 00000400:00000200:0.0:1788880234.110745:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880234.110747:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880234.110750:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880234.110751:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880234.110758:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880234.110759:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305870287 00000100:00000001:3.0:1788880234.110760:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788880234.110764:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:3.0:1788880234.110766:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880234.110767:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000400:00000010:3.0:1788880234.110770:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000003539bb5a. 00000400:00000200:0.0:1788880234.110771:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [86:86:255:4294967295] 192.168.204.111@tcp LPNI seq info [86:86:7:4294967295] 00000100:00000200:3.0:1788880234.110772:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780642176, offset 0 00000100:00000001:1.0:1788880234.110781:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:3.0:1788880234.110781:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880234.110790:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880234.110790:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305870287 00000100:00000010:1.0:1788880234.110794:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000100:00000001:1.0:1788880234.110797:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880234.110802:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305870287 00000100:00000010:1.0:1788880234.110807:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000800:00000200:0.0:1788880234.110809:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880234.110811:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880234.110813:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000010:0.0:1788880234.110816:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000100:00000001:1.0:1788880234.110819:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305870287 00000400:00000200:3.0:1788880234.110819:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1788880234.110826:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:1.0:1788880234.110826:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000400:00000200:3.0:1788880234.110827:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880234.110832:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880234.110833:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788880234.110834:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880234.110837:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:2.0:1788880234.110837:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000800:00000200:0.0:1788880234.110838:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880234.110842:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880234.110842:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788880234.110844:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880234.110847:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880234.110848:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880234.110850:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880234.110852:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880234.110854:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1337455 usecs (1338 jiffies) 00000400:00000200:3.0:1788880234.110854:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [87:87:254:4294967295] 192.168.204.111@tcp LPNI seq info [87:87:6:4294967295] 00000100:00000001:0.0:1788880234.110870:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305866554 00000100:00000001:1.0:1788880234.110872:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880234.110872:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880234.110874:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305870287 00000100:00000001:1.0:1788880234.110875:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880234.110879:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305866554 00000100:00000001:0.0:1788880234.110882:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305870287 00000100:00000001:1.0:1788880234.110882:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880234.110886:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880234.110886:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880234.110887:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305866554 00000100:00000001:1.0:1788880234.110889:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880234.110892:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305870287 00000100:00000001:1.0:1788880234.110894:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880234.110897:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305866554 00000400:00000200:2.0:1788880234.110899:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000487fae38 00000800:00000200:3.0:1788880234.110900:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880234.110902:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305870287 00000400:00000010:2.0:1788880234.110903:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000487fae38. 00000800:00000010:3.0:1788880234.110907:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000749ed09e. 00000100:00000001:0.0:1788880234.110908:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880234.110911:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880234.110913:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880234.110916:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880234.110919:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880234.110919:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880250 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880234.110925:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:3.0:1788880234.110926:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880234.110932:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000749ed09e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880234.110933:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880234.110936:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880234.110941:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880234.110942:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880234.110943:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880234.110946:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880234.110948:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880250 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880234.110950:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880234.110960:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880234.110962:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880234.110976:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880234.110980:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 88553 usecs (89 jiffies) 00000800:00000200:2.0:1788880234.110982:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880234.110983:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305865305 00000800:00000010:2.0:1788880234.110986:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f92e72a2. 00000100:00000001:3.0:1788880234.110987:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305870287 00000400:00000200:2.0:1788880234.110992:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880234.110992:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305865305 00000100:00000001:3.0:1788880234.110995:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305870287 00000400:00000200:2.0:1788880234.110999:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880234.111000:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305865305 00000400:00000200:2.0:1788880234.111002:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002aa6ba25 00000400:00000010:2.0:1788880234.111004:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002aa6ba25. 00000100:00000001:2.0:1788880234.111006:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880234.111010:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880234.111010:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305870287 00000100:00000001:3.0:1788880234.111015:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305865305 00000100:00000001:3.0:1788880234.111019:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305870287 00000100:00000001:2.0:1788880234.111023:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880234.111023:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880234.111027:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880234.111029:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880234.111031:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880234.111035:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880234.111042:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880234.111044:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880234.111046:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880234.111048:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880234.111050:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880234.111054:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880234.111063:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880234.111065:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880234.111102:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880234.111105:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000749ed09e. 00000400:00000200:2.0:1788880234.111109:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880234.111112:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880234.111115:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000003539bb5a 00000400:00000010:2.0:1788880234.111117:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000003539bb5a. 00000100:00000001:2.0:1788880234.111119:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880234.111122:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880234.111129:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880234.111131:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880234.111137:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880234.111138:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880234.112190:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880234.112202:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880234.112205:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880234.112208:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880234.112215:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880234.112226:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b02680 00000400:00000200:2.0:1788880234.112255:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x38d [1] + 224 00000800:00000001:2.0:1788880234.112261:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880234.112267:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880234.112269:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880234.112272:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880234.112276:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880234.112281:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880234.112291:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880234.112316:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880234.112324:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880234.112327:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880234.112358:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880234.112361:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880234.112366:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880234.112368:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880234.112369:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000020fda082. 00000400:00000200:1.0:1788880234.112373:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:1.0:1788880234.112374:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:1.0:1788880234.112377:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880234.112382:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880234.112390:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880234.112397:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880234.112404:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880234.112406:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880234.112409:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880234.112411:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880234.112413:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880234.112416:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880234.112422:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=62 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880234.112426:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=62 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880234.112429:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880234.112430:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880234.112432:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880234.112434:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880234.112437:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880234.112441:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880234.112447:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880234.112449:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880234.112451:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880234.112454:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880234.112461:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780641920:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880234.112466:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880234.112468:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880234.112470:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000629edf55 x1875776780641920/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880234.112476:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880234.112478:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880234.112480:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000790fa4ee. 02000000:00000001:1.0:1788880234.112483:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880234.112485:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880234.112495:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880234.112498:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880234.112500:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 02000000:00000001:1.0:1788880234.112504:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880234.112511:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880234.112512:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000629edf55. 00000100:00000001:1.0:1788880234.112516:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880234.112517:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880234.112519:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880234.112524:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880234.112525:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880234.112527:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880234.112528:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880234.113328:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880234.113335:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880234.113338:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880234.113340:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880234.113452:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880234.113462:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02700 00000400:00000200:3.0:1788880234.113467:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x395 [1] + 224 00000800:00000001:3.0:1788880234.113471:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880234.113475:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880234.113477:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880234.113517:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880234.113521:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880234.113526:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880234.113535:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880234.113553:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880234.113558:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880234.113561:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880234.113567:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880234.113569:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880234.113572:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880234.113577:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02780 00000400:00000200:3.0:1788880234.113580:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x39d [1] + 224 00000800:00000001:3.0:1788880234.113583:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880234.113587:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880234.113587:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880234.113590:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880234.113591:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880234.113595:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:0.0:1788880234.113596:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880234.113599:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880234.113600:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000400:00000200:0.0:1788880234.113605:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000010:0.0:1788880234.113623:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880234.113627:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880234.113629:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880234.113636:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880234.113636:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880234.113663:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880234.113667:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880234.113674:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880234.113676:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880234.113680:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880234.113683:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880234.113684:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880234.113685:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880234.113686:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880234.113691:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=52 v=5 (5 0 0 0) 00000100:00000001:3.0:1788880234.113692:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:0.0:1788880234.113694:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=52 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880234.113697:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880234.113698:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880234.113699:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:0.0:1788880234.113700:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880234.113703:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880234.113703:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880234.113705:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880234.113708:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880234.113710:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880234.113711:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880234.113714:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880234.113718:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880234.113720:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880234.113721:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880234.113724:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880234.113728:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788880234.113729:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780642048:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880234.113733:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880234.113734:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880234.113738:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880234.113742:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000001:1.0:1788880234.113743:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:0.0:1788880234.113747:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000005a8bb1de x1875776780642048/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880234.113747:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000977bf705. 00000400:00000200:1.0:1788880234.113753:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 00000100:00000001:0.0:1788880234.113754:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880234.113758:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000010:1.0:1788880234.113758:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 02000000:00000010:0.0:1788880234.113760:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000482afcf7. 02000000:00000001:0.0:1788880234.113764:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880234.113765:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880234.113765:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000040:0.0:1788880234.113767:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880234.113769:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880234.113770:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 00000100:00000200:1.0:1788880234.113772:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880234.113773:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880234.113776:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880234.113777:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000005a8bb1de. 00000100:00000001:0.0:1788880234.113781:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880234.113782:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880234.113785:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880234.113789:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880234.113790:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880234.113793:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880234.113795:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880234.113799:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880234.113808:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880234.113809:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880234.113811:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880234.113813:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880234.113814:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880234.113815:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880234.113817:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880234.113821:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=62 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880234.113824:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=36 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880234.113826:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880234.113827:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880234.113829:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880234.113832:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880234.113838:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880234.113847:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880234.113848:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880234.113850:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880234.113851:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880234.113854:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880234.113873:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880234.113874:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880234.113876:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880234.113879:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880234.113885:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780642176:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880234.113889:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880234.113891:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880234.113893:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000910cb22c x1875776780642176/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880250 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880234.113898:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880234.113900:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880234.113902:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000046cece22. 02000000:00000001:1.0:1788880234.113905:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880234.113906:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880234.113907:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880234.113909:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880234.113910:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 02000000:00000001:1.0:1788880234.113912:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880234.113913:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880234.113914:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000910cb22c. 00000100:00000001:1.0:1788880234.113917:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880234.113917:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880234.113919:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880234.113922:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880234.113923:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880234.113924:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880234.113931:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880234.174382:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880234.174391:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880234.174393:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880234.174396:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880234.174401:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880234.174403:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880234.174405:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880234.174407:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880234.174410:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880234.174412:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880234.174467:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880234.174470:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880234.174472:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880235.199317:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880235.199326:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880235.199334:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880235.199338:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880235.199341:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880235.199346:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880235.199350:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880235.199353:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880235.199355:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880235.199359:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880235.199636:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880235.199642:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880235.199646:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880236.222309:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880236.222314:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880236.222320:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880236.222321:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880236.222485:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880236.222490:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880236.222492:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880237.246556:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880237.246564:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880237.246569:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880237.246570:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880237.246730:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880237.246734:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880237.246737:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880238.270334:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880238.270340:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880238.270344:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880238.270345:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880238.270400:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880238.270405:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880238.270408:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880239.230325:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305870287, at 4305870336 00000100:00000040:2.0:1788880239.230352:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305870287, at 4305870336 00000100:00000040:1.0:1788880239.230359:0:192984:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305870287, at 4305870336 00000100:00000010:0.0:1788880239.230361:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000010:2.0:1788880239.230362:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:1.0:1788880239.230370:0:192984:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000040:3.0:1788880239.230383:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880239.230388:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880239.230393:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880239.230398:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880239.230402:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880239.230405:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880239.230409:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880239.230416:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:3.0:1788880239.230423:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880239.230434:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880239.230445:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880239.230448:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880239.230454:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_02+2] req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880239.230481:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880239.230482:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880239.230485:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880239.230486:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000020:00000040:3.0:1788880239.230488:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880239.230489:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880239.230490:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880239.230492:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880239.230494:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:3.0:1788880239.230497:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880239.230500:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880239.230501:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788880239.230506:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:1.0:1788880239.230510:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880239.230512:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880239.230513:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880239.230515:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880239.230517:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880239.230525:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_03+3] req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880239.230527:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880239.230532:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880239.230537:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880239.230540:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880239.230542:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880239.230543:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880239.230547:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880239.230549:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880239.230550:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880239.230554:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880239.230554:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880239.230560:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000100:00000001:1.0:1788880239.230565:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000020:00000040:3.0:1788880239.230565:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880239.230570:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880239.230571:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880239.230574:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880239.230576:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880239.230576:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880239.230580:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000010:3.0:1788880239.230581:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 02000000:00000001:1.0:1788880239.230585:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880239.230586:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880239.230590:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780642432:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880239.230593:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880239.230598:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880239.230604:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:1.0:1788880239.230606:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880239.230610:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880239.230613:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880239.230623:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880239.230629:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000070c1ee08. 02000000:00000001:1.0:1788880239.230633:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880239.230633:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_00+0] req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880239.230641:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000020fda082. 00000400:00000010:1.0:1788880239.230650:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000200:1.0:1788880239.230657:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780642432, portal 10 00000100:00000001:1.0:1788880239.230659:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880239.230661:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000100:00000001:3.0:1788880239.230663:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880239.230667:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880239.230669:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880239.230674:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1788880239.230681:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11208) 00000100:00000001:0.0:1788880239.230683:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880239.230689:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880239.230689:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880239.230693:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880239.230694:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880239.230694:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880239.230698:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880239.230699:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000ae22117. 00000100:00000001:2.0:1788880239.230700:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880239.230700:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305870287, at 4305870336 00000100:00000200:1.0:1788880239.230705:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780642432, offset 0 00000100:00000010:3.0:1788880239.230709:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000001:0.0:1788880239.230717:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1788880239.230718:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880239.230721:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880239.230725:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880239.230727:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880239.230729:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880239.230732:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780642560:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880239.230734:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880239.230739:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880239.230744:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1788880239.230745:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788880239.230746:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880239.230748:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880239.230752:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000060f4a118. 00000400:00000200:1.0:1788880239.230756:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880239.230757:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880239.230773:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000400:00000200:1.0:1788880239.230775:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:0.0:1788880239.230779:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000400:00000200:1.0:1788880239.230780:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:0.0:1788880239.230786:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780642560, portal 10 00000400:00000200:1.0:1788880239.230787:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [88:88:256:4294967295] 192.168.204.111@tcp LPNI seq info [88:88:8:4294967295] 00000100:00000001:0.0:1788880239.230790:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880239.230795:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000100:00000040:0.0:1788880239.230803:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880239.230810:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880239.230824:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:1.0:1788880239.230825:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880239.230831:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880239.230833:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000400:00000010:0.0:1788880239.230836:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e73139ca. 00000100:00000200:0.0:1788880239.230841:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780642560, offset 0 00000800:00000200:1.0:1788880239.230844:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880239.230849:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880239.230856:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880239.230859:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880239.230862:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880239.230864:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880239.230876:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880239.230878:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880239.230881:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880239.230884:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880239.230890:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1270780 usecs (1271 jiffies) 00000400:00000200:0.0:1788880239.230893:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [89:89:255:4294967295] 192.168.204.111@tcp LPNI seq info [89:89:7:4294967295] 00000100:00000001:1.0:1788880239.230894:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305871607 00000100:00000010:1.0:1788880239.230900:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000100:00000001:1.0:1788880239.230906:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000100:00000001:1.0:1788880239.230913:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305871607 00000400:00000200:0.0:1788880239.230916:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:1.0:1788880239.230917:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000100:00000001:1.0:1788880239.230923:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880239.230926:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305871607 00000100:00000010:1.0:1788880239.230928:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000800:00000200:0.0:1788880239.230929:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880239.230932:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000800:00000010:0.0:1788880239.230936:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000088d6bc1. 00000100:00000001:1.0:1788880239.230937:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305871607 00000100:00000010:1.0:1788880239.230943:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000800:00000200:0.0:1788880239.230946:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880239.230948:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880239.230953:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880239.230956:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880239.230958:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880239.230962:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000088d6bc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880239.230963:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880239.230969:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880239.230969:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880239.230974:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 83268 usecs (84 jiffies) 00000100:00000001:0.0:1788880239.230981:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305870420 00000100:00000001:0.0:1788880239.230983:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305871607 00000100:00000001:0.0:1788880239.230985:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305870420 00000100:00000001:0.0:1788880239.230987:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305871607 00000100:00000001:0.0:1788880239.230988:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305870420 00000100:00000001:0.0:1788880239.230990:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305871607 00000100:00000001:0.0:1788880239.230994:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305870420 00000100:00000001:0.0:1788880239.230996:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305871607 00000100:00000001:0.0:1788880239.230998:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880239.231002:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880239.231007:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880239.231011:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880239.231014:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880239.231017:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880239.231018:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880239.231021:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000001:1.0:1788880239.231022:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880239.231025:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880239.231026:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880239.231029:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880239.231031:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880239.231032:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880239.231037:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780642304:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880239.231041:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880239.231044:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880239.231045:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880239.231048:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880239.231050:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880239.231054:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a07dd7e3. 02000000:00000001:1.0:1788880239.231056:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880239.231060:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008388b815. 00000400:00000010:1.0:1788880239.231063:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:1.0:1788880239.231066:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780642304, portal 25 00000100:00000001:1.0:1788880239.231067:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880239.231068:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000100:00000040:1.0:1788880239.231072:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880255 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880239.231077:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880239.231079:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880239.231081:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000030b8c48c. 00000100:00000200:1.0:1788880239.231083:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780642304, offset 0 00000400:00000200:1.0:1788880239.231086:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880239.231091:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880239.231094:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880239.231097:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880239.231099:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880239.231101:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880239.231104:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [90:90:254:4294967295] 192.168.204.111@tcp LPNI seq info [90:90:6:4294967295] 00000400:00000200:1.0:1788880239.231110:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880239.231127:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880239.231131:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000800:00000200:1.0:1788880239.231135:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880239.231139:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880239.231142:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880239.231144:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880239.231146:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305875407 00000100:00000001:1.0:1788880239.231151:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305871607 00000100:00000001:1.0:1788880239.231154:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305875407 00000100:00000001:1.0:1788880239.231156:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305871607 00000100:00000001:1.0:1788880239.231158:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305875407 00000100:00000001:1.0:1788880239.231160:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305871607 00000100:00000001:1.0:1788880239.231162:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305875407 00000100:00000001:1.0:1788880239.231164:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305871607 00000100:00000001:1.0:1788880239.231166:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880239.231168:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880239.231170:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880239.231173:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880239.231175:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880239.231177:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880239.231178:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880239.231179:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880239.231181:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880239.231183:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880239.231306:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880239.231315:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:2.0:1788880239.231324:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880239.231334:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880239.231343:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000ae22117 00000400:00000010:2.0:1788880239.231346:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000ae22117. 00000100:00000001:2.0:1788880239.231353:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880239.231359:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880239.231371:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880239.231374:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880239.231382:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880239.231384:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880239.231399:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880239.231401:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000088d6bc1. 00000400:00000200:2.0:1788880239.231404:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880239.231407:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880239.231409:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e73139ca 00000400:00000010:2.0:1788880239.231410:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e73139ca. 00000100:00000001:2.0:1788880239.231412:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880239.231414:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880239.231430:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880239.231432:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880239.231436:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880239.231437:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880239.231498:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880239.231500:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:2.0:1788880239.231502:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880239.231504:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880239.231506:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000030b8c48c 00000400:00000010:2.0:1788880239.231507:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000030b8c48c. 00000100:00000001:2.0:1788880239.231508:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880239.231511:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880255 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880239.231515:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880239.231516:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880255 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880239.231520:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880239.231521:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880239.233141:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880239.233153:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880239.233156:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880239.233157:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880239.233165:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880239.233178:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02900 00000400:00000200:2.0:1788880239.233185:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x3ad [1] + 224 00000800:00000001:2.0:1788880239.233191:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880239.233195:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880239.233197:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880239.233201:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880239.233208:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880239.233225:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880239.233268:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880239.233276:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880239.233284:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880239.233288:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880239.233289:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880239.233290:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880239.233295:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880239.233300:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02880 00000100:00000001:0.0:1788880239.233301:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880239.233305:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x3a5 [1] + 224 00000100:00000001:0.0:1788880239.233306:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880239.233310:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880239.233315:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880239.233316:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880239.233319:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880239.233322:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000400:00000200:2.0:1788880239.233325:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880239.233327:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000010:0.0:1788880239.233332:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000400:00000200:2.0:1788880239.233332:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880239.233338:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880239.233339:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880239.233347:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880239.233348:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880239.233363:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880239.233372:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880239.233374:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880239.233376:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880239.233380:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880239.233384:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880239.233386:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880239.233388:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880239.233391:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880239.233393:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880239.233398:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880239.233399:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=67 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880239.233404:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=41 v=5 (1 0 0 0) 00000800:00000001:2.0:1788880239.233404:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880239.233406:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880239.233408:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880239.233409:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880239.233411:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880239.233414:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880239.233414:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880239.233415:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00010000:00000001:0.0:1788880239.233418:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000001:1.0:1788880239.233418:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880239.233421:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000020fda082. 00010000:00080000:0.0:1788880239.233425:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880239.233426:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000200:2.0:1788880239.233426:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000010:1.0:1788880239.233430:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:1.0:1788880239.233435:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:0.0:1788880239.233438:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880239.233440:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b02800 00000100:00000001:0.0:1788880239.233445:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000200:1.0:1788880239.233446:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880239.233449:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000200:2.0:1788880239.233450:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x3b5 [1] + 224 00000100:00000001:0.0:1788880239.233454:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880239.233457:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880239.233459:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1788880239.233466:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880239.233467:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880239.233472:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880239.233476:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880239.233486:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880239.233495:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880239.233496:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880239.233499:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880239.233500:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880239.233502:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880239.233503:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880239.233506:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880239.233510:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880239.233511:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880239.233517:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=57 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880239.233524:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=57 v=5 (1 0 0 0) 00000100:00000040:2.0:1788880239.233529:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880239.233530:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880239.233534:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880239.233534:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880239.233538:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880239.233541:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00010000:00000001:1.0:1788880239.233542:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880239.233545:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880239.233546:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880239.233547:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880239.233550:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880239.233552:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000800:00000001:2.0:1788880239.233554:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1788880239.233555:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880239.233555:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880239.233558:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880239.233562:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880239.233582:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00100000:0.0:1788880239.233585:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780642560:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880239.233585:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880239.233587:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880239.233593:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880239.233594:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880239.233599:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00100000:1.0:1788880239.233602:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780642432:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880239.233606:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780642560/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880239.233608:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880239.233610:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880239.233612:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008388b815. 00000100:00000001:0.0:1788880239.233616:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880239.233620:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000200:1.0:1788880239.233621:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:1.0:1788880239.233624:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 02000000:00000010:0.0:1788880239.233625:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000060f4a118. 00000100:00000001:1.0:1788880239.233627:0:201288:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:0.0:1788880239.233631:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:1.0:1788880239.233634:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880239.233635:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880239.233638:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880239.233643:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880239.233646:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 02000000:00000001:0.0:1788880239.233650:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880239.233653:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880239.233656:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000200:1.0:1788880239.233657:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880239.233661:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880239.233664:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880239.233668:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880239.233672:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880239.233675:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880239.233676:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880239.233678:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788880239.233680:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880239.233682:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880239.233684:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880239.233684:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880239.233688:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880239.233689:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880239.233690:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880239.233696:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=67 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880239.233699:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=67 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880239.233701:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880239.233703:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880239.233705:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880239.233706:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880239.233708:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880239.233712:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880239.233718:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880239.233720:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880239.233721:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880239.233724:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880239.233731:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780642304:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880239.233735:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880239.233737:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880239.233739:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780642432/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880239.233756:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880239.233758:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880239.233772:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000070c1ee08. 02000000:00000001:1.0:1788880239.233776:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880239.233778:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880239.233779:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880239.233781:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880239.233782:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 02000000:00000001:1.0:1788880239.233785:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880239.233786:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880239.233787:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:1.0:1788880239.233790:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880239.233791:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880239.233793:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880239.233796:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780642304/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880255 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880239.233802:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880239.233803:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880239.233804:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a07dd7e3. 02000000:00000001:1.0:1788880239.233807:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880239.233808:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880239.233809:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880239.233811:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880239.233812:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 02000000:00000001:1.0:1788880239.233814:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880239.233815:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880239.233816:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:1.0:1788880239.233818:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880239.233819:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880239.233820:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880239.233824:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880239.233825:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880239.233827:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880239.233829:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880239.294372:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880239.294379:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880239.294409:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880239.294411:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880239.294464:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880239.294467:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880239.294470:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880240.318387:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880240.318397:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880240.318405:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880240.318406:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880240.318479:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880240.318507:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880240.318510:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880241.342349:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880241.342355:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880241.342361:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880241.342362:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880241.342415:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880241.342419:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880241.342421:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880242.366338:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880242.366346:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880242.366352:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880242.366353:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880242.366413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880242.366417:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880242.366419:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880243.390343:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880243.390352:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880243.390355:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880243.390359:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880243.390365:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880243.390368:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880243.390370:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880243.390374:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880243.390379:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880243.390381:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880243.390445:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880243.390449:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880243.390451:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880244.350397:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305875407, at 4305875456 00000100:00000040:3.0:1788880244.350398:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305875407, at 4305875456 00000100:00000040:2.0:1788880244.350399:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305875407, at 4305875456 00000100:00000040:1.0:1788880244.350411:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305875407, at 4305875456 00000100:00000010:0.0:1788880244.350424:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000010:2.0:1788880244.350428:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000010:3.0:1788880244.350435:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000010:1.0:1788880244.350439:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000040:1.0:1788880244.350460:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880244.350466:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880244.350475:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880244.350480:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880244.350485:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880244.350489:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880244.350494:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880244.350500:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:1.0:1788880244.350506:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880244.350516:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880244.350539:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880244.350541:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880244.350547:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000008b25f66] to pc [ptlrpcd_00_01+1] req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880244.350620:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788880244.350624:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:0.0:1788880244.350626:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880244.350641:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880244.350644:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880244.350645:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880244.350648:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880244.350652:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880244.350655:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001cf64abe. 00000020:00000040:1.0:1788880244.350659:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880244.350661:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880244.350676:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880244.350678:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880244.350682:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:1.0:1788880244.350685:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880244.350689:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880244.350696:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880244.350698:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880244.350701:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001cf64abe] to pc [ptlrpcd_00_02+2] req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880244.350732:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880244.350735:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880244.350738:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880244.350740:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000070db75b2. 00000020:00000040:1.0:1788880244.350742:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880244.350744:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880244.350745:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880244.350747:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880244.350750:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:1.0:1788880244.350757:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880244.350761:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880244.350767:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880244.350768:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880244.350770:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000070db75b2] to pc [ptlrpcd_00_03+3] req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880244.350778:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880244.350780:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880244.350783:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788880244.350788:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11213) 00000100:00000001:2.0:1788880244.350790:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880244.350794:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880244.350796:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880244.350803:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880244.350816:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880244.350819:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880244.350824:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880244.350825:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880244.350827:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880244.350830:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880244.350834:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880244.350835:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880244.350843:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880244.350845:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780642816:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880244.350846:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880244.350853:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880244.350860:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880244.350863:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880244.350867:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880244.350869:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880244.350871:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880244.350874:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:2.0:1788880244.350875:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880244.350878:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880244.350881:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880244.350881:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880244.350890:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cadd275. 02000000:00000001:2.0:1788880244.350895:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880244.350896:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880244.350904:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880244.350906:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:2.0:1788880244.350906:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 02000000:00000001:3.0:1788880244.350910:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880244.350912:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880244.350918:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00100000:3.0:1788880244.350920:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780642944:192.168.204.111@tcp:400: 02000000:00000001:0.0:1788880244.350923:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880244.350930:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000170fdf47. 00000100:00000001:3.0:1788880244.350933:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880244.350934:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880244.350940:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00100000:0.0:1788880244.350942:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780642688:192.168.204.111@tcp:400: 00000100:00000200:2.0:1788880244.350942:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780642816, portal 10 02000000:00000001:3.0:1788880244.350945:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880244.350951:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:3.0:1788880244.350954:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880244.350955:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938054144 : -126823771497472 : ffff8ca78a269c00) 00000100:00000001:0.0:1788880244.350956:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880244.350962:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000010:3.0:1788880244.350964:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008cdf44b0. 02000000:00000001:0.0:1788880244.350965:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880244.350974:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880244.350974:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880244.350975:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880244.350985:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000cf77ae66. 00000400:00000010:3.0:1788880244.350987:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000be05c49f. 00000100:00000001:2.0:1788880244.350996:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880244.350997:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880244.350997:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 00000100:00000040:2.0:1788880244.351005:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000200:3.0:1788880244.351007:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780642944, portal 10 00000400:00000010:0.0:1788880244.351011:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000400:00000010:2.0:1788880244.351011:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000005509e00. 00000100:00000200:2.0:1788880244.351018:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780642816, offset 0 00000400:00000010:0.0:1788880244.351020:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000001:3.0:1788880244.351028:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000200:0.0:1788880244.351030:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780642688, portal 25 00000100:00000001:3.0:1788880244.351031:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938046976 : -126823771504640 : ffff8ca78a268000) 00000400:00000200:2.0:1788880244.351036:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880244.351037:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880244.351041:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000040:3.0:1788880244.351047:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880244.351053:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880260 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880244.351059:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880244.351077:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880244.351090:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880244.351091:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880244.351100:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880244.351104:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880244.351105:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:3.0:1788880244.351110:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000040:0.0:1788880244.351113:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880244.351115:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:0.0:1788880244.351119:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ff62fd67. 00000100:00000200:0.0:1788880244.351127:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780642688, offset 0 00000100:00000200:3.0:1788880244.351127:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780642944, offset 0 00000400:00000200:2.0:1788880244.351128:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [91:91:256:4294967295] 192.168.204.111@tcp LPNI seq info [91:91:8:4294967295] 00000400:00000200:3.0:1788880244.351145:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880244.351155:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880244.351156:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880244.351167:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880244.351172:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880244.351173:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000400:00000200:3.0:1788880244.351179:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880244.351185:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880244.351186:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880244.351194:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880244.351198:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788880244.351205:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880244.351206:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [92:92:255:4294967295] 192.168.204.111@tcp LPNI seq info [92:92:7:4294967295] 00000800:00000200:2.0:1788880244.351218:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880244.351229:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880244.351272:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880244.351281:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880244.351282:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:2.0:1788880244.351282:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1210442 usecs (1211 jiffies) 00000100:00000001:2.0:1788880244.351293:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305876668 00000400:00000200:0.0:1788880244.351295:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:3.0:1788880244.351295:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000100:00000010:2.0:1788880244.351301:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a0959d87. 00000400:00000200:0.0:1788880244.351309:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880244.351309:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000800:00000200:3.0:1788880244.351313:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880244.351321:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880244.351323:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305876668 00000400:00000200:0.0:1788880244.351329:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:2.0:1788880244.351333:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ca0fac21. 00000800:00000200:3.0:1788880244.351334:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880244.351343:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880244.351344:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [93:93:254:4294967295] 192.168.204.111@tcp LPNI seq info [93:93:6:4294967295] 00000100:00000001:2.0:1788880244.351346:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880244.351357:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880244.351358:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305876668 00000100:00000010:2.0:1788880244.351366:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000043ac658c. 00000100:00000040:3.0:1788880244.351366:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 78572 usecs (79 jiffies) 00000800:00000200:1.0:1788880244.351370:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:0.0:1788880244.351371:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880244.351377:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880244.351379:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305875536 00000800:00000010:1.0:1788880244.351380:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000100:00000001:2.0:1788880244.351384:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305876668 00000100:00000001:3.0:1788880244.351389:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305876668 00000800:00000200:0.0:1788880244.351390:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880244.351391:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001e508d7a. 00000400:00000200:1.0:1788880244.351392:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880244.351401:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305875536 00000100:00000001:2.0:1788880244.351403:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000800:00000010:0.0:1788880244.351405:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000088d6bc1. 00000100:00000001:3.0:1788880244.351408:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305876668 00000100:00000001:2.0:1788880244.351410:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880244.351411:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880244.351419:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305875536 00000800:00000200:0.0:1788880244.351421:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880244.351425:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000005509e00 00000100:00000001:3.0:1788880244.351428:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305876668 00000100:00000001:2.0:1788880244.351429:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880244.351432:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000005509e00. 00000100:00000001:2.0:1788880244.351434:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880244.351438:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880244.351439:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880244.351439:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880244.351443:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305875536 00000100:00000001:2.0:1788880244.351447:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:0.0:1788880244.351449:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000088d6bc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000200:1.0:1788880244.351450:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880244.351456:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305876668 00000100:00000001:2.0:1788880244.351457:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880244.351459:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880244.351469:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880244.351470:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880244.351472:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305880528 00000100:00000001:2.0:1788880244.351475:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880244.351483:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880244.351484:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880244.351486:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880244.351487:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305876668 00000100:00000001:3.0:1788880244.351493:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880244.351494:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880244.351497:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305880528 00000100:00000040:1.0:1788880244.351500:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880244.351503:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880244.351510:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305876668 00000100:00000001:3.0:1788880244.351515:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880244.351516:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305880528 00000100:00000001:3.0:1788880244.351519:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880244.351523:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880244.351526:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880244.351529:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880244.351533:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305876668 00000100:00000001:3.0:1788880244.351538:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880244.351543:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305880528 00000100:00000001:1.0:1788880244.351543:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880244.351548:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880244.351550:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305876668 00000100:00000001:0.0:1788880244.351555:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880244.351557:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880244.351559:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880244.351562:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880244.351566:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880244.351570:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000400:00000200:1.0:1788880244.351575:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880244.351579:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880244.351582:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000400:00000010:1.0:1788880244.351590:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000001:1.0:1788880244.351593:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880244.351598:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880244.351607:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880244.351610:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880244.351617:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880244.351619:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880244.351696:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880244.351699:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000088d6bc1. 00000400:00000200:1.0:1788880244.351705:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880244.351711:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880244.351714:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ff62fd67 00000400:00000010:1.0:1788880244.351716:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ff62fd67. 00000100:00000001:1.0:1788880244.351719:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880244.351721:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880260 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880244.351727:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880244.351728:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880260 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880244.351734:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880244.351735:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880244.354609:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880244.354626:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880244.354631:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880244.354633:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880244.354644:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880244.354680:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02a00 00000400:00000200:2.0:1788880244.354689:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x3bd [1] + 224 00000800:00000001:2.0:1788880244.354695:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880244.354704:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880244.354707:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880244.354711:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880244.354718:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880244.354726:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880244.354748:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880244.354781:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880244.354790:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880244.354793:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880244.354896:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880244.354901:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880244.354908:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880244.354910:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880244.354913:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000400:00000200:3.0:1788880244.354917:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000170fdf47 00000400:00000010:3.0:1788880244.354919:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000170fdf47. 00000100:00000001:3.0:1788880244.354923:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880244.354927:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880244.354936:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880244.354942:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880244.354944:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880244.354946:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880244.354948:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880244.354950:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880244.354951:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880244.354954:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880244.354963:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=62 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880244.354967:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=62 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880244.354970:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880244.354971:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880244.354980:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880244.354984:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880244.354987:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880244.354989:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880244.354991:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880244.354993:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880244.354998:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880244.355005:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880244.355007:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880244.355009:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880244.355012:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880244.355016:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780642816:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880244.355019:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880244.355021:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880244.355023:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000001cf64abe x1875776780642816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880244.355027:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880244.355030:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880244.355033:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cadd275. 02000000:00000001:3.0:1788880244.355037:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880244.355039:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880244.355041:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880244.355043:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880244.355045:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:3.0:1788880244.355048:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880244.355049:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880244.355051:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000001cf64abe. 00000100:00000001:3.0:1788880244.355054:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880244.355055:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880244.355057:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880244.355072:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880244.355081:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880244.355084:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880244.355086:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880244.355812:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880244.355826:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880244.355830:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880244.355832:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880244.355910:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880244.355927:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02a80 00000400:00000200:2.0:1788880244.355941:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x3c1 [1] + 224 00000800:00000001:2.0:1788880244.355948:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880244.355953:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880244.355956:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880244.355960:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880244.355967:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880244.355974:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880244.355985:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880244.356002:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880244.356012:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880244.356015:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880244.356016:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880244.356017:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880244.356022:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880244.356028:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b02980 00000400:00000200:2.0:1788880244.356033:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x3c9 [1] + 224 00000800:00000001:2.0:1788880244.356036:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880244.356042:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880244.356042:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880244.356043:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880244.356048:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880244.356057:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880244.356058:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:3.0:1788880244.356060:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880244.356064:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000be05c49f. 00000400:00000200:3.0:1788880244.356069:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 00000400:00000010:3.0:1788880244.356071:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000100:00000001:3.0:1788880244.356075:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880244.356076:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880244.356081:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880244.356082:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880244.356111:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880244.356115:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880244.356125:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880244.356127:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880244.356129:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880244.356131:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:3.0:1788880244.356132:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880244.356136:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880244.356137:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880244.356139:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880244.356141:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880244.356143:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880244.356153:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=72 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880244.356157:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=46 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880244.356159:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880244.356161:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880244.356163:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880244.356167:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880244.356173:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880244.356177:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880244.356178:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880244.356180:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880244.356181:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880244.356190:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880244.356204:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880244.356207:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880244.356209:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880244.356214:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880244.356214:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880244.356219:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880244.356223:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880244.356225:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880244.356226:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000400:00000200:0.0:1788880244.356229:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000100:00100000:3.0:1788880244.356273:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780642944:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880244.356279:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880244.356281:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880244.356283:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000070db75b2 x1875776780642944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880244.356288:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880244.356290:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880244.356292:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008cdf44b0. 02000000:00000001:3.0:1788880244.356295:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880244.356297:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880244.356299:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880244.356301:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880244.356304:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:3.0:1788880244.356308:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880244.356310:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880244.356312:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000070db75b2. 00000100:00000001:3.0:1788880244.356315:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880244.356316:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880244.356318:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880244.356326:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:0.0:1788880244.356328:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:3.0:1788880244.356329:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880244.356334:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880244.356334:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880244.356339:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880244.356340:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880244.356347:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880244.356353:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880244.356355:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880244.356356:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880244.356358:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880244.356359:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880244.356360:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880244.356361:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880244.356363:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=72 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880244.356366:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=72 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880244.356367:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880244.356368:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880244.356369:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880244.356370:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880244.356372:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880244.356375:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880244.356388:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880244.356390:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880244.356391:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880244.356394:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880244.356401:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780642688:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880244.356405:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880244.356407:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880244.356409:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780642688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880260 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880244.356414:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880244.356416:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880244.356418:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000cf77ae66. 02000000:00000001:0.0:1788880244.356421:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880244.356422:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880244.356423:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880244.356425:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880244.356426:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 02000000:00000001:0.0:1788880244.356429:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880244.356429:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880244.356431:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:0.0:1788880244.356433:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880244.356434:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880244.356436:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880244.356440:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880244.356442:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880244.356443:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880244.356445:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880244.414382:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880244.414391:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880244.414398:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880244.414399:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880244.414487:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880244.414494:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880244.414499:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880245.438388:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880245.438470:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880245.438478:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880245.438482:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880245.438484:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880245.438488:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880245.438491:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880245.438522:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880245.438524:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880245.438526:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880245.438691:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880245.438699:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880245.438721:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880246.462387:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880246.462397:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880246.462405:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880246.462406:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880246.462552:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880246.462556:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880246.462559:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880247.486347:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880247.486353:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880247.486358:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880247.486359:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880247.486415:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880247.486419:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880247.486422:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880248.510328:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880248.510335:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880248.510342:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880248.510343:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880248.510387:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880248.510394:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880248.510399:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880249.470354:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305880528, at 4305880576 00000100:00000040:1.0:1788880249.470365:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305880528, at 4305880576 00000100:00000040:2.0:1788880249.470371:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305880528, at 4305880576 00000100:00000010:0.0:1788880249.470372:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a0959d87. 00000100:00000010:1.0:1788880249.470375:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ca0fac21. 00000100:00000010:2.0:1788880249.470390:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000043ac658c. 00000100:00000040:3.0:1788880249.470477:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880249.470482:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880249.470489:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880249.470513:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880249.470518:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880249.470521:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880249.470526:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880249.470537:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:3.0:1788880249.470544:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880249.470553:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880249.470563:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880249.470576:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880249.470582:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_00+0] req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880249.470611:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880249.470614:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880249.470616:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880249.470619:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000020:00000040:3.0:1788880249.470621:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880249.470623:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880249.470625:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880249.470626:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880249.470628:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880249.470629:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:0.0:1788880249.470632:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880249.470633:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880249.470635:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880249.470642:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:0.0:1788880249.470643:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880249.470651:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880249.470662:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880249.470676:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880249.470677:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880249.470679:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880249.470683:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_01+1] req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880249.470684:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880249.470692:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880249.470695:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880249.470698:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880249.470698:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880249.470704:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788880249.470706:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780643072:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880249.470712:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880249.470713:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880249.470717:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880249.470718:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880249.470719:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880249.470719:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1788880249.470725:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000010:3.0:1788880249.470725:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 02000000:00000001:0.0:1788880249.470726:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880249.470736:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000cf77ae66. 00000020:00000040:3.0:1788880249.470739:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:2.0:1788880249.470743:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880249.470744:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880249.470753:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:0.0:1788880249.470755:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 02000000:00000001:3.0:1788880249.470758:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880249.470763:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880249.470769:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 02000000:00000010:3.0:1788880249.470777:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000200:0.0:1788880249.470782:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780643072, portal 25 00000100:00000001:0.0:1788880249.470790:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880249.470790:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880249.470791:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880249.470797:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880249.470798:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 02000000:00000001:2.0:1788880249.470804:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880249.470809:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880249.470812:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:0.0:1788880249.470815:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880265 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880249.470817:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880249.470832:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780643200:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880249.470855:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880249.470863:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880249.470863:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788880249.470865:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880249.470867:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880249.470870:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880249.470870:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880249.470876:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 02000000:00000001:2.0:1788880249.470877:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880249.470880:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_02+2] req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880249.470881:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780643072, offset 0 02000000:00000010:2.0:1788880249.470886:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000099f3ac40. 00000400:00000200:0.0:1788880249.470895:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880249.470896:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880249.470910:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 00000400:00000200:0.0:1788880249.470915:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:2.0:1788880249.470922:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000793778c8. 00000400:00000200:0.0:1788880249.470923:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880249.470926:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880249.470929:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:2.0:1788880249.470931:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780643200, portal 10 00000400:00000200:0.0:1788880249.470933:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788880249.470934:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1788880249.470940:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11218) 00000400:00000200:0.0:1788880249.470941:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880249.470948:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880249.470950:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880249.470952:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000100:00000001:3.0:1788880249.470953:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880249.470958:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000400:00000200:0.0:1788880249.470959:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [94:94:256:4294967295] 192.168.204.111@tcp LPNI seq info [94:94:8:4294967295] 00000100:00000001:1.0:1788880249.470962:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880249.470962:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880249.470968:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880249.470971:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880249.470975:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880249.470980:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880249.470981:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880249.470993:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880249.470999:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:0.0:1788880249.471001:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880249.471005:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000800:00000200:0.0:1788880249.471015:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880249.471015:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880249.471016:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880249.471025:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880249.471029:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880249.471031:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:2.0:1788880249.471032:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d208f26. 00000100:00000200:2.0:1788880249.471041:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780643200, offset 0 00000800:00000200:0.0:1788880249.471042:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880249.471056:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880249.471059:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880249.471066:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:2.0:1788880249.471067:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880249.471069:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880249.471076:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305885647 02000000:00000001:3.0:1788880249.471078:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:0.0:1788880249.471082:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009f22a9a0. 00000400:00000200:2.0:1788880249.471083:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880249.471084:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880249.471097:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:2.0:1788880249.471099:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:3.0:1788880249.471100:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780643328:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880249.471109:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305885647 00000400:00000200:2.0:1788880249.471111:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880249.471113:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880249.471119:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:0.0:1788880249.471120:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000085563e6f. 02000000:00000001:3.0:1788880249.471120:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880249.471126:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880249.471130:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305885647 02000000:00000001:3.0:1788880249.471132:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880249.471135:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:0.0:1788880249.471139:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009a4dee8c. 02000000:00000001:3.0:1788880249.471139:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880249.471146:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [95:95:255:4294967295] 192.168.204.111@tcp LPNI seq info [95:95:7:4294967295] 02000000:00000010:3.0:1788880249.471147:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ed94d533. 00000100:00000001:0.0:1788880249.471148:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 02000000:00000001:3.0:1788880249.471156:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880249.471159:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305885647 00000400:00000010:3.0:1788880249.471166:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000be05c49f. 00000400:00000010:3.0:1788880249.471171:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 00000100:00000001:0.0:1788880249.471173:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305880528 00000100:00000200:3.0:1788880249.471180:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780643328, portal 10 00000400:00000200:2.0:1788880249.471185:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880249.471186:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880249.471186:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:1.0:1788880249.471196:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880249.471201:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000800:00000010:1.0:1788880249.471203:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f92e72a2. 00000100:00000001:0.0:1788880249.471206:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880249.471207:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880249.471214:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880249.471216:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880249.471217:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 00000100:00000040:3.0:1788880249.471218:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880249.471224:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880249.471227:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880249.471256:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000400:00000010:1.0:1788880249.471260:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000800:00000200:2.0:1788880249.471262:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880249.471263:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880249.471269:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880249.471272:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000200:1.0:1788880249.471280:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880265 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880249.471282:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880249.471282:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1788880249.471283:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880249.471285:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880249.471289:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880249.471292:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880249.471294:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880249.471309:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880249.471314:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880249.471317:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1155663 usecs (1156 jiffies) 00000100:00000001:2.0:1788880249.471322:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305881733 00000100:00000001:2.0:1788880249.471325:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305885647 00000100:00000001:2.0:1788880249.471328:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305881733 00000100:00000040:1.0:1788880249.471331:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880265 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880249.471333:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305885647 00000100:00000001:2.0:1788880249.471341:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305881733 00000100:00000001:2.0:1788880249.471348:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305885647 00000100:00000040:3.0:1788880249.471352:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880249.471356:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880249.471359:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e78d7264. 00000100:00000001:2.0:1788880249.471362:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305881733 00000100:00000001:1.0:1788880249.471364:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000200:3.0:1788880249.471367:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780643328, offset 0 00000100:00000001:2.0:1788880249.471368:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305885647 00000100:00000001:2.0:1788880249.471374:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880249.471377:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880249.471381:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880249.471389:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880249.471394:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880249.471395:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880249.471410:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880249.471412:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:1.0:1788880249.471416:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880249.471417:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:3.0:1788880249.471418:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880249.471427:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000010:1.0:1788880249.471428:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000100:00000001:2.0:1788880249.471428:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880249.471431:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880249.471435:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880249.471435:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880249.471443:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880249.471445:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880249.471450:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [96:96:255:4294967295] 192.168.204.111@tcp LPNI seq info [96:96:7:4294967295] 00000400:00000200:1.0:1788880249.471456:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880249.471466:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880249.471470:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880249.471474:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000400:00000200:1.0:1788880249.471476:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d208f26 00000800:00000200:3.0:1788880249.471480:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:1.0:1788880249.471481:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d208f26. 00000100:00000001:1.0:1788880249.471488:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880249.471497:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880249.471505:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880249.471507:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880249.471516:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880249.471518:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880249.471525:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880249.471534:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880249.471548:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880249.471551:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 74578 usecs (75 jiffies) 00000100:00000001:3.0:1788880249.471554:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305880652 00000100:00000001:3.0:1788880249.471557:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305885647 00000100:00000001:3.0:1788880249.471558:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305880652 00000100:00000001:3.0:1788880249.471561:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305885647 00000100:00000001:3.0:1788880249.471564:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305880652 00000100:00000001:3.0:1788880249.471574:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305885647 00000100:00000001:3.0:1788880249.471578:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305880652 00000100:00000001:3.0:1788880249.471580:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305885647 00000100:00000001:3.0:1788880249.471582:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880249.471584:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880249.471586:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880249.471590:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880249.471678:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880249.471683:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000400:00000200:2.0:1788880249.471689:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880249.471695:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880249.471700:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e78d7264 00000400:00000010:2.0:1788880249.471703:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e78d7264. 00000100:00000001:2.0:1788880249.471710:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880249.471716:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880249.471727:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880249.471730:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880249.471737:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880249.471740:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880249.472834:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880249.472843:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880249.472846:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880249.472848:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880249.472855:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880249.472866:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b02b00 00000400:00000200:2.0:1788880249.472872:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x3d5 [1] + 224 00000800:00000001:2.0:1788880249.472885:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880249.472887:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880249.472889:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880249.472893:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880249.472897:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880249.472902:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880249.472912:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880249.472929:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880249.472936:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880249.472941:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880249.472943:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880249.472945:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880249.472949:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880249.472955:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02b80 00000100:00000001:0.0:1788880249.472959:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880249.472961:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x3dd [1] + 224 00000100:00000001:0.0:1788880249.472964:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880249.472966:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880249.472970:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880249.472972:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880249.472974:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788880249.472976:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:0.0:1788880249.472978:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000400:00000200:2.0:1788880249.472983:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880249.472984:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000010:0.0:1788880249.472988:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880249.472991:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880249.472995:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880249.472996:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880249.473002:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880249.473017:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880249.473020:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880249.473041:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880249.473044:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880249.473048:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880249.473050:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880249.473050:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880249.473053:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880249.473056:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880249.473056:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:0.0:1788880249.473061:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880249.473061:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880249.473065:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880249.473068:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=77 v=5 (5 0 0 0) 00000800:00000001:2.0:1788880249.473068:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880249.473073:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=77 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880249.473079:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:2.0:1788880249.473080:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880249.473082:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880249.473086:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:0.0:1788880249.473087:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880249.473090:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:0.0:1788880249.473091:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880249.473095:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880249.473097:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880249.473098:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000400:00000200:2.0:1788880249.473102:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02c00 00000400:00000200:1.0:1788880249.473104:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000793778c8 00000100:00000001:0.0:1788880249.473105:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880249.473108:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000793778c8. 00000100:00000001:1.0:1788880249.473112:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:0.0:1788880249.473113:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880249.473118:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x3e5 [1] + 224 00000100:00000200:1.0:1788880249.473119:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880249.473128:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880249.473136:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880249.473141:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880249.473142:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788880249.473146:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880249.473149:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880249.473153:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880249.473155:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880249.473159:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880249.473159:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880249.473169:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:0.0:1788880249.473172:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880249.473173:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880249.473178:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:2.0:1788880249.473179:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880249.473181:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880249.473184:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880249.473186:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880249.473191:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880249.473200:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780643072:192.168.204.111@tcp:400: 00000100:00001000:1.0:1788880249.473204:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=67 v=5 (5 0 0 0) 00000100:00000001:0.0:1788880249.473207:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880249.473211:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00001000:1.0:1788880249.473212:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=67 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880249.473213:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:0.0:1788880249.473216:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780643072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880249.473217:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:2.0:1788880249.473220:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880249.473225:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880249.473227:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880249.473229:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00010000:00000001:1.0:1788880249.473253:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:0.0:1788880249.473255:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00010000:00000001:1.0:1788880249.473257:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880249.473260:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880249.473261:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000cf77ae66. 00000100:00000001:1.0:1788880249.473263:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880249.473264:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880249.473267:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:1.0:1788880249.473267:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:3.0:1788880249.473269:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000001:0.0:1788880249.473271:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880249.473272:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000001:3.0:1788880249.473272:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000020:00000040:0.0:1788880249.473275:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000010:3.0:1788880249.473276:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000be05c49f. 00000100:00000001:1.0:1788880249.473277:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880249.473281:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000200:3.0:1788880249.473282:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 02000000:00000010:0.0:1788880249.473286:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 00000100:00000040:1.0:1788880249.473286:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880249.473287:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000100:00000001:3.0:1788880249.473292:0:201288:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:0.0:1788880249.473295:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000200:3.0:1788880249.473298:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880249.473299:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880249.473300:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000010:0.0:1788880249.473303:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:1.0:1788880249.473305:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880249.473309:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880249.473311:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880249.473315:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880249.473318:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880249.473319:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880249.473320:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880249.473324:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880249.473327:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880249.473332:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880249.473336:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880249.473352:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780643200:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880249.473352:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880249.473355:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880249.473358:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880249.473359:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880249.473362:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880249.473365:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780643200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880249.473369:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880249.473372:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880249.473375:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880249.473376:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880249.473379:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:3.0:1788880249.473380:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880249.473382:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000099f3ac40. 00000100:00001000:3.0:1788880249.473384:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=77 v=5 (5 0 0 0) 02000000:00000001:1.0:1788880249.473386:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880249.473389:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00001000:3.0:1788880249.473390:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=51 v=5 (1 0 0 0) 00000020:00000040:1.0:1788880249.473392:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880249.473393:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880249.473395:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880249.473399:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:3.0:1788880249.473399:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000010:1.0:1788880249.473404:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 00010000:00080000:3.0:1788880249.473405:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880249.473409:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880249.473412:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00010000:00000001:3.0:1788880249.473413:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880249.473414:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000001:3.0:1788880249.473416:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880249.473419:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880249.473420:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880249.473421:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880249.473425:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880249.473425:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880249.473430:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880249.473431:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880249.473436:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880249.473439:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880249.473441:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880249.473443:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788880249.473443:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880249.473446:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880249.473451:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880249.473454:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880249.473455:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780643328:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880249.473459:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880249.473460:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880249.473462:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780643328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880265 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880249.473466:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880249.473467:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880249.473468:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ed94d533. 02000000:00000001:3.0:1788880249.473471:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880249.473476:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880249.473477:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880249.473478:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880249.473479:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 02000000:00000001:3.0:1788880249.473481:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880249.473482:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880249.473483:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:3.0:1788880249.473485:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880249.473485:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880249.473486:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880249.473489:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880249.473490:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880249.473491:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880249.473492:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880249.534368:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880249.534376:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880249.534386:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880249.534387:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880249.534414:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880249.534420:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880249.534425:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880250.558323:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880250.558329:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880250.558332:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880250.558333:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880250.558365:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880250.558368:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880250.558371:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880251.582354:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880251.582361:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880251.582367:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880251.582369:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880251.582411:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880251.582415:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880251.582418:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880252.606380:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880252.606399:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880252.606408:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880252.606409:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880252.606447:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880252.606452:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880252.606455:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880253.630327:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880253.630337:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880253.630340:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880253.630344:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880253.630350:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880253.630353:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880253.630354:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880253.630357:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880253.630361:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880253.630362:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880253.630402:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880253.630407:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880253.630411:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880254.590377:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305885647, at 4305885696 00000100:00000040:2.0:1788880254.590377:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305885647, at 4305885696 00000100:00000040:3.0:1788880254.590380:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305885647, at 4305885696 00000100:00000010:3.0:1788880254.590404:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001e508d7a. 00000100:00000010:2.0:1788880254.590410:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009a4dee8c. 00000100:00000010:1.0:1788880254.590413:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000085563e6f. 00000100:00000040:1.0:1788880254.590478:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880254.590486:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305885647, at 4305885696 00000100:00000001:1.0:1788880254.590486:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880254.590501:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002dcbd5d0. 00000100:00000010:0.0:1788880254.590508:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009f22a9a0. 00000020:00000040:1.0:1788880254.590509:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880254.590521:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880254.590549:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880254.590556:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880254.590569:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:1.0:1788880254.590579:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880254.590592:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880254.590605:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880254.590608:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880254.590613:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002dcbd5d0] to pc [ptlrpcd_00_03+3] req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880254.590644:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880254.590647:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880254.590651:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880254.590654:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000003b122924. 00000020:00000040:1.0:1788880254.590657:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880254.590660:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880254.590662:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880254.590664:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880254.590669:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:3.0:1788880254.590669:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880254.590675:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880254.590676:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880254.590680:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880254.590681:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880254.590689:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880254.590691:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880254.590693:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880254.590699:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003b122924] to pc [ptlrpcd_00_00+0] req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880254.590714:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880254.590717:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880254.590722:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880254.590725:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880254.590728:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880254.590729:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880254.590735:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788880254.590736:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780643456:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880254.590740:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880254.590743:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000010:1.0:1788880254.590747:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c09231ec. 00000100:00000001:0.0:1788880254.590750:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:1.0:1788880254.590752:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:3.0:1788880254.590752:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880254.590757:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880254.590757:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880254.590758:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880254.590768:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880254.590769:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880254.590770:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788880254.590771:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880254.590772:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880254.590782:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000063883087. 02000000:00000001:1.0:1788880254.590783:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880254.590787:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880254.590790:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000040:0.0:1788880254.590793:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880254.590796:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880254.590796:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000be05c49f. 00000100:00000040:1.0:1788880254.590806:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:3.0:1788880254.590807:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 00000100:00000001:2.0:1788880254.590811:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880254.590820:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880254.590829:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780643456, portal 25 00000100:00000001:3.0:1788880254.590838:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880254.590843:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880254.590843:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938047872 : -126823771503744 : ffff8ca78a268380) 00000100:00000001:1.0:1788880254.590848:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880254.590858:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880254.590862:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c09231ec] to pc [ptlrpcd_00_01+1] req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880254.590863:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880270 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880254.590865:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880254.590881:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880254.590891:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880254.590894:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880254.590894:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880254.590897:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880254.590900:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880254.590908:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1788880254.590908:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880254.590915:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11223) 00000400:00000010:3.0:1788880254.590915:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000100:00100000:0.0:1788880254.590921:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780643584:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880254.590921:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:3.0:1788880254.590922:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780643456, offset 0 00000100:00000001:2.0:1788880254.590927:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880254.590936:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880254.590937:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1788880254.590943:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880254.590948:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880254.590949:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880254.590953:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880254.590958:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880254.590963:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880254.590965:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880254.590968:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000010:0.0:1788880254.590969:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cd0104b. 02000000:00000001:0.0:1788880254.590976:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880254.590977:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:0.0:1788880254.590983:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000400:00000200:3.0:1788880254.590984:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:2.0:1788880254.590988:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880254.590995:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000400:00000200:3.0:1788880254.590997:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:2.0:1788880254.590998:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880254.591005:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780643584, portal 10 00000100:00100000:2.0:1788880254.591013:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780643712:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880254.591013:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880254.591020:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880254.591028:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [97:97:256:4294967295] 192.168.204.111@tcp LPNI seq info [97:97:8:4294967295] 00000100:00000001:0.0:1788880254.591046:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880254.591047:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880254.591049:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938062208 : -126823771489408 : ffff8ca78a26bb80) 02000000:00000001:2.0:1788880254.591052:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880254.591058:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880254.591063:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880254.591065:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880254.591070:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008d264e32. 02000000:00000001:2.0:1788880254.591080:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880254.591080:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880254.591089:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:2.0:1788880254.591095:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ffb4c8a2. 00000400:00000010:2.0:1788880254.591103:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cc94f399. 00000800:00000200:3.0:1788880254.591104:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788880254.591109:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780643712, portal 10 00000100:00000040:0.0:1788880254.591115:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880254.591116:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000010:3.0:1788880254.591117:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000749ed09e. 00000100:00000001:2.0:1788880254.591121:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938056832 : -126823771494784 : ffff8ca78a26a680) 00000400:00000010:0.0:1788880254.591122:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000096cf0c4a. 00000100:00000200:0.0:1788880254.591130:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780643584, offset 0 00000800:00000200:3.0:1788880254.591137:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:2.0:1788880254.591144:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880254.591147:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880254.591154:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880254.591165:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000749ed09e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880254.591171:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788880254.591172:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880254.591180:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:2.0:1788880254.591180:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880254.591185:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000084794ee4. 00000100:00000001:3.0:1788880254.591188:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880254.591191:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780643712, offset 0 00000100:00000001:3.0:1788880254.591196:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305890767 00000400:00000200:0.0:1788880254.591197:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880254.591205:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788880254.591208:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000837757ae. 00000400:00000200:0.0:1788880254.591215:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880254.591219:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880254.591223:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880254.591226:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305890767 00000100:00000010:3.0:1788880254.591265:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000025d9193b. 00000400:00000200:0.0:1788880254.591267:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [98:98:255:4294967295] 192.168.204.111@tcp LPNI seq info [98:98:7:4294967295] 00000100:00000001:3.0:1788880254.591270:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880254.591274:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305890768 00000100:00000010:3.0:1788880254.591282:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d5dcfb55. 00000100:00000001:3.0:1788880254.591285:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880254.591288:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305890768 00000100:00000010:3.0:1788880254.591292:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a5ad6403. 00000100:00000001:3.0:1788880254.591297:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880254.591301:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880254.591307:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880254.591308:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880254.591314:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880254.591321:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880254.591329:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880254.591329:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880254.591331:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880254.591336:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880254.591341:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:0.0:1788880254.591342:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 00000400:00000200:2.0:1788880254.591342:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880254.591343:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880254.591351:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880254.591358:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880254.591359:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880254.591361:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:0.0:1788880254.591365:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880254.591370:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000010:1.0:1788880254.591371:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000749ed09e. 00000400:00000200:2.0:1788880254.591379:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788880254.591380:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880254.591383:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:0.0:1788880254.591390:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880254.591396:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [99:99:254:4294967295] 192.168.204.111@tcp LPNI seq info [99:99:6:4294967295] 00000100:00000001:0.0:1788880254.591398:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880254.591402:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:0.0:1788880254.591412:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1105552 usecs (1106 jiffies) 00000100:00000001:0.0:1788880254.591418:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305886803 00000100:00000001:0.0:1788880254.591422:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305890767 00000400:00000200:2.0:1788880254.591426:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880254.591427:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305886803 00000100:00000001:0.0:1788880254.591431:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305890767 00000100:00000001:0.0:1788880254.591436:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305886803 00000400:00000200:1.0:1788880254.591439:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000800:00000200:2.0:1788880254.591440:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880254.591442:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305890768 00000400:00000010:1.0:1788880254.591445:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000001:1.0:1788880254.591454:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880254.591455:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305886803 00000100:00000001:0.0:1788880254.591459:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305890768 00000100:00000200:1.0:1788880254.591460:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880270 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:2.0:1788880254.591461:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000100:00000001:0.0:1788880254.591466:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880254.591494:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880254.591497:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880254.591499:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880254.591500:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880254.591508:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880270 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880254.591509:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788880254.591517:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880254.591521:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880254.591524:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880254.591526:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880254.591529:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880254.591540:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880254.591542:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880254.591543:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 70816 usecs (71 jiffies) 00000100:00000001:1.0:1788880254.591545:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880254.591547:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880254.591553:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880254.591554:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305885768 00000100:00000001:0.0:1788880254.591557:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880254.591560:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305890767 00000100:00000001:2.0:1788880254.591566:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305885768 00000100:00000001:2.0:1788880254.591570:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305890767 00000100:00000001:2.0:1788880254.591574:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305885768 00000800:00000200:1.0:1788880254.591577:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880254.591578:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305890768 00000800:00000010:1.0:1788880254.591580:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000100:00000001:2.0:1788880254.591585:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305885768 00000400:00000200:1.0:1788880254.591594:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880254.591594:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305890768 00000400:00000200:1.0:1788880254.591605:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880254.591606:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880254.591610:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000096cf0c4a 00000400:00000010:1.0:1788880254.591615:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000096cf0c4a. 00000100:00000001:2.0:1788880254.591616:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880254.591620:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880254.591623:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880254.591624:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880254.591629:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880254.591639:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880254.591641:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880254.591643:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880254.591645:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880254.591648:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880254.591649:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880254.591654:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880254.591660:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880254.591662:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880254.591667:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880254.591690:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880254.591693:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000400:00000200:1.0:1788880254.591701:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880254.591704:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880254.591707:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000084794ee4 00000400:00000010:1.0:1788880254.591708:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000084794ee4. 00000100:00000001:1.0:1788880254.591711:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880254.591714:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880254.591721:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880254.591723:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880254.591729:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880254.591731:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880254.593477:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880254.593506:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880254.593511:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880254.593514:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880254.593526:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880254.593543:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b02c80 00000400:00000200:2.0:1788880254.593552:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x3ed [1] + 224 00000800:00000001:2.0:1788880254.593559:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880254.593570:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880254.593573:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880254.593578:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880254.593585:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880254.593604:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880254.593615:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880254.593634:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880254.593642:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880254.593646:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880254.593675:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880254.593680:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880254.593688:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880254.593690:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880254.593693:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000be05c49f. 00000400:00000200:3.0:1788880254.593699:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 00000400:00000010:3.0:1788880254.593701:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000100:00000001:3.0:1788880254.593704:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880254.593711:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880254.593721:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880254.593729:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880254.593730:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880254.593733:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880254.593736:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880254.593739:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880254.593742:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880254.593745:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880254.593752:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=82 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880254.593854:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=82 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880254.593859:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880254.593861:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880254.593865:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880254.593869:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880254.593873:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880254.593879:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880254.593883:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880254.593893:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:2.0:1788880254.593896:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880254.593898:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000800:00000001:2.0:1788880254.593901:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880254.593904:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880254.593910:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880254.593918:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02d80 00000400:00000200:2.0:1788880254.593923:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x3f9 [1] + 224 00000800:00000001:2.0:1788880254.593927:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880254.593930:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880254.593933:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880254.593936:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880254.593940:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880254.593946:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880254.593946:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880254.593958:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880254.593973:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780643456:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880254.593990:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880254.593990:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880254.593995:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880254.594002:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002dcbd5d0 x1875776780643456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880254.594030:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880254.594042:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880254.594045:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000063883087. 00000100:00000001:2.0:1788880254.594047:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:3.0:1788880254.594050:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880254.594054:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880254.594057:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000800:00000001:2.0:1788880254.594058:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000001:3.0:1788880254.594063:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880254.594067:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 02000000:00000001:3.0:1788880254.594072:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000800:00000001:2.0:1788880254.594076:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880254.594076:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880254.594082:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002dcbd5d0. 00000800:00000001:2.0:1788880254.594083:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880254.594092:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880254.594092:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880254.594096:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880254.594101:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880254.594110:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880254.594111:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880254.594114:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880254.594117:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880254.594120:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02d00 00000400:00000001:1.0:1788880254.594130:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880254.594134:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880254.594136:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880254.594137:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ffb4c8a2. 00000100:00000001:3.0:1788880254.594144:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880254.594147:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cc94f399 00000100:00000001:3.0:1788880254.594149:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880254.594152:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cc94f399. 00000100:00000001:1.0:1788880254.594159:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880254.594163:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x3f5 [1] + 224 00000100:00000200:1.0:1788880254.594174:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880254.594177:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788880254.594185:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880254.594186:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880254.594190:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880254.594198:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:2.0:1788880254.594198:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880254.594202:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880254.594203:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880254.594206:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000200:2.0:1788880254.594210:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880254.594211:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880254.594214:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880254.594217:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880254.594222:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880254.594230:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=82 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880254.594257:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=56 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880254.594259:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880254.594260:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880254.594262:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:2.0:1788880254.594265:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00080000:1.0:1788880254.594267:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880254.594284:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880254.594286:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880254.594289:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880254.594292:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:2.0:1788880254.594293:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880254.594294:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880254.594296:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880254.594299:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880254.594301:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880254.594306:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880254.594308:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880254.594310:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880254.594314:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880254.594314:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880254.594319:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00100000:1.0:1788880254.594323:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780643712:192.168.204.111@tcp:400: 00000400:00000001:0.0:1788880254.594325:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880254.594328:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880254.594329:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000400:00000200:0.0:1788880254.594335:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000100:00000001:1.0:1788880254.594337:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880254.594338:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:1.0:1788880254.594340:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880254.594344:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:1.0:1788880254.594344:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c09231ec x1875776780643712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880254.594351:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880254.594352:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880254.594355:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880254.594358:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008d264e32. 02000000:00000001:1.0:1788880254.594363:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880254.594365:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880254.594367:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000200:0.0:1788880254.594368:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788880254.594370:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880254.594373:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:1.0:1788880254.594378:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880254.594380:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880254.594381:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000010:1.0:1788880254.594382:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c09231ec. 00000100:00000001:0.0:1788880254.594384:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880254.594386:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880254.594387:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880254.594388:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880254.594390:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880254.594391:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880254.594393:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880254.594395:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880254.594397:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1788880254.594398:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880254.594399:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880254.594403:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880254.594404:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880254.594405:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=72 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880254.594410:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=72 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880254.594413:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880254.594414:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880254.594422:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880254.594424:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880254.594427:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880254.594428:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880254.594430:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880254.594431:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880254.594434:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880254.594441:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880254.594443:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880254.594444:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880254.594447:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880254.594453:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780643584:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880254.594458:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880254.594460:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880254.594462:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000003b122924 x1875776780643584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880270 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880254.594479:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880254.594481:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880254.594483:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cd0104b. 02000000:00000001:0.0:1788880254.594485:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880254.594487:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880254.594488:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880254.594489:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880254.594491:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:0.0:1788880254.594493:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880254.594494:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880254.594495:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000003b122924. 00000100:00000001:0.0:1788880254.594498:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880254.594499:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880254.594501:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880254.594504:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880254.594510:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880254.594511:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880254.594513:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880254.654359:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880254.654366:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880254.654373:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880254.654374:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880254.654404:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880254.654408:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880254.654411:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880255.678359:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880255.678368:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880255.678375:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880255.678379:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880255.678382:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880255.678385:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880255.678389:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880255.678391:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880255.678393:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880255.678396:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880255.678464:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880255.678468:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880255.678471:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880256.702378:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880256.702385:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880256.702391:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880256.702392:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880256.702419:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880256.702423:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880256.702435:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880257.726955:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880257.726964:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880257.726971:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880257.726972:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880257.727026:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880257.727030:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880257.727033:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880258.750412:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880258.750419:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880258.750425:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880258.750426:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880258.750989:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880258.750993:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880258.750996:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880259.710379:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305890767, at 4305890816 00000100:00000040:1.0:1788880259.710379:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305890767, at 4305890816 00000100:00000040:2.0:1788880259.710379:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305890768, at 4305890816 00000100:00000040:3.0:1788880259.710389:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305890768, at 4305890816 00000100:00000010:0.0:1788880259.710399:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000837757ae. 00000100:00000010:2.0:1788880259.710401:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d5dcfb55. 00000100:00000010:1.0:1788880259.710406:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000025d9193b. 00000100:00000010:3.0:1788880259.710409:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a5ad6403. 00000100:00000040:3.0:1788880259.710447:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880259.710455:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880259.710467:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000020:00000040:3.0:1788880259.710473:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880259.710478:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880259.710482:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880259.710488:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880259.710498:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000055d7d92. 00000100:00000001:3.0:1788880259.710507:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880259.710520:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880259.710533:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880259.710537:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880259.710545:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_02+2] req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880259.710582:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880259.710585:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880259.710589:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880259.710593:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000020:00000040:3.0:1788880259.710597:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880259.710599:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880259.710602:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880259.710604:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880259.710610:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5b428a0. 00000100:00000001:3.0:1788880259.710613:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880259.710617:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880259.710624:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880259.710629:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880259.710631:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880259.710634:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880259.710639:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880259.710646:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880259.710655:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880259.710657:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880259.710662:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_03+3] req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880259.710691:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880259.710697:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880259.710699:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880259.710702:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880259.710703:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880259.710707:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880259.710709:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:2.0:1788880259.710712:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880259.710714:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000100:00100000:2.0:1788880259.710716:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780643840:192.168.204.111@tcp:400: 00000020:00000040:3.0:1788880259.710719:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880259.710723:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880259.710725:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880259.710725:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880259.710728:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880259.710730:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880259.710733:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880259.710733:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880259.710734:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880259.710742:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880259.710743:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 02000000:00000001:2.0:1788880259.710747:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880259.710750:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880259.710750:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880259.710756:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000051bfba0. 00000100:00000040:3.0:1788880259.710763:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880259.710765:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880259.710783:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000400:00000010:2.0:1788880259.710784:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001a8af943. 00000100:00000001:3.0:1788880259.710788:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880259.710800:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_00+0] req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880259.710801:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880259.710806:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880259.710808:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c0c07443. 02000000:00000001:0.0:1788880259.710815:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880259.710823:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788880259.710826:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780643840, portal 25 02000000:00000001:0.0:1788880259.710828:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880259.710830:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880259.710836:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000001:3.0:1788880259.710842:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880259.710843:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780643968:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880259.710851:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880275 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880259.710851:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880259.710858:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880259.710867:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:0.0:1788880259.710871:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880259.710875:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880259.710877:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11228) 00000100:00000040:2.0:1788880259.710879:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880259.710885:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009c31f261. 00000100:00000200:2.0:1788880259.710888:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780643840, offset 0 00000100:00000001:1.0:1788880259.710889:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880259.710891:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880259.710895:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880259.710897:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880259.710900:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880259.710904:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000010:0.0:1788880259.710908:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e6a93e02. 00000100:00000040:1.0:1788880259.710910:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880259.710915:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880259.710915:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880259.710922:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880259.710926:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:0.0:1788880259.710929:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000400:00000010:0.0:1788880259.710942:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000001:1.0:1788880259.710942:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880259.710947:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880259.710947:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:0.0:1788880259.710954:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780643968, portal 10 02000000:00000001:1.0:1788880259.710955:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:2.0:1788880259.710956:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788880259.710962:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880259.710965:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788880259.710969:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880259.710971:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000400:00000200:2.0:1788880259.710982:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:1.0:1788880259.710983:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880259.710988:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880259.710993:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [100:100:256:4294967295] 192.168.204.111@tcp LPNI seq info [100:100:8:4294967295] 00000100:00100000:1.0:1788880259.710997:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780644096:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880259.711018:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880259.711029:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880259.711035:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880259.711038:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:1.0:1788880259.711048:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880259.711051:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:1.0:1788880259.711056:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000010c9710f. 00000800:00000200:2.0:1788880259.711057:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880259.711065:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880259.711066:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880259.711067:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c52725cf. 00000400:00000010:0.0:1788880259.711077:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f4d84419. 00000400:00000010:1.0:1788880259.711079:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008388b815. 00000800:00000200:2.0:1788880259.711088:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:0.0:1788880259.711089:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780643968, offset 0 00000400:00000010:1.0:1788880259.711092:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000800:00000200:2.0:1788880259.711098:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:1.0:1788880259.711099:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780644096, portal 10 00000100:00000001:1.0:1788880259.711105:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880259.711109:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880259.711112:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c52725cf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880259.711116:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000040:1.0:1788880259.711125:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880259.711131:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880259.711135:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880259.711137:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880259.711144:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880259.711146:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:1.0:1788880259.711148:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000001:2.0:1788880259.711148:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305895887 00000100:00000200:1.0:1788880259.711155:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780644096, offset 0 00000400:00000200:0.0:1788880259.711159:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:2.0:1788880259.711159:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ce680e22. 00000400:00000200:1.0:1788880259.711168:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880259.711171:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880259.711174:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880259.711179:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305895887 00000400:00000200:0.0:1788880259.711180:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:2.0:1788880259.711187:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb1a14f4. 00000400:00000200:0.0:1788880259.711191:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [101:101:255:4294967295] 192.168.204.111@tcp LPNI seq info [101:101:7:4294967295] 00000100:00000001:2.0:1788880259.711193:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:2.0:1788880259.711210:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305895887 00000400:00000200:0.0:1788880259.711211:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:2.0:1788880259.711218:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a3bf26bf. 00000100:00000001:2.0:1788880259.711224:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:0.0:1788880259.711228:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880259.711228:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880259.711230:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305895887 00000100:00000010:2.0:1788880259.711257:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000043ac658c. 00000100:00000001:2.0:1788880259.711264:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788880259.711267:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880259.711268:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880259.711273:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880259.711275:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880259.711278:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880259.711279:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880259.711282:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:0.0:1788880259.711284:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000088542ca3. 00000400:00000200:1.0:1788880259.711284:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880259.711292:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [102:102:254:4294967295] 192.168.204.111@tcp LPNI seq info [102:102:6:4294967295] 00000100:00000001:2.0:1788880259.711292:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880259.711295:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:0.0:1788880259.711304:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880259.711304:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880259.711305:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880259.711309:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880259.711310:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880259.711314:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880259.711318:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c52725cf. 00000100:00000001:2.0:1788880259.711320:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880259.711329:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:1.0:1788880259.711336:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880259.711339:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000010:1.0:1788880259.711346:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000800:00000200:0.0:1788880259.711347:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000088542ca3 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880259.711353:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000200:1.0:1788880259.711358:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880259.711361:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880259.711364:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009c31f261 00000800:00000200:1.0:1788880259.711368:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:3.0:1788880259.711381:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009c31f261. 00000100:00000040:0.0:1788880259.711385:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1059678 usecs (1060 jiffies) 00000800:00000200:1.0:1788880259.711385:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880259.711388:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880259.711395:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305891877 00000100:00000001:1.0:1788880259.711396:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880259.711401:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 67482 usecs (68 jiffies) 00000100:00000001:1.0:1788880259.711409:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305890885 00000100:00000001:0.0:1788880259.711410:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305895887 00000100:00000001:0.0:1788880259.711416:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305891877 00000100:00000001:1.0:1788880259.711417:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305895887 00000100:00000001:0.0:1788880259.711421:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305895887 00000100:00000001:1.0:1788880259.711423:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305890885 00000100:00000001:0.0:1788880259.711427:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305891877 00000100:00000001:1.0:1788880259.711429:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305895887 00000100:00000001:0.0:1788880259.711434:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305895887 00000100:00000001:1.0:1788880259.711436:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305890885 00000100:00000001:0.0:1788880259.711440:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305891877 00000100:00000001:1.0:1788880259.711443:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305895887 00000100:00000001:0.0:1788880259.711447:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305895887 00000100:00000001:1.0:1788880259.711448:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305890885 00000100:00000001:0.0:1788880259.711452:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880259.711455:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880259.711462:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880259.711464:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305895887 00000100:00000001:0.0:1788880259.711467:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880259.711469:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:3.0:1788880259.711474:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880275 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880259.711477:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880259.711481:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880259.711484:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880259.711488:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880259.711491:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880259.711494:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880259.711495:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880259.711500:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880259.711500:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880275 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880259.711501:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880259.711508:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880259.711509:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880259.711511:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880259.711519:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880259.711522:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880259.711523:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880259.711523:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880259.711527:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880259.711534:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880259.711544:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880259.711548:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000088542ca3. 00000400:00000200:3.0:1788880259.711553:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880259.711557:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880259.711560:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f4d84419 00000400:00000010:3.0:1788880259.711561:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f4d84419. 00000100:00000001:3.0:1788880259.711563:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880259.711567:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880259.711573:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880259.711575:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880259.711581:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880259.711582:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880259.711617:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880259.711619:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:3.0:1788880259.711623:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880259.711626:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880259.711628:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:3.0:1788880259.711629:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:3.0:1788880259.711631:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880259.711633:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880259.711645:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880259.711647:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880259.711652:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880259.711654:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880259.713034:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880259.713049:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880259.713053:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880259.713056:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880259.713064:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880259.713079:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b02e00 00000400:00000200:2.0:1788880259.713087:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x405 [1] + 224 00000800:00000001:2.0:1788880259.713093:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880259.713097:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880259.713100:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880259.713103:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880259.713107:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880259.713111:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880259.713118:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880259.713151:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880259.713159:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880259.713163:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880259.713165:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880259.713168:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880259.713173:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880259.713177:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02e80 00000400:00000200:2.0:1788880259.713182:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x40d [1] + 224 00000100:00000001:1.0:1788880259.713183:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880259.713185:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880259.713187:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880259.713189:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880259.713191:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880259.713194:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:1.0:1788880259.713195:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880259.713198:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880259.713200:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001a8af943. 00000400:00000200:1.0:1788880259.713206:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c0c07443 00000400:00000010:1.0:1788880259.713208:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c0c07443. 00000100:00000001:1.0:1788880259.713212:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880259.713213:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880259.713218:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880259.713219:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880259.713262:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880259.713266:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880259.713276:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880259.713279:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880259.713282:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880259.713282:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880259.713287:0:201287:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788880259.713287:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880259.713292:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788880259.713292:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880259.713296:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880259.713296:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880259.713299:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880259.713303:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880259.713304:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880259.713310:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:1.0:1788880259.713312:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=87 v=5 (5 0 0 0) 00000400:00000200:2.0:1788880259.713318:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b02f00 00000100:00000001:0.0:1788880259.713319:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:1.0:1788880259.713321:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=87 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880259.713327:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:2.0:1788880259.713328:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x415 [1] + 224 00000100:00000001:1.0:1788880259.713330:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880259.713335:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788880259.713337:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880259.713338:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00010000:00000001:1.0:1788880259.713340:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880259.713342:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:2.0:1788880259.713346:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880259.713347:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000100:00000001:1.0:1788880259.713352:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880259.713352:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880259.713361:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000200:2.0:1788880259.713361:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:1.0:1788880259.713367:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880259.713383:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880259.713395:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880259.713395:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880259.713396:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880259.713401:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880259.713405:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880259.713410:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880259.713410:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880259.713415:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880259.713462:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880259.713490:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780643840:192.168.204.111@tcp:400: 00000100:00000200:0.0:1788880259.713501:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880259.713503:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880259.713514:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880259.713518:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880259.713522:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780643840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880259.713530:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880259.713537:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880259.713546:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880259.713547:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880259.713553:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880259.713554:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880259.713554:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880259.713557:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000051bfba0. 00000100:00000001:3.0:1788880259.713566:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880259.713567:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880259.713568:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880259.713572:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880259.713575:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000001:3.0:1788880259.713575:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880259.713579:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880259.713585:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008388b815. 02000000:00000001:0.0:1788880259.713587:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:1.0:1788880259.713587:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880259.713595:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:3.0:1788880259.713595:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000100:00000001:0.0:1788880259.713598:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880259.713601:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880259.713605:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000055d7d92. 00000400:00000010:3.0:1788880259.713605:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00001000:0.0:1788880259.713614:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=77 v=5 (5 0 0 0) 02000000:00000001:1.0:1788880259.713615:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880259.713615:0:201285:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:1.0:1788880259.713622:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000200:3.0:1788880259.713626:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880259.713628:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=77 v=5 (1 0 0 0) 00000100:00000010:1.0:1788880259.713628:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:1.0:1788880259.713643:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880259.713648:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880259.713651:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880259.713652:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880259.713658:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880259.713664:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880259.713664:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:3.0:1788880259.713665:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880259.713671:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880259.713673:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880259.713679:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880259.713682:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880259.713686:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880259.713687:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880259.713688:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880259.713691:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:0.0:1788880259.713694:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:3.0:1788880259.713694:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880259.713697:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880259.713699:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880259.713701:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880259.713703:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880259.713708:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880259.713714:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880259.713717:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=87 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880259.713731:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=61 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880259.713746:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880259.713749:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880259.713749:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880259.713758:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880259.713760:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880259.713764:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00010000:00080000:3.0:1788880259.713765:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880259.713778:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880259.713781:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880259.713791:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880259.713794:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880259.713799:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880259.713802:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880259.713807:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780643968:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880259.713810:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880259.713820:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880259.713828:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880259.713838:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780643968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880259.713843:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880259.713854:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880259.713858:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880259.713863:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880259.713867:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:3.0:1788880259.713870:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880259.713871:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e6a93e02. 02000000:00000001:0.0:1788880259.713881:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880259.713886:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880259.713889:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00100000:3.0:1788880259.713891:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780644096:192.168.204.111@tcp:400: 00000020:00000001:0.0:1788880259.713896:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880259.713901:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788880259.713909:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5b428a0. 00000100:00000001:3.0:1788880259.713909:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880259.713915:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780644096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880275 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880259.713920:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880259.713921:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880259.713926:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:3.0:1788880259.713927:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000010:0.0:1788880259.713932:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 02000000:00000010:3.0:1788880259.713932:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000010c9710f. 02000000:00000001:3.0:1788880259.713947:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880259.713948:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000020:00000001:3.0:1788880259.713952:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880259.713953:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:3.0:1788880259.713956:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880259.713959:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880259.713962:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880259.713966:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 00000100:00000001:0.0:1788880259.713968:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880259.713972:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880259.713972:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880259.713976:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880259.713978:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:3.0:1788880259.713980:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:0.0:1788880259.713984:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880259.713986:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880259.713990:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880259.713993:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880259.713998:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880259.714000:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880259.714003:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880259.714012:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880259.774380:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880259.774389:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880259.774396:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880259.774420:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880259.774544:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880259.774549:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880259.774552:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880260.798361:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880260.798371:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880260.798381:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880260.798382:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880260.798469:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880260.798475:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880260.798479:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880261.822368:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880261.822375:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880261.822380:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880261.822381:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880261.822459:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880261.822467:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880261.822471:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880262.846357:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880262.846364:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880262.846370:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880262.846371:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880262.846455:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880262.846458:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880262.846460:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880263.870389:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880263.870399:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880263.870402:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880263.870405:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880263.870410:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880263.870413:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880263.870414:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880263.870418:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880263.870423:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880263.870425:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880263.870480:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880263.870484:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880263.870486:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880264.830394:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305895887, at 4305895936 00000100:00000040:2.0:1788880264.830395:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305895887, at 4305895936 00000100:00000040:1.0:1788880264.830399:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305895887, at 4305895936 00000100:00000010:2.0:1788880264.830417:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a3bf26bf. 00000100:00000010:0.0:1788880264.830421:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ce680e22. 00000100:00000010:1.0:1788880264.830423:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb1a14f4. 00000100:00000040:2.0:1788880264.830464:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880264.830468:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305895887, at 4305895936 00000100:00000001:2.0:1788880264.830471:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880264.830477:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000043ac658c. 00000100:00000010:2.0:1788880264.830481:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000071512652. 00000020:00000040:2.0:1788880264.830487:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880264.830491:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880264.830494:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880264.830499:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880264.830506:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 00000100:00000001:2.0:1788880264.830513:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880264.830523:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880264.830533:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880264.830535:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880264.830541:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000071512652] to pc [ptlrpcd_00_01+1] req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880264.830575:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880264.830579:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880264.830583:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880264.830583:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:2.0:1788880264.830588:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ea101a56. 00000100:00000001:3.0:1788880264.830589:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:2.0:1788880264.830592:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880264.830593:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880264.830597:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:3.0:1788880264.830601:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880264.830610:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:2.0:1788880264.830614:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880264.830614:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880264.830618:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880264.830619:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000010:2.0:1788880264.830623:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006aca3ff7. 02000000:00000001:3.0:1788880264.830624:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880264.830627:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880264.830631:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000071512652 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780644224:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880264.830633:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880264.830638:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880264.830642:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:3.0:1788880264.830645:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880264.830648:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880264.830653:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880264.830655:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880264.830658:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880264.830659:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000dc9842d3. 02000000:00000001:3.0:1788880264.830662:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880264.830663:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ea101a56] to pc [ptlrpcd_00_02+2] req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880264.830670:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000be05c49f. 00000400:00000010:3.0:1788880264.830678:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 00000100:00000200:3.0:1788880264.830684:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780644224, portal 25 00000100:00000001:3.0:1788880264.830686:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880264.830688:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430080 : -126823771121536 : ffff8ca78a2c5880) 00000100:00000001:2.0:1788880264.830689:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880264.830693:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880264.830694:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880280 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880264.830700:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880264.830706:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000910cb22c. 00000100:00000001:3.0:1788880264.830707:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000020:00000040:2.0:1788880264.830711:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880264.830712:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880264.830715:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880264.830718:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880264.830718:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:3.0:1788880264.830721:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a165c7f9. 00000100:00000001:0.0:1788880264.830722:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880264.830723:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880264.830728:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880264.830728:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780644224, offset 0 00000100:00000040:0.0:1788880264.830732:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880264.830736:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003139f6ba. 00000100:00000001:1.0:1788880264.830742:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880264.830743:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880264.830743:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880264.830752:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880264.830753:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880264.830755:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880264.830758:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880264.830764:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:3.0:1788880264.830767:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880264.830771:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788880264.830774:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880264.830774:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880264.830781:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880264.830782:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000910cb22c] to pc [ptlrpcd_00_03+3] req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880264.830785:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880264.830786:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:0.0:1788880264.830789:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780644352:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880264.830792:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880264.830800:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880264.830803:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880264.830810:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:3.0:1788880264.830810:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1788880264.830814:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880264.830816:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880264.830819:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880264.830820:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880264.830824:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:0.0:1788880264.830826:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003720a685. 00000100:00000001:1.0:1788880264.830827:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880264.830827:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [103:103:256:4294967295] 192.168.204.111@tcp LPNI seq info [103:103:8:4294967295] 00000100:00000001:1.0:1788880264.830834:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880264.830835:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880264.830836:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880264.830840:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880264.830845:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000100:00000040:2.0:1788880264.830850:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11233) 00000400:00000010:0.0:1788880264.830852:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000040:1.0:1788880264.830853:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880264.830859:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:0.0:1788880264.830862:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780644352, portal 10 00000100:00000001:0.0:1788880264.830873:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880264.830878:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938436352 : -126823771115264 : ffff8ca78a2c7100) 00000100:00000001:1.0:1788880264.830892:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880264.830895:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880264.830897:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880264.830900:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880264.830906:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880264.830909:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880264.830911:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000dc289a80. 00000100:00000200:0.0:1788880264.830914:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780644352, offset 0 02000000:00000001:1.0:1788880264.830923:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880264.830928:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880264.830931:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880264.830939:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00100000:1.0:1788880264.830939:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780644480:192.168.204.111@tcp:400: 00000800:00000200:3.0:1788880264.830953:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880264.830954:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880264.830955:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880264.830965:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880264.830966:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000010:3.0:1788880264.830966:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 02000000:00000001:1.0:1788880264.830968:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880264.830973:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:1.0:1788880264.830980:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880264.830981:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788880264.830984:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000010:1.0:1788880264.830989:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000057e92cbc. 00000400:00000200:0.0:1788880264.830990:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [104:104:255:4294967295] 192.168.204.111@tcp LPNI seq info [104:104:7:4294967295] 02000000:00000001:1.0:1788880264.830995:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880264.831002:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:1.0:1788880264.831008:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000020fda082. 00000800:00000200:3.0:1788880264.831009:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880264.831021:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:1.0:1788880264.831023:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 00000800:00000200:0.0:1788880264.831028:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880264.831030:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880264.831037:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305901007 00000100:00000200:1.0:1788880264.831038:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780644480, portal 10 00000100:00000010:3.0:1788880264.831043:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000067a34a59. 00000100:00000001:1.0:1788880264.831044:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880264.831050:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938437248 : -126823771114368 : ffff8ca78a2c7480) 00000800:00000010:0.0:1788880264.831051:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 00000100:00000001:3.0:1788880264.831053:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880264.831066:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305901007 00000100:00000040:1.0:1788880264.831067:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880264.831071:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:3.0:1788880264.831081:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002c2d3615. 00000100:00000001:1.0:1788880264.831093:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880264.831094:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:1.0:1788880264.831100:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880264.831101:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305901007 00000800:00000200:0.0:1788880264.831102:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:1.0:1788880264.831105:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000002fa6ff6. 00000100:00000010:3.0:1788880264.831108:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a5ad6403. 00000100:00000200:1.0:1788880264.831110:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780644480, offset 0 00000800:00000200:0.0:1788880264.831117:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880264.831117:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880264.831124:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305901007 00000400:00000200:1.0:1788880264.831128:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880264.831135:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880264.831139:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008ea3b290. 00000100:00000040:0.0:1788880264.831143:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 1017436 usecs (1018 jiffies) 00000400:00000200:1.0:1788880264.831148:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880264.831149:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880264.831157:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305896954 00000400:00000200:1.0:1788880264.831164:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880264.831165:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880264.831166:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305901007 00000800:00000200:2.0:1788880264.831167:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:1.0:1788880264.831170:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000010:2.0:1788880264.831178:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000100:00000001:3.0:1788880264.831178:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880264.831180:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880264.831181:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305896954 00000100:00000001:3.0:1788880264.831187:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880264.831190:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305901007 00000400:00000200:2.0:1788880264.831190:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880264.831193:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880264.831196:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880264.831200:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305896954 00000100:00000001:0.0:1788880264.831204:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305901007 00000400:00000200:1.0:1788880264.831205:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [105:105:254:4294967295] 192.168.204.111@tcp LPNI seq info [105:105:6:4294967295] 00000100:00000001:3.0:1788880264.831208:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880264.831213:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880264.831214:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880264.831216:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305896954 00000100:00000001:0.0:1788880264.831227:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305901007 00000100:00000001:3.0:1788880264.831230:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880264.831231:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880264.831263:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880264.831271:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880264.831283:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a165c7f9 00000800:00000010:1.0:1788880264.831288:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000400:00000010:2.0:1788880264.831289:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a165c7f9. 00000100:00000001:2.0:1788880264.831299:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:1.0:1788880264.831302:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880264.831308:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880280 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880264.831316:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880264.831322:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880264.831327:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880280 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880264.831329:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880264.831342:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880264.831345:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880264.831349:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 64440 usecs (65 jiffies) 00000100:00000001:2.0:1788880264.831350:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880264.831350:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880264.831357:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.831359:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305896002 00000100:00000001:0.0:1788880264.831365:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880264.831366:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.831367:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305901007 00000100:00000001:0.0:1788880264.831373:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.831374:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305896002 00000800:00000200:2.0:1788880264.831377:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880264.831379:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880264.831381:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305901007 00000800:00000010:2.0:1788880264.831383:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000100:00000001:0.0:1788880264.831387:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880264.831388:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305896002 00000100:00000001:0.0:1788880264.831392:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880264.831392:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880264.831394:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305901007 00000100:00000001:0.0:1788880264.831398:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880264.831401:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305896002 00000100:00000001:0.0:1788880264.831402:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880264.831405:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880264.831407:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.831409:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305901007 00000400:00000200:2.0:1788880264.831412:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000dc289a80 00000100:00000001:0.0:1788880264.831413:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.831417:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880264.831419:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000dc289a80. 00000100:00000001:1.0:1788880264.831422:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.831427:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880264.831429:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880264.831430:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880264.831437:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880264.831442:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880264.831446:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880264.831449:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880264.831451:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880264.831455:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880264.831456:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880264.831459:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.831465:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880264.831475:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880264.831477:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880264.831519:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880264.831521:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:2.0:1788880264.831525:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880264.831528:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880264.831531:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000002fa6ff6 00000400:00000010:2.0:1788880264.831538:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000002fa6ff6. 00000100:00000001:2.0:1788880264.831542:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880264.831545:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880264.831550:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880264.831551:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880264.831555:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880264.831556:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880264.832967:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880264.832979:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880264.832982:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880264.832984:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880264.832991:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880264.833002:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b02f80 00000400:00000200:3.0:1788880264.833008:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x41d [1] + 224 00000800:00000001:3.0:1788880264.833052:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880264.833058:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880264.833061:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880264.833064:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880264.833069:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880264.833073:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880264.833081:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880264.833100:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880264.833106:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880264.833109:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880264.833150:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880264.833154:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880264.833160:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880264.833161:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880264.833163:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000be05c49f. 00000400:00000200:1.0:1788880264.833167:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 00000400:00000010:1.0:1788880264.833169:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000100:00000001:1.0:1788880264.833172:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880264.833176:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880264.833185:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880264.833199:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880264.833201:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880264.833203:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880264.833206:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880264.833208:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880264.833210:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880264.833213:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880264.833218:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=92 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880264.833257:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=92 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880264.833261:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880264.833263:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880264.833266:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880264.833268:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.833271:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880264.833286:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880264.833293:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880264.833296:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880264.833298:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880264.833301:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880264.833309:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000071512652 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780644224:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880264.833314:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880264.833316:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880264.833318:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000071512652 x1875776780644224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880264.833324:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880264.833326:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880264.833329:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000dc9842d3. 02000000:00000001:1.0:1788880264.833333:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880264.833335:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880264.833337:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880264.833345:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880264.833347:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 02000000:00000001:1.0:1788880264.833351:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880264.833353:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880264.833355:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000071512652. 00000100:00000001:1.0:1788880264.833360:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880264.833361:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880264.833363:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880264.833367:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880264.833369:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.833370:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880264.833372:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880264.833616:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880264.833623:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880264.833625:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880264.833627:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880264.833633:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880264.833641:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03000 00000400:00000200:2.0:1788880264.833647:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x425 [1] + 224 00000800:00000001:2.0:1788880264.833651:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880264.833657:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880264.833659:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880264.833662:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880264.833665:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880264.833668:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880264.833677:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880264.833696:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880264.833703:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880264.833715:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880264.833718:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880264.833721:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880264.833726:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880264.833734:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880264.833734:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03080 00000100:00000001:0.0:1788880264.833739:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880264.833742:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880264.833746:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880264.833747:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000400:00000200:0.0:1788880264.833751:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000010:0.0:1788880264.833752:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880264.833755:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880264.833759:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x42d [1] + 224 00000100:00000200:0.0:1788880264.833761:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880264.833768:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880264.833774:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880264.833777:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880264.833782:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880264.833791:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880264.833793:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880264.833796:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880264.833796:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880264.833799:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880264.833802:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000200:2.0:1788880264.833803:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880264.833804:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880264.833807:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880264.833811:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880264.833819:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=82 v=5 (5 0 0 0) 00000100:00000040:2.0:1788880264.833821:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880264.833824:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=82 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880264.833841:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880264.833844:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880264.833846:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880264.833849:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880264.833852:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880264.833853:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:0.0:1788880264.833854:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880264.833858:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880264.833859:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880264.833860:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1788880264.833864:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880264.833865:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880264.833874:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880264.833877:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880264.833879:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880264.833881:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880264.833888:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780644352:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880264.833893:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880264.833895:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880264.833897:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ea101a56 x1875776780644352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880264.833915:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880264.833916:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880264.833924:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003720a685. 00000100:00000001:1.0:1788880264.833925:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880264.833927:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880264.833929:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000001:0.0:1788880264.833930:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880264.833933:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000001:1.0:1788880264.833935:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000001:0.0:1788880264.833936:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880264.833939:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006aca3ff7. 00000400:00000001:1.0:1788880264.833939:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880264.833944:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000020fda082. 02000000:00000001:0.0:1788880264.833945:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000200:1.0:1788880264.833949:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000400:00000010:1.0:1788880264.833951:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000001:1.0:1788880264.833954:0:201288:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:0.0:1788880264.833955:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880264.833960:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ea101a56. 00000100:00000200:1.0:1788880264.833960:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880264.833966:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880264.833968:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880264.833972:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880264.833978:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:1.0:1788880264.833978:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880264.833981:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880264.833985:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880264.833988:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.833995:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880264.833998:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880264.834000:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880264.834002:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880264.834004:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880264.834005:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880264.834008:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880264.834012:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=92 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880264.834015:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=66 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880264.834018:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880264.834019:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880264.834021:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880264.834024:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880264.834037:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.834039:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880264.834041:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880264.834042:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880264.834044:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880264.834047:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880264.834053:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880264.834055:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880264.834056:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880264.834059:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880264.834067:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780644480:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880264.834071:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880264.834073:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880264.834075:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000910cb22c x1875776780644480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880280 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880264.834081:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880264.834083:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880264.834084:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000057e92cbc. 02000000:00000001:1.0:1788880264.834087:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880264.834088:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880264.834089:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880264.834091:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880264.834092:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003139f6ba. 02000000:00000001:1.0:1788880264.834094:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880264.834096:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880264.834097:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000910cb22c. 00000100:00000001:1.0:1788880264.834099:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880264.834100:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880264.834107:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880264.834111:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880264.834112:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880264.834114:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880264.834115:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880264.894365:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880264.894372:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880264.894378:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880264.894380:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880264.894465:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880264.894469:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880264.894472:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880265.918383:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880265.918390:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880265.918395:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880265.918399:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880265.918402:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880265.918404:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880265.918408:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880265.918411:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880265.918414:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880265.918416:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880265.918480:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880265.918484:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880265.918487:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880266.942331:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880266.942340:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880266.942346:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880266.942348:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880266.942407:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880266.942410:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880266.942413:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880267.966394:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880267.966406:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880267.966415:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880267.966416:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880267.966480:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880267.966484:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880267.966487:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880268.990360:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880268.990366:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880268.990370:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880268.990371:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880268.990419:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880268.990422:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880268.990424:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880269.950305:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305901007, at 4305901056 00000100:00000040:3.0:1788880269.950327:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305901007, at 4305901056 00000100:00000010:1.0:1788880269.950329:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002c2d3615. 00000100:00000010:3.0:1788880269.950338:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008ea3b290. 00000100:00000040:2.0:1788880269.951319:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305901007, at 4305901057 00000100:00000040:3.0:1788880269.951326:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:2.0:1788880269.951328:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a5ad6403. 00000100:00000001:3.0:1788880269.951335:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880269.951343:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880269.951348:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880269.951352:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880269.951354:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880269.951359:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880269.951367:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:3.0:1788880269.951384:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880269.951393:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880269.951404:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880269.951407:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880269.951412:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_00+0] req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880269.951440:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880269.951442:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880269.951446:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880269.951448:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880269.951451:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880269.951453:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880269.951455:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880269.951456:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880269.951458:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880269.951460:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5b428a0. 00000100:00000001:3.0:1788880269.951464:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880269.951465:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880269.951468:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880269.951469:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880269.951470:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880269.951476:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880269.951477:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880269.951487:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880269.951489:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880269.951494:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_01+1] req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880269.951518:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880269.951520:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880269.951521:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880269.951526:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880269.951528:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880269.951531:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1788880269.951532:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:3.0:1788880269.951535:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 02000000:00000001:1.0:1788880269.951536:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880269.951540:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00100000:1.0:1788880269.951543:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780644608:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880269.951545:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880269.951547:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880269.951550:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880269.951555:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880269.951556:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000010:3.0:1788880269.951559:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000055d7d92. 02000000:00000001:1.0:1788880269.951560:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880269.951564:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880269.951565:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880269.951571:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000089aec2fa. 00000100:00000040:3.0:1788880269.951571:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880269.951576:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880269.951583:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000020fda082. 00000400:00000010:1.0:1788880269.951591:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 00000100:00000001:3.0:1788880269.951591:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880269.951593:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880269.951599:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780644608, portal 25 00000100:00000040:3.0:1788880269.951599:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_02+2] req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880269.951605:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880269.951608:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000040:1.0:1788880269.951616:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880285 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880269.951624:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880269.951625:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880269.951629:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880269.951630:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:1.0:1788880269.951635:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000100:00000040:3.0:1788880269.951635:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000200:1.0:1788880269.951641:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780644608, offset 0 00000100:00000040:0.0:1788880269.951645:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305901007, at 4305901057 00000400:00000200:1.0:1788880269.951649:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880269.951653:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11238) 00000100:00000010:0.0:1788880269.951654:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000067a34a59. 00000400:00000200:1.0:1788880269.951665:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880269.951669:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880269.951673:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880269.951673:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880269.951676:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788880269.951679:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788880269.951683:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880269.951687:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880269.951693:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880269.951697:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880269.951698:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880269.951699:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880269.951702:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880269.951702:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [106:106:256:4294967295] 192.168.204.111@tcp LPNI seq info [106:106:8:4294967295] 02000000:00000001:3.0:1788880269.951704:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880269.951707:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880269.951711:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880269.951715:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780644864:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880269.951721:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788880269.951724:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:3.0:1788880269.951725:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880269.951727:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880269.951729:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880269.951732:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a581a743. 02000000:00000001:3.0:1788880269.951735:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880269.951739:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880269.951739:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000de273b16. 00000800:00000010:1.0:1788880269.951745:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000400:00000010:3.0:1788880269.951751:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000999bcd04. 00000800:00000200:1.0:1788880269.951755:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:3.0:1788880269.951756:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780644864, portal 10 00000100:00000001:3.0:1788880269.951760:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880269.951761:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000800:00000200:1.0:1788880269.951767:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:3.0:1788880269.951768:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880269.951773:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880269.951781:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880269.951784:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880269.951786:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cf82a3fa. 00000100:00000200:3.0:1788880269.951788:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780644864, offset 0 00000400:00000200:3.0:1788880269.951792:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880269.951797:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880269.951797:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880269.951803:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880269.951804:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305906128 00000400:00000200:3.0:1788880269.951806:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880269.951810:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880269.951812:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880269.951815:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [107:107:255:4294967295] 192.168.204.111@tcp LPNI seq info [107:107:7:4294967295] 00000100:00000010:1.0:1788880269.951824:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000400:00000200:3.0:1788880269.951825:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880269.951831:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:3.0:1788880269.951835:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880269.951838:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305906128 00000800:00000010:3.0:1788880269.951840:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000749ed09e. 00000100:00000010:1.0:1788880269.951843:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000100:00000001:2.0:1788880269.951847:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880269.951848:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:3.0:1788880269.951848:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880269.951852:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880269.951855:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880269.951856:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305906128 00000800:00000200:3.0:1788880269.951859:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:1.0:1788880269.951863:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000100:00000040:2.0:1788880269.951864:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880269.951864:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000749ed09e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880269.951868:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880269.951870:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880269.951874:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305906128 00000100:00000040:3.0:1788880269.951876:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 61692 usecs (62 jiffies) 00000100:00000010:1.0:1788880269.951880:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000100:00000001:3.0:1788880269.951882:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305901119 00000100:00000001:1.0:1788880269.951886:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880269.951890:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880269.951892:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880269.951895:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305906128 00000100:00000001:1.0:1788880269.951896:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880269.951901:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880269.951901:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880269.951902:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305901119 00000100:00000001:2.0:1788880269.951905:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880269.951906:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305906128 00000100:00000001:1.0:1788880269.951910:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788880269.951911:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:0.0:1788880269.951912:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880269.951914:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000001:2.0:1788880269.951915:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880269.951918:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305901119 00000800:00000010:0.0:1788880269.951919:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:1.0:1788880269.951919:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880269.951922:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305906128 00000100:00000001:1.0:1788880269.951923:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880269.951923:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880269.951926:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305901119 00000100:00000001:1.0:1788880269.951927:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880269.951932:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00100000:2.0:1788880269.951932:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780644736:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880269.951932:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305906128 00000100:00000001:3.0:1788880269.951937:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880269.951940:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880269.951942:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880269.951944:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880269.951944:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880269.951945:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880269.951946:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880269.951949:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880269.951951:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880269.951953:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880269.951957:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880269.951958:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880269.951963:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880269.951965:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 02000000:00000010:2.0:1788880269.951965:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008e4ae4dd. 00000100:00000001:3.0:1788880269.951966:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:0.0:1788880269.951969:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 02000000:00000001:2.0:1788880269.951970:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880269.951970:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880269.951977:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000400:00000010:2.0:1788880269.951977:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000064bd5a64. 00000100:00000001:3.0:1788880269.951979:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880269.951983:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880285 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880269.951996:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000032b7316b. 00000100:00000001:0.0:1788880269.952000:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:2.0:1788880269.952001:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780644736, portal 10 00000100:00000040:0.0:1788880269.952005:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880285 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880269.952005:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880269.952008:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000100:00000001:0.0:1788880269.952017:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880269.952019:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880269.952020:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880269.952029:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880269.952031:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880269.952035:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c013629f. 00000100:00000200:2.0:1788880269.952038:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780644736, offset 0 00000800:00000200:0.0:1788880269.952039:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880269.952043:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000749ed09e. 00000400:00000200:2.0:1788880269.952046:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880269.952048:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880269.952055:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880269.952055:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880269.952063:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880269.952067:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880269.952069:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880269.952072:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880269.952074:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [108:108:255:4294967295] 192.168.204.111@tcp LPNI seq info [108:108:7:4294967295] 00000400:00000200:2.0:1788880269.952093:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880269.952098:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880269.952099:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cf82a3fa 00000400:00000010:0.0:1788880269.952103:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cf82a3fa. 00000800:00000010:2.0:1788880269.952103:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f9812869. 00000100:00000001:0.0:1788880269.952106:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:2.0:1788880269.952110:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:0.0:1788880269.952112:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880269.952120:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880269.952125:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f9812869 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880269.952127:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880269.952132:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880269.952132:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880269.952137:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 978455 usecs (979 jiffies) 00000100:00000001:2.0:1788880269.952142:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305902036 00000100:00000001:0.0:1788880269.952145:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880269.952147:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305906128 00000100:00000001:0.0:1788880269.952148:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880269.952158:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305902036 00000100:00000001:2.0:1788880269.952161:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305906128 00000100:00000001:2.0:1788880269.952163:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305902036 00000100:00000001:2.0:1788880269.952165:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305906128 00000100:00000001:2.0:1788880269.952167:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305902036 00000100:00000001:2.0:1788880269.952169:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305906128 00000100:00000001:2.0:1788880269.952172:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880269.952186:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880269.952190:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880269.952202:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880269.952203:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880269.952206:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f9812869. 00000100:00000001:2.0:1788880269.952208:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880269.952209:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880269.952212:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880269.952214:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880269.952218:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c013629f 00000400:00000010:0.0:1788880269.952219:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c013629f. 00000100:00000001:0.0:1788880269.952222:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880269.952222:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880269.952225:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880269.952226:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880269.952231:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880269.952266:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880269.952275:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880269.952278:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880269.952284:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880269.952286:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880269.954457:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880269.954478:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880269.954483:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880269.954486:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880269.954499:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880269.954517:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03200 00000400:00000200:3.0:1788880269.954527:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x43d [1] + 224 00000800:00000001:3.0:1788880269.954542:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880269.954547:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880269.954550:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880269.954555:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880269.954563:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880269.954571:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880269.954584:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880269.954623:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880269.954650:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880269.954655:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880269.954658:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880269.954660:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880269.954668:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880269.954670:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880269.954676:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880269.954677:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b03100 00000400:00000200:3.0:1788880269.954689:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x435 [1] + 224 00000800:00000001:3.0:1788880269.954693:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880269.954695:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880269.954699:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880269.954701:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000de273b16. 00000400:00000200:0.0:1788880269.954708:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000999bcd04 00000800:00000001:3.0:1788880269.954709:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880269.954713:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000999bcd04. 00000400:00000200:3.0:1788880269.954715:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880269.954721:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:3.0:1788880269.954724:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880269.954730:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880269.954734:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880269.954741:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880269.954775:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880269.954779:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880269.954814:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880269.954819:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880269.954821:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880269.954824:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880269.954826:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880269.954829:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880269.954829:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788880269.954835:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880269.954838:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880269.954844:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880269.954845:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=97 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880269.954851:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=71 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880269.954854:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880269.954856:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880269.954858:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880269.954865:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880269.954872:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880269.954887:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880269.954890:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880269.954892:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880269.954893:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880269.954898:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880269.954905:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880269.954908:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880269.954910:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880269.954913:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880269.954920:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780644864:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880269.954924:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880269.954926:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880269.954928:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780644864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880269.954935:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880269.954937:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880269.954941:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a581a743. 02000000:00000001:0.0:1788880269.954945:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880269.954947:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880269.954948:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880269.954950:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880269.954953:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000055d7d92. 02000000:00000001:0.0:1788880269.954957:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880269.954959:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880269.954960:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:0.0:1788880269.954963:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880269.954964:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880269.954966:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880269.954971:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880269.954972:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880269.954975:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880269.954982:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880269.955214:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880269.955218:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880269.955222:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880269.955257:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880269.955268:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000020fda082. 00000400:00000200:1.0:1788880269.955271:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000400:00000010:1.0:1788880269.955273:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000001:1.0:1788880269.955276:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880269.955281:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880269.955289:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880269.955296:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880269.955297:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880269.955299:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880269.955301:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880269.955303:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880269.955304:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880269.955306:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880269.955308:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=97 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880269.955311:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=97 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880269.955313:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880269.955314:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880269.955316:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880269.955317:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880269.955319:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880269.955322:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880269.955327:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880269.955329:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880269.955330:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880269.955333:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880269.955339:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780644608:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880269.955343:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880269.955344:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880269.955346:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780644608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880269.955357:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880269.955358:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880269.955360:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000089aec2fa. 02000000:00000001:1.0:1788880269.955363:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880269.955364:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880269.955365:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880269.955367:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880269.955368:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 02000000:00000001:1.0:1788880269.955371:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880269.955373:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880269.955374:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:1.0:1788880269.955376:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880269.955376:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880269.955377:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880269.955380:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880269.955381:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880269.955382:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880269.955383:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880269.955504:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880269.955519:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880269.955522:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880269.955525:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880269.955532:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880269.955544:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03180 00000400:00000200:2.0:1788880269.955550:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x445 [1] + 224 00000800:00000001:2.0:1788880269.955556:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880269.955561:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880269.955563:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880269.955566:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880269.955570:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880269.955573:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880269.955580:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880269.955601:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880269.955607:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880269.955609:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880269.955634:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880269.955638:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880269.955642:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880269.955643:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880269.955645:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000064bd5a64. 00000400:00000200:1.0:1788880269.955649:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000032b7316b 00000400:00000010:1.0:1788880269.955650:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000032b7316b. 00000100:00000001:1.0:1788880269.955653:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880269.955657:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880269.955676:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880269.955682:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880269.955684:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880269.955686:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880269.955687:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880269.955689:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880269.955697:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880269.955699:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880269.955704:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=87 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880269.955707:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=87 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880269.955709:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880269.955711:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880269.955713:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880269.955715:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880269.955717:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880269.955719:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880269.955720:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880269.955722:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880269.955725:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880269.955731:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880269.955747:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880269.955749:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880269.955752:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880269.955759:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780644736:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880269.955763:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880269.955765:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880269.955767:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780644736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880285 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880269.955774:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880269.955775:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880269.955778:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008e4ae4dd. 02000000:00000001:1.0:1788880269.955781:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880269.955782:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880269.955783:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880269.955785:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880269.955787:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5b428a0. 02000000:00000001:1.0:1788880269.955790:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880269.955797:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880269.955798:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:1.0:1788880269.955800:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880269.955802:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880269.955803:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880269.955807:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880269.955808:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880269.955810:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880269.955811:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880270.014361:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880270.014368:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880270.014373:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880270.014374:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880270.014413:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880270.014416:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880270.014418:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880271.038341:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880271.038347:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880271.038353:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880271.038354:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880271.038407:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880271.038411:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880271.038412:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880272.062320:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880272.062332:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880272.062342:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880272.062344:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880272.062456:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880272.062460:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880272.062462:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880273.086348:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880273.086356:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880273.086362:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880273.086364:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880273.086417:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880273.086421:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880273.086424:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880274.110364:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880274.110373:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880274.110376:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880274.110378:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880274.110383:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880274.110385:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880274.110386:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880274.110388:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880274.110392:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880274.110393:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880274.110442:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880274.110445:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880274.110447:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880275.070319:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305906128, at 4305906176 00000100:00000040:1.0:1788880275.070319:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305906128, at 4305906176 00000100:00000040:2.0:1788880275.070319:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305906128, at 4305906176 00000100:00000010:1.0:1788880275.070335:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:2.0:1788880275.070335:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000010:0.0:1788880275.070338:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000040:3.0:1788880275.070358:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880275.070363:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880275.070368:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000020:00000040:3.0:1788880275.070371:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880275.070375:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880275.070378:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880275.070381:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880275.070387:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:3.0:1788880275.070392:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880275.070400:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880275.070408:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880275.070410:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880275.070425:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_03+3] req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880275.070468:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880275.070471:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880275.070473:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880275.070476:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880275.070480:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880275.070481:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880275.070491:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880275.070493:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880275.070497:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880275.070499:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880275.070502:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880275.070505:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780644992:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880275.070508:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880275.070513:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880275.070516:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880275.070519:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880275.070523:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880275.070524:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b1120e74. 02000000:00000001:0.0:1788880275.070527:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880275.070532:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000400:00000010:0.0:1788880275.070538:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000200:0.0:1788880275.070544:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780644992, portal 25 00000100:00000001:0.0:1788880275.070546:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880275.070547:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000100:00000040:0.0:1788880275.070552:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880291 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880275.070567:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880275.070570:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880275.070573:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000100:00000200:0.0:1788880275.070577:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780644992, offset 0 00000100:00000040:3.0:1788880275.070583:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:0.0:1788880275.070587:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880275.070590:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880275.070594:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880275.070597:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880275.070600:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000200:0.0:1788880275.070602:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880275.070603:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880275.070606:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880275.070610:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:3.0:1788880275.070611:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000400:00000200:0.0:1788880275.070616:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880275.070616:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880275.070622:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:0.0:1788880275.070623:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880275.070629:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880275.070634:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880275.070636:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880275.070637:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [109:109:256:4294967295] 192.168.204.111@tcp LPNI seq info [109:109:8:4294967295] 00000100:00000040:3.0:1788880275.070642:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_00+0] req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880275.070653:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880275.070659:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880275.070661:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880275.070662:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880275.070667:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000800:00000010:0.0:1788880275.070669:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 00000100:00000010:3.0:1788880275.070669:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000100:00000001:1.0:1788880275.070672:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:3.0:1788880275.070673:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880275.070676:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880275.070677:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880275.070680:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:0.0:1788880275.070681:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:3.0:1788880275.070681:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880275.070684:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880275.070686:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880275.070688:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000252f4482. 00000100:00000001:2.0:1788880275.070690:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880275.070693:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880275.070696:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880275.070696:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880275.070702:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000200:0.0:1788880275.070704:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880275.070705:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880275.070709:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880275.070715:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880275.070717:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880275.070718:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880275.070724:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305911247 02000000:00000001:1.0:1788880275.070725:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880275.070733:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009f22a9a0. 00000100:00100000:1.0:1788880275.070734:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780645120:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880275.070737:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880275.070740:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880275.070740:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880275.070741:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880275.070747:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880275.070748:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305911247 00000100:00000040:3.0:1788880275.070750:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_01+1] req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880275.070751:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880275.070755:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000bc13d1e4. 02000000:00000001:1.0:1788880275.070757:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880275.070762:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 02000000:00000010:1.0:1788880275.070764:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000044022300. 00000100:00000001:0.0:1788880275.070768:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305911247 02000000:00000001:1.0:1788880275.070768:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880275.070769:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880275.070774:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:0.0:1788880275.070775:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d2bf5f. 00000400:00000010:1.0:1788880275.070777:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000020fda082. 00000100:00000040:3.0:1788880275.070777:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880275.070780:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:2.0:1788880275.070780:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:3.0:1788880275.070785:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11243) 00000400:00000010:1.0:1788880275.070786:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 00000800:00000010:2.0:1788880275.070786:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000100:00000001:0.0:1788880275.070787:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305911247 00000100:00000200:1.0:1788880275.070793:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780645120, portal 10 00000400:00000200:2.0:1788880275.070794:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880275.070795:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305906128 00000100:00000001:1.0:1788880275.070796:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880275.070798:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000001:3.0:1788880275.070799:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880275.070802:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880275.070803:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880275.070806:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880275.070809:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880275.070809:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880275.070810:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000100:00000040:3.0:1788880275.070811:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880275.070814:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00000001:0.0:1788880275.070816:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880275.070820:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880275.070824:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880275.070827:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880291 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880275.070830:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880275.070841:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880275.070843:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880275.070845:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880275.070848:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880275.070850:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880275.070851:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880275.070853:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880291 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880275.070854:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000001:0.0:1788880275.070855:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880275.070858:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880275.070860:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780645120, offset 0 00000100:00000001:0.0:1788880275.070864:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880275.070870:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880275.070870:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880275.070873:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:1.0:1788880275.070877:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880275.070878:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880275.070881:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880275.070882:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:3.0:1788880275.070884:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880275.070886:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880275.070887:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880275.070889:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880275.070894:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780645248:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880275.070895:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880275.070900:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880275.070900:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788880275.070904:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [110:110:255:4294967295] 192.168.204.111@tcp LPNI seq info [110:110:7:4294967295] 02000000:00000001:3.0:1788880275.070904:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880275.070906:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880275.070910:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880275.070913:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004080df20. 00000400:00000200:1.0:1788880275.070916:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:3.0:1788880275.070917:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880275.070922:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f2f8fd80. 00000800:00000200:1.0:1788880275.070925:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880275.070927:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ec90f18d. 00000800:00000010:1.0:1788880275.070929:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000200:3.0:1788880275.070931:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780645248, portal 10 00000100:00000001:3.0:1788880275.070935:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:1.0:1788880275.070936:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880275.070938:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000800:00000200:1.0:1788880275.070940:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880275.070942:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000040:3.0:1788880275.070952:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880275.070957:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880275.070958:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880275.070959:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880275.070962:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 942408 usecs (943 jiffies) 00000400:00000010:3.0:1788880275.070963:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000035d99bc2. 00000100:00000001:1.0:1788880275.070967:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305907119 00000100:00000200:3.0:1788880275.070967:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780645248, offset 0 00000100:00000001:1.0:1788880275.070972:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305911247 00000400:00000200:3.0:1788880275.070976:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880275.070977:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305907119 00000100:00000001:1.0:1788880275.070980:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305911247 00000100:00000001:1.0:1788880275.070981:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305907119 00000100:00000001:1.0:1788880275.070982:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305911247 00000100:00000001:1.0:1788880275.070983:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305907119 00000100:00000001:1.0:1788880275.070986:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305911247 00000400:00000200:3.0:1788880275.070986:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880275.070987:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880275.070991:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880275.070994:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880275.070994:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880275.070998:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880275.071000:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880275.071002:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880275.071004:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880275.071006:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880275.071007:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880275.071008:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880275.071009:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880275.071011:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1788880275.071013:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:3.0:1788880275.071018:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000010:2.0:1788880275.071019:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:2.0:1788880275.071023:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880275.071024:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [111:111:255:4294967295] 192.168.204.111@tcp LPNI seq info [111:111:7:4294967295] 00000400:00000200:2.0:1788880275.071030:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880275.071038:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:2.0:1788880275.071043:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000800:00000200:3.0:1788880275.071043:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880275.071047:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000400:00000010:2.0:1788880275.071051:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000800:00000200:3.0:1788880275.071054:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880275.071056:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880275.071060:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880275.071066:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880275.071068:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880275.071071:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880275.071072:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880275.071113:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880275.071116:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880275.071126:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880275.071129:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 59183 usecs (60 jiffies) 00000100:00000001:3.0:1788880275.071132:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305906236 00000100:00000001:3.0:1788880275.071134:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305911247 00000100:00000001:3.0:1788880275.071136:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305906236 00000100:00000001:3.0:1788880275.071139:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305911247 00000100:00000001:3.0:1788880275.071141:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305906236 00000100:00000001:3.0:1788880275.071144:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305911247 00000100:00000001:3.0:1788880275.071146:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305906236 00000100:00000001:3.0:1788880275.071165:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305911247 00000100:00000001:3.0:1788880275.071168:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880275.071169:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880275.071170:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880275.071172:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000100:00000001:3.0:1788880275.071175:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880275.071176:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880275.071179:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880275.071182:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880275.071185:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000035d99bc2 00000400:00000010:2.0:1788880275.071186:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000035d99bc2. 00000100:00000001:3.0:1788880275.071186:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880275.071190:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880275.071191:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880275.071195:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880275.071195:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880275.071198:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880275.071201:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880275.071206:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880275.071207:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880275.071209:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880275.071214:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880275.071215:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880275.072471:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880275.072480:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880275.072483:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880275.072485:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880275.072492:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880275.072502:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b03280 00000400:00000200:2.0:1788880275.072508:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x44d [1] + 224 00000800:00000001:2.0:1788880275.072513:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880275.072515:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880275.072530:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880275.072533:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880275.072537:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880275.072540:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880275.072549:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880275.072564:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880275.072569:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880275.072572:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880275.072573:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880275.072574:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880275.072577:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880275.072581:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03300 00000400:00000200:2.0:1788880275.072584:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x455 [1] + 224 00000800:00000001:2.0:1788880275.072587:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880275.072589:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880275.072590:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880275.072593:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880275.072593:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880275.072596:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880275.072597:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880275.072601:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880275.072602:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880275.072604:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880275.072607:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000400:00000200:0.0:1788880275.072611:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000100:00000040:2.0:1788880275.072613:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880275.072614:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880275.072619:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880275.072624:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880275.072627:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:0.0:1788880275.072632:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880275.072633:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880275.072637:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880275.072641:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880275.072642:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880275.072645:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880275.072645:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880275.072650:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880275.072653:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000200:2.0:1788880275.072653:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880275.072656:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880275.072657:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880275.072659:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880275.072660:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880275.072663:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03380 02000000:00000001:0.0:1788880275.072665:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880275.072665:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880275.072668:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:0.0:1788880275.072670:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=103 v=5 (5 0 0 0) 00000400:00000010:1.0:1788880275.072671:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000020fda082. 00000100:00001000:0.0:1788880275.072676:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=103 v=5 (1 0 0 0) 00000400:00000200:1.0:1788880275.072676:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000400:00000010:1.0:1788880275.072680:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000001:0.0:1788880275.072681:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880275.072683:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880275.072683:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:0.0:1788880275.072685:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:2.0:1788880275.072687:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x45d [1] + 224 00010000:00000001:0.0:1788880275.072688:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880275.072690:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880275.072692:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880275.072695:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880275.072701:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880275.072705:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:0.0:1788880275.072707:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880275.072710:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880275.072716:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880275.072722:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880275.072728:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880275.072733:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880275.072738:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880275.072740:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880275.072743:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880275.072745:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880275.072746:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880275.072748:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880275.072752:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:0.0:1788880275.072753:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880275.072755:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880275.072758:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880275.072764:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880275.072768:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:0.0:1788880275.072771:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780644992:192.168.204.111@tcp:400: 00000100:00001000:1.0:1788880275.072772:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=93 v=5 (5 0 0 0) 00000800:00000001:2.0:1788880275.072772:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880275.072776:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880275.072778:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880275.072779:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=93 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880275.072781:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880275.072785:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780644992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880275.072785:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880275.072788:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880275.072794:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880275.072795:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880275.072796:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880275.072799:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00010000:00000001:1.0:1788880275.072799:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880275.072800:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:0.0:1788880275.072801:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b1120e74. 00000400:00000001:3.0:1788880275.072804:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:0.0:1788880275.072806:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880275.072807:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000001:3.0:1788880275.072807:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000020:00000001:0.0:1788880275.072810:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:1.0:1788880275.072811:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000010:3.0:1788880275.072811:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000f2f8fd80. 00000020:00000040:0.0:1788880275.072813:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880275.072817:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000200:3.0:1788880275.072817:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ec90f18d 00000020:00000001:0.0:1788880275.072818:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000010:3.0:1788880275.072821:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ec90f18d. 02000000:00000010:0.0:1788880275.072822:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 00000100:00000001:1.0:1788880275.072822:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880275.072824:0:201286:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:0.0:1788880275.072829:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000200:3.0:1788880275.072830:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880275.072832:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:1.0:1788880275.072833:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880275.072835:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:0.0:1788880275.072840:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880275.072843:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880275.072847:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880275.072853:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880275.072855:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880275.072856:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880275.072858:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880275.072859:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880275.072862:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880275.072864:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880275.072866:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880275.072873:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880275.072873:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880275.072875:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880275.072876:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880275.072878:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880275.072879:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880275.072881:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880275.072885:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880275.072890:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=103 v=5 (5 0 0 0) 00000100:00100000:1.0:1788880275.072893:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780645120:192.168.204.111@tcp:400: 00000100:00001000:3.0:1788880275.072894:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=77 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880275.072898:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880275.072898:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880275.072901:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880275.072902:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00010000:00000001:3.0:1788880275.072903:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:1.0:1788880275.072905:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780645120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880275.072910:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880275.072912:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880275.072913:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000044022300. 00010000:00080000:3.0:1788880275.072915:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880275.072916:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880275.072918:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880275.072920:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880275.072925:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880275.072927:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 00010000:00000001:3.0:1788880275.072929:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880275.072930:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880275.072932:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880275.072932:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880275.072935:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000010:1.0:1788880275.072936:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:3.0:1788880275.072939:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880275.072940:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880275.072941:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880275.072942:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880275.072946:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880275.072948:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880275.072950:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880275.072953:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880275.072955:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880275.072958:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880275.072959:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880275.072963:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880275.072965:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880275.072966:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880275.072972:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780645248:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880275.072975:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880275.072976:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880275.072977:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780645248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880291 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880275.072981:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880275.072982:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880275.072983:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004080df20. 02000000:00000001:3.0:1788880275.072985:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880275.072985:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880275.072986:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880275.072988:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880275.072989:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000252f4482. 02000000:00000001:3.0:1788880275.072990:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880275.072991:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880275.072992:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:3.0:1788880275.072994:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880275.072997:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880275.072998:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880275.073001:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880275.073002:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880275.073003:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880275.073004:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880275.134279:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880275.134285:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880275.134289:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880275.134290:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880275.134339:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880275.134343:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880275.134345:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880276.159277:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880276.159283:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880276.159286:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880276.159288:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880276.159290:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880276.159293:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880276.159295:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880276.159310:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880276.159311:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880276.159313:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880276.159363:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880276.159367:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880276.159370:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880277.182328:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880277.182335:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880277.182342:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880277.182343:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880277.182406:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880277.182411:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880277.182416:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880278.206385:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880278.206395:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880278.206403:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880278.206404:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880278.206464:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880278.206468:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880278.206472:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880279.230340:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880279.230347:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880279.230354:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880279.230355:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880279.230402:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880279.230420:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880279.230423:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880280.190327:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305911247, at 4305911296 00000100:00000040:1.0:1788880280.190327:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305911247, at 4305911296 00000100:00000040:2.0:1788880280.190329:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305911247, at 4305911296 00000100:00000040:3.0:1788880280.190333:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305911247, at 4305911296 00000100:00000010:1.0:1788880280.190344:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000bc13d1e4. 00000100:00000010:2.0:1788880280.190346:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d2bf5f. 00000100:00000010:0.0:1788880280.190348:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009f22a9a0. 00000100:00000010:3.0:1788880280.190350:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000040:3.0:1788880280.190366:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880280.190371:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880280.190380:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788880280.190384:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880280.190388:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880280.190391:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880280.190396:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880280.190402:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000252f4482. 00000100:00000001:3.0:1788880280.190409:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880280.190419:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880280.190429:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880280.190431:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880280.190437:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_02+2] req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880280.190454:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880280.190456:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880280.190458:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880280.190461:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880280.190463:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880280.190465:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880280.190466:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880280.190467:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880280.190468:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880280.190472:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:3.0:1788880280.190472:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:1.0:1788880280.190475:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880280.190476:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880280.190481:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880280.190481:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880280.190486:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880280.190492:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880280.190495:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880280.190495:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880280.190500:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880280.190503:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880280.190504:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:1.0:1788880280.190506:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880280.190507:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880280.190513:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780645376:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880280.190514:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_03+3] req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880280.190520:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880280.190525:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880280.190527:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880280.190528:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880280.190529:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880280.190531:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:1.0:1788880280.190534:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e8a109c2. 00000100:00000001:3.0:1788880280.190535:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1788880280.190538:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880280.190543:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000020fda082. 00000100:00000001:0.0:1788880280.190545:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:3.0:1788880280.190545:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000100:00000001:0.0:1788880280.190549:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:3.0:1788880280.190550:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880280.190553:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880280.190554:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:1.0:1788880280.190557:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 02000000:00000001:3.0:1788880280.190557:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880280.190558:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880280.190560:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880280.190564:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000200:1.0:1788880280.190565:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780645376, portal 25 00000100:00000001:3.0:1788880280.190569:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880280.190570:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880280.190573:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000001:0.0:1788880280.190575:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788880280.190577:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880280.190578:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880280.190579:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880296 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880280.190582:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880280.190584:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880280.190589:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880280.190595:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780645504:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880280.190602:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880280.190604:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880280.190605:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880280.190606:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880280.190611:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880280.190612:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_00+0] req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880280.190614:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 02000000:00000001:0.0:1788880280.190618:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000200:1.0:1788880280.190618:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780645376, offset 0 02000000:00000001:0.0:1788880280.190622:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880280.190629:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880280.190629:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000010:0.0:1788880280.190636:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000026ba42a. 02000000:00000001:0.0:1788880280.190639:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880280.190642:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:0.0:1788880280.190644:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000400:00000200:1.0:1788880280.190648:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880280.190648:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880280.190650:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000400:00000200:1.0:1788880280.190652:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788880280.190654:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:0.0:1788880280.190657:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780645504, portal 10 00000400:00000200:1.0:1788880280.190658:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880280.190662:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880280.190663:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880280.190664:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880280.190665:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880280.190667:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000001:2.0:1788880280.190669:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880280.190671:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11248) 00000400:00000200:1.0:1788880280.190673:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [112:112:256:4294967295] 192.168.204.111@tcp LPNI seq info [112:112:8:4294967295] 00000100:00000001:2.0:1788880280.190673:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880280.190678:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880280.190680:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880280.190685:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880280.190689:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880280.190692:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880280.190706:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:1.0:1788880280.190707:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880280.190710:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000010:1.0:1788880280.190712:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000040:0.0:1788880280.190713:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880280.190713:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880280.190718:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:0.0:1788880280.190720:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ee6cf6a3. 00000800:00000200:1.0:1788880280.190722:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:2.0:1788880280.190726:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880280.190731:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880280.190732:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880280.190737:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00100000:2.0:1788880280.190742:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780645632:192.168.204.111@tcp:400: 00000100:00000200:0.0:1788880280.190743:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780645504, offset 0 00000100:00000001:2.0:1788880280.190750:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880280.190751:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880280.190754:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880280.190755:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880280.190758:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880280.190759:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305916367 02000000:00000001:2.0:1788880280.190762:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880280.190763:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:1.0:1788880280.190766:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000400:00000200:0.0:1788880280.190767:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:2.0:1788880280.190768:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008e4ae4dd. 00000400:00000200:0.0:1788880280.190772:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880280.190774:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 02000000:00000001:2.0:1788880280.190774:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880280.190776:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880280.190781:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:2.0:1788880280.190781:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009a2f7e83. 00000100:00000001:1.0:1788880280.190783:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305916367 00000100:00000010:1.0:1788880280.190789:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000400:00000010:2.0:1788880280.190789:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e1dae22. 00000400:00000200:0.0:1788880280.190791:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [113:113:255:4294967295] 192.168.204.111@tcp LPNI seq info [113:113:7:4294967295] 00000100:00000200:2.0:1788880280.190795:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780645632, portal 10 00000100:00000001:1.0:1788880280.190796:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:2.0:1788880280.190799:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880280.190801:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000400:00000200:0.0:1788880280.190808:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880280.190810:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305916367 00000100:00000040:2.0:1788880280.190811:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880280.190815:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000100:00000001:1.0:1788880280.190820:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:0.0:1788880280.190821:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880280.190824:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305916367 00000800:00000010:0.0:1788880280.190828:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 00000100:00000010:1.0:1788880280.190830:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000100:00000001:2.0:1788880280.190835:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880280.190836:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880280.190836:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000040:2.0:1788880280.190841:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880280.190842:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880280.190846:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009543d7d4. 00000800:00000200:0.0:1788880280.190847:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880280.190847:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880280.190851:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780645632, offset 0 00000800:00000200:0.0:1788880280.190852:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880280.190852:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880280.190856:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880280.190859:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880280.190859:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880280.190860:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880280.190864:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880280.190865:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 908845 usecs (909 jiffies) 00000100:00000001:1.0:1788880280.190867:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880280.190868:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880280.190869:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305912205 00000100:00000001:1.0:1788880280.190874:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:3.0:1788880280.190874:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000100:00000001:0.0:1788880280.190876:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305916367 00000400:00000200:2.0:1788880280.190877:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880280.190879:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880280.190881:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305912205 00000100:00000001:1.0:1788880280.190882:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880280.190882:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880280.190883:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880280.190886:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305916367 00000100:00000001:1.0:1788880280.190889:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880280.190889:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880280.190890:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305912205 00000100:00000001:0.0:1788880280.190895:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305916367 00000400:00000200:3.0:1788880280.190895:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880280.190896:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880280.190898:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305912205 00000100:00000001:0.0:1788880280.190903:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305916367 00000400:00000200:2.0:1788880280.190903:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [114:114:254:4294967295] 192.168.204.111@tcp LPNI seq info [114:114:6:4294967295] 00000100:00000001:0.0:1788880280.190906:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880280.190909:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880280.190911:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880280.190913:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880280.190917:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880280.190923:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880280.190926:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:2.0:1788880280.190927:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880280.190927:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000100:00000001:0.0:1788880280.190928:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880280.190932:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:3.0:1788880280.190932:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:0.0:1788880280.190935:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880280.190935:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5d1e8ed. 00000100:00000001:3.0:1788880280.190938:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880280.190939:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880280.190944:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:3.0:1788880280.190945:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880296 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880280.190953:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880280.190962:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5d1e8ed type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880280.190966:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880280.190969:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 56828 usecs (57 jiffies) 00000100:00000001:2.0:1788880280.190972:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305911353 00000100:00000001:2.0:1788880280.190974:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305916367 00000100:00000001:2.0:1788880280.190976:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305911353 00000100:00000001:2.0:1788880280.190977:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305916367 00000100:00000001:2.0:1788880280.190991:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305911353 00000100:00000001:2.0:1788880280.190993:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305916367 00000100:00000001:2.0:1788880280.190994:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305911353 00000100:00000001:2.0:1788880280.190996:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305916367 00000100:00000001:2.0:1788880280.190998:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880280.191000:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880280.191002:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880280.191004:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880280.191009:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880280.191011:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880280.191012:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880280.191014:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880280.191015:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880280.191019:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880280.191027:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880280.191030:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880296 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880280.191035:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880280.191037:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880280.191049:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880280.191051:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000400:00000200:3.0:1788880280.191053:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880280.191056:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880280.191057:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ee6cf6a3 00000400:00000010:3.0:1788880280.191058:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ee6cf6a3. 00000100:00000001:3.0:1788880280.191060:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880280.191061:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880280.191065:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880280.191067:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880280.191070:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880280.191071:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880280.191090:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880280.191092:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5d1e8ed. 00000400:00000200:3.0:1788880280.191094:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880280.191095:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880280.191097:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009543d7d4 00000400:00000010:3.0:1788880280.191098:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009543d7d4. 00000100:00000001:3.0:1788880280.191104:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880280.191106:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880280.191110:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880280.191111:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880280.191115:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880280.191116:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880280.192532:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880280.192543:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880280.192547:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880280.192559:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880280.192567:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880280.192578:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b03400 00000400:00000200:2.0:1788880280.192586:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x465 [1] + 224 00000800:00000001:2.0:1788880280.192591:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880280.192594:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880280.192597:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880280.192606:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880280.192610:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880280.192615:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880280.192624:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880280.192639:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880280.192646:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880280.192649:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880280.192651:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880280.192654:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880280.192659:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880280.192660:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880280.192664:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880280.192669:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880280.192669:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03480 00000400:00000001:1.0:1788880280.192671:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880280.192675:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000020fda082. 00000400:00000200:1.0:1788880280.192678:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000400:00000010:1.0:1788880280.192679:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000001:1.0:1788880280.192681:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880280.192686:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880280.192689:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x46d [1] + 224 00000800:00000001:2.0:1788880280.192698:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788880280.192701:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880280.192704:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880280.192707:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:2.0:1788880280.192708:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880280.192709:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880280.192714:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880280.192718:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000200:2.0:1788880280.192718:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880280.192721:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880280.192723:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880280.192724:0:201279:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:1.0:1788880280.192725:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880280.192729:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880280.192738:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=108 v=5 (5 0 0 0) 00000100:00000040:2.0:1788880280.192741:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880280.192743:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=108 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880280.192750:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880280.192752:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880280.192756:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880280.192759:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880280.192761:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880280.192765:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880280.192769:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880280.192771:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880280.192772:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880280.192772:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880280.192776:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880280.192778:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:1.0:1788880280.192783:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780645376:192.168.204.111@tcp:400: 00000800:00000001:2.0:1788880280.192784:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880280.192789:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880280.192791:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880280.192793:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:2.0:1788880280.192794:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880280.192796:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780645376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880280.192804:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880280.192807:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880280.192809:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880280.192810:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880280.192812:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000001:0.0:1788880280.192814:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000010:1.0:1788880280.192816:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e8a109c2. 00000400:00000200:2.0:1788880280.192816:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03500 00000400:00000001:0.0:1788880280.192817:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880280.192820:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 02000000:00000001:1.0:1788880280.192821:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880280.192825:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000200:0.0:1788880280.192826:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000020:00000040:1.0:1788880280.192827:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000010:0.0:1788880280.192828:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880280.192831:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000001:1.0:1788880280.192832:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880280.192836:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000252f4482. 00000400:00000200:2.0:1788880280.192836:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x475 [1] + 224 00000100:00000200:0.0:1788880280.192837:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880280.192841:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880280.192844:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880280.192847:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000200:0.0:1788880280.192849:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880280.192852:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000800:00000001:2.0:1788880280.192854:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880280.192855:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880280.192857:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880280.192860:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880280.192861:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880280.192863:0:201288:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788880280.192864:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880280.192866:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880280.192868:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880280.192869:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:2.0:1788880280.192869:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880280.192871:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880280.192872:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880280.192877:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880280.192877:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880280.192881:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880280.192881:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00001000:0.0:1788880280.192884:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=98 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880280.192888:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=98 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880280.192889:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880280.192891:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880280.192893:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880280.192894:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000200:2.0:1788880280.192896:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880280.192897:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880280.192901:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880280.192903:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880280.192906:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880280.192908:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880280.192913:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880280.192917:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880280.192935:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880280.192936:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880280.192937:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880280.192940:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:0.0:1788880280.192941:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880280.192947:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:0.0:1788880280.192948:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780645504:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880280.192953:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880280.192953:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880280.192955:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880280.192959:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880280.192959:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880280.192964:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780645504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880280.192965:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880280.192967:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880280.192970:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009a2f7e83. 00000100:00000001:0.0:1788880280.192973:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000200:1.0:1788880280.192975:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e1dae22 02000000:00000001:0.0:1788880280.192977:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000010:1.0:1788880280.192978:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e1dae22. 02000000:00000010:0.0:1788880280.192990:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000026ba42a. 00000100:00000001:1.0:1788880280.192992:0:201285:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:0.0:1788880280.192995:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:1.0:1788880280.192997:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880280.192999:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880280.193002:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880280.193007:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000200:1.0:1788880280.193010:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880280.193011:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 02000000:00000001:0.0:1788880280.193017:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880280.193018:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788880280.193019:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880280.193020:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:0.0:1788880280.193024:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:1.0:1788880280.193024:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880280.193026:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880280.193028:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880280.193029:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880280.193030:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:1.0:1788880280.193032:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880280.193033:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880280.193036:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=108 v=5 (5 0 0 0) 00000100:00000001:0.0:1788880280.193037:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880280.193042:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=82 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880280.193044:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880280.193045:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880280.193048:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880280.193049:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880280.193050:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880280.193053:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00080000:1.0:1788880280.193055:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880280.193056:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880280.193063:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880280.193065:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880280.193066:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880280.193067:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880280.193068:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880280.193075:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880280.193078:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880280.193080:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880280.193081:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880280.193082:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880280.193086:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780645632:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880280.193088:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880280.193089:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880280.193090:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780645632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880296 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880280.193093:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880280.193094:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880280.193095:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008e4ae4dd. 02000000:00000001:1.0:1788880280.193097:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880280.193097:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880280.193098:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880280.193099:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880280.193100:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 02000000:00000001:1.0:1788880280.193101:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880280.193102:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880280.193103:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:1.0:1788880280.193105:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880280.193105:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880280.193106:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880280.193109:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880280.193109:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880280.193110:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880280.193111:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880280.254305:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880280.254312:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880280.254317:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880280.254333:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880280.254373:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880280.254377:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880280.254381:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880281.278350:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880281.278358:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880281.278364:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880281.278365:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880281.278423:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880281.278427:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880281.278430:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880282.302323:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880282.302333:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880282.302341:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880282.302342:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880282.302420:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880282.302427:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880282.302431:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880283.326387:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880283.326396:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880283.326403:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880283.326407:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880283.326412:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880283.326415:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880283.326417:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880283.326419:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880283.326424:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880283.326425:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880283.326480:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880283.326499:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880283.326501:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880284.350392:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880284.350401:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880284.350407:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880284.350409:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880284.350465:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880284.350477:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880284.350480:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880285.310305:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305916367, at 4305916416 00000100:00000010:1.0:1788880285.310330:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000040:2.0:1788880285.310346:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880285.310347:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305916367, at 4305916416 00000100:00000001:2.0:1788880285.310357:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880285.310366:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000005a8bb1de. 00000100:00000010:3.0:1788880285.310369:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000020:00000040:2.0:1788880285.310375:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880285.310383:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880285.310388:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880285.310405:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880285.310415:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000001:2.0:1788880285.310424:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880285.310439:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880285.310466:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880285.310470:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880285.310477:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a8bb1de] to pc [ptlrpcd_00_01+1] req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880285.310504:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880285.310507:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880285.310511:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880285.310514:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000629edf55. 00000020:00000040:2.0:1788880285.310517:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880285.310520:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880285.310522:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880285.310523:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880285.310526:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880285.310531:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:2.0:1788880285.310533:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008eccda10. 00000100:00000001:1.0:1788880285.310535:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880285.310537:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880285.310540:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880285.310541:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880285.310548:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:3.0:1788880285.310560:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880285.310566:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880285.310570:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880285.310583:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000629edf55] to pc [ptlrpcd_00_02+2] req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880285.310600:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880285.310608:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880285.310616:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880285.310620:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880285.310624:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880285.310629:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880285.310630:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780645760:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880285.310634:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880285.310636:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880285.310640:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880285.310643:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880285.310645:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880285.310645:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880285.310650:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880285.310654:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cc13c26b. 00000100:00000001:3.0:1788880285.310655:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000020:00000040:2.0:1788880285.310663:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:3.0:1788880285.310667:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880285.310668:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:1.0:1788880285.310669:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880285.310674:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880285.310675:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880285.310683:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880285.310686:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880285.310690:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880285.310693:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880285.310694:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5d1e8ed. 02000000:00000010:3.0:1788880285.310697:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a5da162d. 02000000:00000001:1.0:1788880285.310698:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880285.310708:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880285.310710:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880285.310711:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880285.310717:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880285.310717:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f2f8fd80. 00000100:00000040:2.0:1788880285.310723:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:3.0:1788880285.310728:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ec90f18d. 00000100:00100000:1.0:1788880285.310739:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780645888:192.168.204.111@tcp:400: 00000100:00000200:3.0:1788880285.310739:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780645760, portal 25 00000100:00000001:2.0:1788880285.310746:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880285.310749:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880285.310750:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880285.310754:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938427392 : -126823771124224 : ffff8ca78a2c4e00) 00000100:00000001:1.0:1788880285.310757:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880285.310763:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cc13c26b] to pc [ptlrpcd_00_03+3] req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880285.310766:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880285.310767:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880301 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880285.310771:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880285.310777:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880285.310783:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000c4cba31. 00000100:00000001:2.0:1788880285.310789:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880285.310792:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880285.310793:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880285.310797:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880285.310798:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880285.310799:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:1.0:1788880285.310802:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000020fda082. 00000100:00000001:0.0:1788880285.310805:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880285.310814:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1788880285.310815:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880285.310816:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 00000100:00000040:0.0:1788880285.310820:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880285.310824:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11254) 00000400:00000010:3.0:1788880285.310827:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 00000100:00000200:1.0:1788880285.310828:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780645888, portal 10 00000100:00000001:1.0:1788880285.310838:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880285.310841:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880285.310842:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938432768 : -126823771118848 : ffff8ca78a2c6300) 00000100:00000200:3.0:1788880285.310842:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780645760, offset 0 00000100:00000001:0.0:1788880285.310849:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880285.310858:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:1.0:1788880285.310864:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880285.310866:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880285.310867:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880285.310878:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880285.310898:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00100000:0.0:1788880285.310899:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780646016:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880285.310903:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:1.0:1788880285.310908:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880285.310912:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880285.310919:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880285.310922:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880285.310924:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000065ad0e62. 00000400:00000200:3.0:1788880285.310926:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788880285.310931:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:1.0:1788880285.310934:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780645888, offset 0 02000000:00000010:0.0:1788880285.310939:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a4a3549f. 00000400:00000200:3.0:1788880285.310940:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880285.310946:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880285.310952:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880285.310954:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000400:00000200:3.0:1788880285.310954:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:0.0:1788880285.310966:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000400:00000200:3.0:1788880285.310969:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:0.0:1788880285.310974:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780646016, portal 10 00000100:00000001:0.0:1788880285.310982:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788880285.310982:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [115:115:256:4294967295] 192.168.204.111@tcp LPNI seq info [115:115:8:4294967295] 00000100:00000001:0.0:1788880285.310987:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938433664 : -126823771117952 : ffff8ca78a2c6680) 00000100:00000040:0.0:1788880285.311004:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880285.311026:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880285.311032:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880285.311040:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880285.311048:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880285.311049:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ee6cf6a3. 00000400:00000200:1.0:1788880285.311054:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:0.0:1788880285.311057:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780646016, offset 0 00000800:00000010:3.0:1788880285.311058:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000400:00000200:1.0:1788880285.311062:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880285.311073:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880285.311080:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880285.311081:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788880285.311087:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880285.311092:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880285.311099:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [116:116:255:4294967295] 192.168.204.111@tcp LPNI seq info [116:116:7:4294967295] 00000800:00000200:3.0:1788880285.311106:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880285.311116:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880285.311124:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 320 niov 1 nkiov 1 00000800:00000200:1.0:1788880285.311131:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880285.311132:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880285.311141:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000400:00000200:0.0:1788880285.311142:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880285.311147:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880285.311149:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880285.311150:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:1.0:1788880285.311158:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880285.311160:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880285.311164:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305921487 00000400:00000200:0.0:1788880285.311169:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [117:117:254:4294967295] 192.168.204.111@tcp LPNI seq info [117:117:6:4294967295] 00000800:00000200:1.0:1788880285.311177:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880285.311184:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880285.311186:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305916367 00000400:00000200:0.0:1788880285.311190:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880285.311204:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880285.311205:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880285.311205:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305921487 00000800:00000010:0.0:1788880285.311216:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 00000100:00000040:1.0:1788880285.311217:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 877606 usecs (878 jiffies) 00000100:00000010:3.0:1788880285.311220:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000007dfb4281. 00000100:00000001:1.0:1788880285.311226:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305917294 00000800:00000200:0.0:1788880285.311228:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880285.311228:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880285.311231:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880285.311259:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305921487 00000100:00000001:1.0:1788880285.311265:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305917295 00000800:00000200:0.0:1788880285.311280:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880285.311283:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880285.311284:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305921488 00000100:00000001:1.0:1788880285.311285:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305921487 00000100:00000001:0.0:1788880285.311287:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880285.311291:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000100:00000040:0.0:1788880285.311296:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 54675 usecs (55 jiffies) 00000100:00000001:1.0:1788880285.311301:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305917295 00000100:00000001:0.0:1788880285.311304:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305916472 00000100:00000001:3.0:1788880285.311307:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305916367 00000400:00000200:2.0:1788880285.311308:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880285.311310:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305921487 00000100:00000001:0.0:1788880285.311318:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305916472 00000100:00000001:3.0:1788880285.311324:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305921488 00000100:00000001:0.0:1788880285.311325:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305921487 00000100:00000001:1.0:1788880285.311325:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305921488 00000400:00000200:2.0:1788880285.311326:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880285.311337:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305917295 00000100:00000001:0.0:1788880285.311340:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305916472 00000100:00000010:3.0:1788880285.311346:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002aba003e. 00000100:00000001:0.0:1788880285.311347:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305921488 00000400:00000200:2.0:1788880285.311348:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000100:00000001:0.0:1788880285.311355:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305916472 00000400:00000010:2.0:1788880285.311357:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:3.0:1788880285.311357:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880285.311366:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880285.311375:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305921488 00000100:00000200:2.0:1788880285.311375:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880301 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880285.311376:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880285.311383:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305921488 00000100:00000001:1.0:1788880285.311383:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880285.311389:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880285.311392:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880285.311392:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880285.311404:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880285.311405:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880285.311406:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880285.311411:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880285.311412:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880285.311415:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880285.311418:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880285.311420:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880285.311423:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880301 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880285.311430:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880285.311433:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880285.311435:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880285.311438:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880285.311438:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880285.311441:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880285.311448:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880285.311449:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880285.311450:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880285.311450:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880285.311457:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880285.311461:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880285.311462:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880285.311463:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880285.311466:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880285.311469:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880285.311471:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880285.311472:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880285.311474:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880285.311480:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880285.311484:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000001:3.0:1788880285.311484:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880285.311493:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880285.311500:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880285.311503:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000065ad0e62 00000400:00000010:2.0:1788880285.311505:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000065ad0e62. 00000100:00000001:2.0:1788880285.311516:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880285.311520:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880285.311529:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880285.311532:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880285.311539:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880285.311541:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880285.311619:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880285.311622:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000400:00000200:2.0:1788880285.311627:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880285.311631:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880285.311633:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ee6cf6a3 00000400:00000010:2.0:1788880285.311635:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ee6cf6a3. 00000100:00000001:2.0:1788880285.311637:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880285.311640:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880285.311648:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880285.311650:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880285.311656:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880285.311658:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880285.313407:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880285.313453:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880285.313457:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880285.313460:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880285.313470:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880285.313486:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b03580 00000400:00000200:3.0:1788880285.313494:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x47d [1] + 224 00000800:00000001:3.0:1788880285.313501:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880285.313505:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880285.313515:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880285.313520:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880285.313528:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880285.313534:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880285.313545:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880285.313566:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880285.313576:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880285.313581:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880285.313583:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880285.313586:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880285.313590:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880285.313598:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880285.313600:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03600 00000100:00000001:0.0:1788880285.313603:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880285.313611:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x481 [1] + 224 00000800:00000001:3.0:1788880285.313624:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880285.313626:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788880285.313630:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880285.313631:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:3.0:1788880285.313634:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:0.0:1788880285.313637:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000f2f8fd80. 00000400:00000200:3.0:1788880285.313644:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880285.313647:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ec90f18d 00000400:00000010:0.0:1788880285.313651:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ec90f18d. 00000100:00000001:0.0:1788880285.313656:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880285.313657:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880285.313665:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880285.313667:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880285.313708:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880285.313716:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880285.313725:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880285.313727:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880285.313734:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880285.313739:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880285.313743:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880285.313746:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880285.313751:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880285.313754:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:0.0:1788880285.313759:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=113 v=5 (5 0 0 0) 00000800:00000001:3.0:1788880285.313762:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880285.313769:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=113 v=5 (1 0 0 0) 00000800:00000001:3.0:1788880285.313770:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880285.313781:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:3.0:1788880285.313782:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880285.313784:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880285.313789:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880285.313790:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880285.313795:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880285.313800:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880285.313802:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880285.313810:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03680 00000100:00000040:0.0:1788880285.313813:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880285.313830:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880285.313831:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x48d [1] + 224 00000100:00000001:0.0:1788880285.313838:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880285.313844:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880285.313846:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1788880285.313851:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880285.313858:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880285.313861:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880285.313878:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00100000:0.0:1788880285.313893:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780645760:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880285.313899:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880285.313902:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880285.313905:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:0.0:1788880285.313906:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000005a8bb1de x1875776780645760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880285.313914:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:3.0:1788880285.313915:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880285.313919:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880285.313924:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a5da162d. 02000000:00000001:0.0:1788880285.313934:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880285.313939:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:3.0:1788880285.313939:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1788880285.313944:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880285.313951:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880285.313958:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 02000000:00000001:0.0:1788880285.313966:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880285.313969:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880285.313969:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000010:0.0:1788880285.313975:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000005a8bb1de. 00000100:00000001:0.0:1788880285.313982:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000800:00000001:3.0:1788880285.313984:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880285.313985:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880285.313991:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880285.313994:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880285.313999:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880285.314001:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880285.314003:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880285.314004:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880285.314021:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880285.314024:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880285.314025:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880285.314033:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880285.314033:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880285.314039:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880285.314041:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000400:00000200:0.0:1788880285.314045:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000010:0.0:1788880285.314048:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880285.314052:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:3.0:1788880285.314054:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:0.0:1788880285.314060:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:3.0:1788880285.314060:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880285.314066:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000020fda082. 00000400:00000200:3.0:1788880285.314075:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000400:00000010:3.0:1788880285.314080:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000200:0.0:1788880285.314081:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880285.314087:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880285.314098:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880285.314109:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880285.314114:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880285.314121:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880285.314125:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880285.314129:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880285.314135:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880285.314137:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880285.314141:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880285.314151:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=113 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880285.314165:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=87 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880285.314177:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880285.314181:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880285.314188:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880285.314198:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880285.314219:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:0.0:1788880285.314220:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880285.314228:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880285.314228:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880285.314230:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880285.314250:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880285.314252:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880285.314257:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880285.314266:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880285.314270:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880285.314272:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880285.314277:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:0.0:1788880285.314282:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880285.314285:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880285.314289:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880285.314293:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880285.314301:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880285.314304:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780646016:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880285.314312:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788880285.314312:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=103 v=5 (5 0 0 0) 00000100:00000001:0.0:1788880285.314316:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00001000:3.0:1788880285.314321:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=103 v=5 (1 0 0 0) 00000100:00000040:0.0:1788880285.314325:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cc13c26b x1875776780646016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880285.314328:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880285.314335:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880285.314337:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880285.314341:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880285.314345:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880285.314347:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880285.314349:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880285.314353:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880285.314367:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:3.0:1788880285.314367:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880285.314374:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880285.314379:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a4a3549f. 02000000:00000001:0.0:1788880285.314389:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880285.314395:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880285.314396:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000020:00000040:0.0:1788880285.314397:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880285.314404:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880285.314407:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5d1e8ed. 00000100:00000001:3.0:1788880285.314407:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880285.314413:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880285.314415:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880285.314419:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880285.314422:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cc13c26b. 00000100:00000040:3.0:1788880285.314422:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880285.314430:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880285.314434:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880285.314440:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880285.314448:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880285.314453:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880285.314460:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880285.314465:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880285.314476:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780645888:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880285.314483:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880285.314486:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880285.314489:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000629edf55 x1875776780645888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880301 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880285.314495:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880285.314498:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880285.314501:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000c4cba31. 02000000:00000001:3.0:1788880285.314505:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880285.314508:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880285.314510:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880285.314512:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880285.314514:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008eccda10. 02000000:00000001:3.0:1788880285.314518:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880285.314519:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880285.314520:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000629edf55. 00000100:00000001:3.0:1788880285.314524:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880285.314525:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880285.314527:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880285.314533:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880285.314535:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880285.314537:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880285.314539:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880285.374285:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880285.374293:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880285.374298:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880285.374302:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880285.374304:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880285.374307:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880285.374309:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880285.374311:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880285.374313:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880285.374315:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880285.374369:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880285.374373:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880285.374375:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880286.398408:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880286.398418:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880286.398458:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880286.398459:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880286.398612:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880286.398619:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880286.398622:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880287.422338:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880287.422346:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880287.422353:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880287.422355:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880287.422411:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880287.422415:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880287.422418:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880288.446403:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880288.446414:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880288.446422:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880288.446423:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880288.446492:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880288.446497:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880288.446501:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880289.470306:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880289.470313:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880289.470319:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880289.470321:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880289.470376:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880289.470379:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880289.470381:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880290.430348:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305921488, at 4305921536 00000100:00000040:3.0:1788880290.430348:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305921488, at 4305921536 00000100:00000040:0.0:1788880290.430349:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305921487, at 4305921536 00000100:00000040:1.0:1788880290.430356:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305921487, at 4305921536 00000100:00000010:2.0:1788880290.430371:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000010:3.0:1788880290.430373:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002aba003e. 00000100:00000010:0.0:1788880290.430377:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000010:1.0:1788880290.430381:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000007dfb4281. 00000100:00000040:2.0:1788880290.430405:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880290.430410:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880290.430418:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000884e74b8. 00000020:00000040:2.0:1788880290.430422:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880290.430425:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880290.430428:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880290.430432:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880290.430439:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f9812869. 00000100:00000001:2.0:1788880290.430447:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880290.430456:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880290.430464:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880290.430466:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880290.430470:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000884e74b8] to pc [ptlrpcd_00_00+0] req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880290.430494:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880290.430496:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880290.430500:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880290.430503:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000020:00000040:2.0:1788880290.430506:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880290.430509:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880290.430512:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880290.430515:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880290.430517:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003139f6ba. 00000100:00000001:0.0:1788880290.430519:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880290.430519:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880290.430522:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880290.430527:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880290.430529:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880290.430530:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880290.430532:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880290.430547:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880290.430570:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880290.430572:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880290.430573:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880290.430575:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880290.430580:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880290.430581:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_01+1] req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880290.430584:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880290.430590:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880290.430597:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780646144:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880290.430602:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880290.430608:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880290.430610:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880290.430614:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880290.430619:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880290.430619:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880290.430623:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 02000000:00000001:0.0:1788880290.430624:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000020:00000040:2.0:1788880290.430627:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880290.430630:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000010:0.0:1788880290.430631:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e84cf675. 02000000:00000001:2.0:1788880290.430632:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880290.430635:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880290.430637:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880290.430639:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880290.430640:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006aca3ff7. 00000400:00000010:0.0:1788880290.430647:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000100:00000001:1.0:1788880290.430649:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880290.430650:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880290.430655:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000001:1.0:1788880290.430655:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880290.430661:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000200:0.0:1788880290.430667:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780646144, portal 25 00000100:00000040:1.0:1788880290.430672:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880290.430675:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880290.430679:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938423808 : -126823771127808 : ffff8ca78a2c4000) 00000100:00000001:2.0:1788880290.430687:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880290.430693:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880290.430698:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880306 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880290.430711:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_02+2] req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880290.430741:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880290.430742:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880290.430747:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880290.430755:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880290.430756:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880290.430757:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880290.430759:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000100:00000040:2.0:1788880290.430765:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880290.430766:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880290.430767:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780646144, offset 0 02000000:00000001:1.0:1788880290.430775:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880290.430783:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788880290.430795:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780646272:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880290.430796:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880290.430796:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880290.430805:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880290.430810:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11259) 00000100:00000001:1.0:1788880290.430813:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880290.430813:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:0.0:1788880290.430819:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788880290.430821:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880290.430825:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880290.430827:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880290.430827:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880290.430836:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880290.430838:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880290.430840:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880290.430844:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000010:1.0:1788880290.430847:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008545eb31. 00000400:00000200:0.0:1788880290.430854:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:1.0:1788880290.430857:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880290.430859:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:0.0:1788880290.430863:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:1.0:1788880290.430868:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000400:00000200:0.0:1788880290.430870:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [118:118:256:4294967295] 192.168.204.111@tcp LPNI seq info [118:118:8:4294967295] 00000100:00000001:3.0:1788880290.430871:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880290.430879:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:1.0:1788880290.430881:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 02000000:00000001:3.0:1788880290.430882:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880290.430887:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880290.430891:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00100000:3.0:1788880290.430896:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780646400:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880290.430905:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780646272, portal 10 00000100:00000001:3.0:1788880290.430905:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880290.430912:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:3.0:1788880290.430912:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000800:00000200:0.0:1788880290.430914:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880290.430918:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 02000000:00000001:3.0:1788880290.430918:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880290.430922:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 02000000:00000001:3.0:1788880290.430929:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880290.430934:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000089aec2fa. 00000100:00000040:1.0:1788880290.430940:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880290.430941:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:3.0:1788880290.430941:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880290.430952:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f2f8fd80. 00000400:00000010:3.0:1788880290.430965:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005b101ac6. 00000800:00000200:0.0:1788880290.430967:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880290.430976:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000100:00000200:3.0:1788880290.430977:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780646400, portal 10 00000100:00000001:1.0:1788880290.430979:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880290.430987:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880290.430991:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000100:00000040:1.0:1788880290.430993:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880290.430998:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880290.431003:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000066e8f89d. 00000100:00000001:0.0:1788880290.431007:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305926607 00000100:00000040:3.0:1788880290.431010:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880290.431012:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780646272, offset 0 00000100:00000010:0.0:1788880290.431014:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000455e6647. 00000100:00000001:0.0:1788880290.431027:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880290.431031:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880290.431032:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880290.431037:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305926607 00000100:00000040:3.0:1788880290.431040:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880290.431043:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000506ae1c0. 00000400:00000010:3.0:1788880290.431045:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000082116f14. 00000100:00000001:0.0:1788880290.431048:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:1.0:1788880290.431051:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:3.0:1788880290.431053:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780646400, offset 0 00000100:00000001:0.0:1788880290.431055:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305926607 00000400:00000200:1.0:1788880290.431062:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:0.0:1788880290.431063:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001d590ced. 00000400:00000200:3.0:1788880290.431071:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880290.431072:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:1.0:1788880290.431076:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880290.431081:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305926607 00000400:00000200:1.0:1788880290.431083:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880290.431093:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788880290.431093:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:0.0:1788880290.431095:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009f22a9a0. 00000800:00000010:2.0:1788880290.431099:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000100:00000001:0.0:1788880290.431103:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788880290.431105:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [119:119:255:4294967295] 192.168.204.111@tcp LPNI seq info [119:119:7:4294967295] 00000400:00000200:2.0:1788880290.431110:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880290.431111:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880290.431116:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880290.431124:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880290.431131:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880290.431132:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880290.431134:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880290.431145:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880290.431150:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:1.0:1788880290.431159:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880290.431159:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880290.431160:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880290.431167:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000010:1.0:1788880290.431170:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:0.0:1788880290.431172:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880290.431174:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880290.431184:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880290.431190:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880290.431193:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880290.431201:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:1.0:1788880290.431206:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880290.431210:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880290.431212:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880290.431217:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880290.431221:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 848506 usecs (849 jiffies) 00000100:00000001:1.0:1788880290.431224:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305922385 00000100:00000001:1.0:1788880290.431226:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305926607 00000100:00000001:1.0:1788880290.431229:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305922385 00000100:00000001:1.0:1788880290.431231:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305926607 00000100:00000001:1.0:1788880290.431263:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305922386 00000100:00000001:1.0:1788880290.431270:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305926607 00000100:00000001:1.0:1788880290.431272:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305922386 00000100:00000001:1.0:1788880290.431274:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305926607 00000100:00000001:1.0:1788880290.431277:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880290.431279:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880290.431282:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880290.431284:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880290.431291:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880290.431293:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880290.431295:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880290.431298:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880290.431299:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880290.431301:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880290.431312:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [120:120:254:4294967295] 192.168.204.111@tcp LPNI seq info [120:120:6:4294967295] 00000400:00000200:3.0:1788880290.431323:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880290.431329:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880290.431331:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000800:00000010:3.0:1788880290.431334:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000400:00000010:2.0:1788880290.431335:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00000001:2.0:1788880290.431341:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880290.431350:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880306 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880290.431351:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880290.431360:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880290.431366:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880290.431370:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880290.431371:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880290.431376:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880306 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880290.431377:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 52732 usecs (53 jiffies) 00000100:00000001:3.0:1788880290.431382:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305921590 00000100:00000001:3.0:1788880290.431387:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305926607 00000100:00000001:3.0:1788880290.431390:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305921590 00000100:00000001:2.0:1788880290.431393:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880290.431394:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305926607 00000100:00000001:2.0:1788880290.431399:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880290.431399:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305921590 00000100:00000001:3.0:1788880290.431403:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305926607 00000100:00000001:3.0:1788880290.431404:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305921590 00000100:00000001:3.0:1788880290.431406:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305926607 00000100:00000001:3.0:1788880290.431408:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880290.431410:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880290.431413:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880290.431414:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880290.431418:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880290.431420:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:2.0:1788880290.431421:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880290.431423:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:2.0:1788880290.431425:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000100:00000001:3.0:1788880290.431426:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880290.431428:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880290.431429:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880290.431433:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880290.431434:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880290.431439:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000066e8f89d 00000400:00000010:2.0:1788880290.431440:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000066e8f89d. 00000100:00000001:2.0:1788880290.431442:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880290.431445:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880290.431450:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880290.431452:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880290.431456:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880290.431457:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880290.431480:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880290.431482:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000400:00000200:2.0:1788880290.431485:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880290.431487:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880290.431488:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000082116f14 00000400:00000010:2.0:1788880290.431489:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000082116f14. 00000100:00000001:2.0:1788880290.431490:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880290.431492:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880290.431501:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880290.431502:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880290.431506:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880290.431507:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880290.432513:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880290.432523:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880290.432527:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880290.432529:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880290.432535:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880290.432545:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b03700 00000400:00000200:2.0:1788880290.432551:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x495 [1] + 224 00000800:00000001:2.0:1788880290.432557:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880290.432562:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880290.432565:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880290.432568:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880290.432572:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880290.432576:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880290.432584:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880290.432601:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880290.432607:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880290.432622:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880290.432625:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880290.432628:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880290.432631:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880290.432632:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880290.432636:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880290.432641:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03780 00000400:00000001:0.0:1788880290.432643:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880290.432646:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880290.432648:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000400:00000200:0.0:1788880290.432652:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000010:0.0:1788880290.432653:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880290.432656:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880290.432659:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x49d [1] + 224 00000100:00000200:0.0:1788880290.432662:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880290.432666:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880290.432675:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880290.432678:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880290.432682:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880290.432688:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:2.0:1788880290.432689:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880290.432693:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880290.432697:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880290.432699:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880290.432705:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880290.432709:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:2.0:1788880290.432712:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880290.432713:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880290.432717:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880290.432722:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880290.432731:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=118 v=5 (5 0 0 0) 00000100:00000001:2.0:1788880290.432735:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:0.0:1788880290.432736:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=118 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880290.432741:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:2.0:1788880290.432741:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880290.432743:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880290.432746:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880290.432747:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880290.432751:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880290.432755:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880290.432761:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880290.432768:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880290.432771:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880290.432773:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880290.432777:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880290.432778:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880290.432781:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880290.432796:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880290.432799:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00100000:0.0:1788880290.432802:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780646144:192.168.204.111@tcp:400: 00000400:00000010:1.0:1788880290.432803:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000400:00000200:1.0:1788880290.432810:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000100:00000001:0.0:1788880290.432812:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880290.432813:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:0.0:1788880290.432817:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880290.432819:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:0.0:1788880290.432823:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000884e74b8 x1875776780646144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880290.432823:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788880290.432827:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880290.432831:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880290.432837:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880290.432840:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880290.432850:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000200:3.0:1788880290.432854:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 02000000:00000001:0.0:1788880290.432856:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000200:1.0:1788880290.432859:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880290.432861:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e84cf675. 02000000:00000001:0.0:1788880290.432867:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000200:3.0:1788880290.432869:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03800 00000020:00000001:0.0:1788880290.432871:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880290.432875:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880290.432879:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:0.0:1788880290.432880:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000200:3.0:1788880290.432881:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x4a1 [1] + 224 00000100:00000001:1.0:1788880290.432883:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788880290.432885:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f9812869. 00000100:00000001:1.0:1788880290.432888:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880290.432891:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880290.432892:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880290.432895:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880290.432898:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880290.432901:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:0.0:1788880290.432902:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:1.0:1788880290.432902:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880290.432906:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000884e74b8. 00000100:00001000:1.0:1788880290.432910:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=108 v=5 (5 0 0 0) 00000800:00000001:3.0:1788880290.432911:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880290.432912:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880290.432915:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880290.432917:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00001000:1.0:1788880290.432918:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=108 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880290.432919:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880290.432925:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880290.432927:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880290.432930:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880290.432931:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880290.432932:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880290.432937:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880290.432937:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880290.432937:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880290.432941:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880290.432942:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880290.432947:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880290.432953:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880290.432956:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000040:3.0:1788880290.432957:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880290.432960:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880290.432963:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880290.432971:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880290.432977:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880290.432978:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880290.432979:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880290.432982:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880290.432983:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880290.432990:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:1.0:1788880290.432991:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780646272:192.168.204.111@tcp:400: 00000800:00000001:3.0:1788880290.432996:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880290.432997:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880290.433001:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880290.433004:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880290.433004:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780646272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880290.433008:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880290.433013:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880290.433015:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000001:0.0:1788880290.433016:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000010:1.0:1788880290.433017:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008545eb31. 00000400:00000001:0.0:1788880290.433019:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:1.0:1788880290.433021:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000010:0.0:1788880290.433023:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000f2f8fd80. 00000020:00000001:1.0:1788880290.433023:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880290.433025:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880290.433029:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000200:0.0:1788880290.433030:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005b101ac6 02000000:00000010:1.0:1788880290.433031:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003139f6ba. 02000000:00000001:1.0:1788880290.433035:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880290.433037:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880290.433037:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000100:00000001:1.0:1788880290.433040:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880290.433040:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880290.433044:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005b101ac6. 00000100:00000001:1.0:1788880290.433046:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880290.433051:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880290.433052:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880290.433054:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880290.433055:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880290.433056:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880290.433063:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880290.433072:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880290.433079:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880290.433080:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880290.433082:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880290.433083:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880290.433085:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880290.433086:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880290.433088:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880290.433091:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=118 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880290.433094:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=92 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880290.433096:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880290.433097:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880290.433098:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880290.433101:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880290.433116:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880290.433124:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880290.433125:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880290.433127:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880290.433128:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880290.433131:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880290.433138:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880290.433140:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880290.433141:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880290.433144:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880290.433151:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780646400:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880290.433155:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880290.433157:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880290.433159:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780646400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880306 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880290.433165:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880290.433167:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880290.433169:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000089aec2fa. 02000000:00000001:0.0:1788880290.433171:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880290.433173:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880290.433174:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880290.433176:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880290.433177:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006aca3ff7. 02000000:00000001:0.0:1788880290.433179:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880290.433181:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880290.433182:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 00000100:00000001:0.0:1788880290.433184:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880290.433185:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880290.433186:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880290.433190:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880290.433191:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880290.433193:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880290.433200:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880290.494346:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880290.494360:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880290.494366:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880290.494368:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880290.494430:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880290.494434:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880290.494436:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880291.518379:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880291.518386:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880291.518392:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880291.518392:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880291.518480:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880291.518488:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880291.518492:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880292.542382:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880292.542391:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880292.542399:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880292.542400:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880292.542458:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880292.542462:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880292.542466:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880293.566351:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880293.566360:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880293.566362:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880293.566365:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880293.566370:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880293.566373:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880293.566376:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880293.566378:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880293.566382:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880293.566383:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880293.566455:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880293.566458:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880293.566461:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880294.590328:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880294.590334:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880294.590340:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880294.590342:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880294.590403:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880294.590407:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880294.590410:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880295.550325:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305926607, at 4305926656 00000100:00000040:2.0:1788880295.550336:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305926607, at 4305926656 00000100:00000010:1.0:1788880295.550339:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000506ae1c0. 00000100:00000040:0.0:1788880295.550340:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305926607, at 4305926656 00000100:00000010:2.0:1788880295.550348:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001d590ced. 00000100:00000010:0.0:1788880295.550350:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000455e6647. 00000100:00000040:1.0:1788880295.550367:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880295.550371:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880295.550375:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c09231ec. 00000100:00000040:3.0:1788880295.550377:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305926607, at 4305926656 00000020:00000040:1.0:1788880295.550379:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:3.0:1788880295.550383:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009f22a9a0. 00000100:00000001:1.0:1788880295.550393:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880295.550396:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880295.550399:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880295.550404:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:1.0:1788880295.550410:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880295.550417:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880295.550425:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880295.550427:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880295.550431:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c09231ec] to pc [ptlrpcd_00_03+3] req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880295.550453:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880295.550455:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880295.550457:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880295.550459:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000094565713. 00000020:00000040:1.0:1788880295.550460:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880295.550461:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880295.550462:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880295.550463:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880295.550466:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:1.0:1788880295.550467:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880295.550469:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880295.550473:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880295.550474:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880295.550484:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880295.550484:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000094565713] to pc [ptlrpcd_00_00+0] req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880295.550489:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:0.0:1788880295.550493:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.550502:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880295.550504:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.550507:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880295.550507:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880295.550509:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880295.550510:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880295.550512:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880295.550513:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880295.550513:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880295.550519:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000040:0.0:1788880295.550520:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880295.550526:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 00000100:00000001:0.0:1788880295.550528:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000020:00000040:1.0:1788880295.550529:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880295.550530:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880295.550533:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880295.550535:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880295.550535:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880295.550539:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880295.550539:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880295.550544:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880295.550545:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00100000:0.0:1788880295.550548:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780646528:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880295.550549:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880295.550555:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880295.550557:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880295.550560:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880295.550564:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880295.550564:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788880295.550567:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880295.550567:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880295.550572:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f77f54b9. 00000100:00000040:1.0:1788880295.550573:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_00_01+1] req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880295.550575:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880295.550582:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880295.550583:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000100:00000040:1.0:1788880295.550584:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880295.550588:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:0.0:1788880295.550589:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000040:1.0:1788880295.550591:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11264) 00000100:00000200:0.0:1788880295.550595:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780646528, portal 25 00000100:00000001:0.0:1788880295.550598:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880295.550599:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938056832 : -126823771494784 : ffff8ca78a26a680) 00000100:00000001:1.0:1788880295.550600:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880295.550601:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880295.550604:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880295.550605:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880295.550606:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880311 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880295.550607:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880295.550608:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880295.550615:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880295.550619:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880295.550621:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880295.550626:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880295.550637:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:0.0:1788880295.550638:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000dc289a80. 00000100:00000001:1.0:1788880295.550639:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880295.550645:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:0.0:1788880295.550646:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780646528, offset 0 02000000:00000001:1.0:1788880295.550647:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880295.550650:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:1.0:1788880295.550651:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880295.550654:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880295.550658:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780646784:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880295.550659:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880295.550665:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880295.550666:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880295.550669:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880295.550670:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880295.550673:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880295.550675:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880295.550675:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880295.550683:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780646656:192.168.204.111@tcp:400: 02000000:00000001:1.0:1788880295.550684:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880295.550694:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:1.0:1788880295.550700:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b936126e. 00000100:00000001:3.0:1788880295.550700:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880295.550704:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788880295.550705:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880295.550705:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880295.550707:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880295.550709:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880295.550711:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:1.0:1788880295.550712:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000400:00000200:0.0:1788880295.550714:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000010:3.0:1788880295.550715:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a21494dd. 02000000:00000001:3.0:1788880295.550719:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880295.550720:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000400:00000200:0.0:1788880295.550722:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [121:121:256:4294967295] 192.168.204.111@tcp LPNI seq info [121:121:8:4294967295] 00000400:00000010:3.0:1788880295.550723:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000de273b16. 00000100:00000200:1.0:1788880295.550726:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780646784, portal 10 00000100:00000001:1.0:1788880295.550730:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880295.550733:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 00000400:00000010:3.0:1788880295.550734:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f73021f1. 00000400:00000200:0.0:1788880295.550738:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:3.0:1788880295.550740:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780646656, portal 10 00000100:00000001:3.0:1788880295.550743:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880295.550744:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055040 : -126823771496576 : ffff8ca78a269f80) 00000100:00000040:1.0:1788880295.550747:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880295.550748:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880295.550754:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 00000100:00000040:3.0:1788880295.550754:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880295.550765:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880295.550769:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880295.550774:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880295.550774:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880295.550779:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:1.0:1788880295.550780:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e249bdf9. 00000100:00000040:3.0:1788880295.550780:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880295.550784:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000100:00000200:1.0:1788880295.550786:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780646784, offset 0 00000400:00000010:3.0:1788880295.550786:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005f2b805f. 00000100:00000200:3.0:1788880295.550791:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780646656, offset 0 00000400:00000200:1.0:1788880295.550795:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880295.550796:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880295.550801:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880295.550802:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305931727 00000400:00000200:1.0:1788880295.550805:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880295.550808:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:0.0:1788880295.550809:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000455e6647. 00000400:00000200:1.0:1788880295.550811:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880295.550815:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880295.550816:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:1.0:1788880295.550820:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [122:122:255:4294967295] 192.168.204.111@tcp LPNI seq info [122:122:7:4294967295] 00000100:00000001:0.0:1788880295.550824:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305931727 00000100:00000010:0.0:1788880295.550830:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000751d0eed. 00000400:00000200:1.0:1788880295.550830:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880295.550836:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:1.0:1788880295.550837:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880295.550842:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880295.550844:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305931727 00000800:00000010:1.0:1788880295.550844:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000010:0.0:1788880295.550850:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e602733. 00000400:00000200:3.0:1788880295.550851:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880295.550853:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880295.550858:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:3.0:1788880295.550859:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:1.0:1788880295.550863:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880295.550865:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305931727 00000400:00000200:3.0:1788880295.550867:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880295.550868:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000010:0.0:1788880295.550870:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014321469. 00000400:00000200:3.0:1788880295.550872:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880295.550874:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880295.550876:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788880295.550878:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788880295.550879:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 50888 usecs (51 jiffies) 00000100:00000001:0.0:1788880295.550880:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880295.550881:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880295.550883:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305926707 00000400:00000200:3.0:1788880295.550886:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [123:123:254:4294967295] 192.168.204.111@tcp LPNI seq info [123:123:6:4294967295] 00000100:00000001:0.0:1788880295.550887:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880295.550888:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000100:00000001:1.0:1788880295.550889:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305931727 00000100:00000001:0.0:1788880295.550895:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880295.550896:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880295.550898:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305926707 00000100:00000001:0.0:1788880295.550901:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880295.550903:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880295.550904:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305931727 00000100:00000001:1.0:1788880295.550910:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305926707 00000400:00000200:2.0:1788880295.550911:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880295.550913:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305931727 00000800:00000200:3.0:1788880295.550917:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880295.550918:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305926707 00000100:00000001:1.0:1788880295.550923:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305931727 00000800:00000010:3.0:1788880295.550924:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000100:00000001:1.0:1788880295.550925:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880295.550926:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000dc289a80 00000100:00000001:1.0:1788880295.550930:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880295.550930:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:2.0:1788880295.550931:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000dc289a80. 00000100:00000001:1.0:1788880295.550933:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880295.550937:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880295.550939:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880295.550942:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880295.550943:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880295.550947:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880295.550947:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880311 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880295.550949:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880295.550950:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880295.550954:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880295.550965:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880295.550969:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880311 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880295.550973:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880295.550978:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880295.550983:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880295.550985:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880295.550986:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 821227 usecs (822 jiffies) 00000100:00000001:2.0:1788880295.550989:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880295.550990:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305927478 00000100:00000001:3.0:1788880295.550994:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305931727 00000100:00000001:3.0:1788880295.550995:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305927478 00000100:00000001:3.0:1788880295.550997:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305931727 00000100:00000001:3.0:1788880295.551004:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305927478 00000100:00000001:3.0:1788880295.551006:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305931727 00000800:00000200:2.0:1788880295.551007:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880295.551008:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305927478 00000100:00000001:3.0:1788880295.551012:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305931727 00000800:00000010:2.0:1788880295.551013:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000100:00000001:3.0:1788880295.551014:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880295.551018:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880295.551018:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880295.551020:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880295.551023:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880295.551027:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880295.551028:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880295.551030:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e249bdf9 00000100:00000001:3.0:1788880295.551031:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000010:2.0:1788880295.551034:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e249bdf9. 00000100:00000001:3.0:1788880295.551036:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880295.551039:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880295.551040:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880295.551042:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880295.551046:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880295.551047:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880295.551058:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880295.551061:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880295.551077:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880295.551079:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880295.551106:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880295.551108:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000400:00000200:2.0:1788880295.551112:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880295.551115:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880295.551118:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005f2b805f 00000400:00000010:2.0:1788880295.551119:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005f2b805f. 00000100:00000001:2.0:1788880295.551122:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880295.551124:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880295.551149:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880295.551151:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880295.551158:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880295.551159:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880295.552159:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880295.552169:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880295.552172:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880295.552174:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880295.552180:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880295.552190:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b03880 00000400:00000200:2.0:1788880295.552198:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x4ad [1] + 224 00000800:00000001:2.0:1788880295.552203:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880295.552208:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880295.552210:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880295.552213:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880295.552217:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880295.552221:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880295.552229:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880295.552263:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880295.552270:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880295.552274:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880295.552275:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880295.552277:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880295.552281:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880295.552288:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03980 00000400:00000200:2.0:1788880295.552292:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x4b5 [1] + 224 00000800:00000001:2.0:1788880295.552296:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880295.552300:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880295.552303:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880295.552308:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880295.552309:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880295.552312:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:0.0:1788880295.552313:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880295.552317:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000400:00000200:2.0:1788880295.552321:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880295.552322:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000010:0.0:1788880295.552325:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880295.552327:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880295.552328:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880295.552331:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880295.552336:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880295.552348:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880295.552357:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:2.0:1788880295.552358:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880295.552359:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.552363:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880295.552366:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880295.552369:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880295.552371:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880295.552374:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880295.552378:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=123 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880295.552381:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=123 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880295.552381:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880295.552383:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880295.552385:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880295.552387:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788880295.552388:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:0.0:1788880295.552390:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880295.552392:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.552393:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880295.552401:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880295.552405:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880295.552406:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880295.552407:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880295.552409:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880295.552410:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880295.552413:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:0.0:1788880295.552414:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880295.552416:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880295.552418:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000400:00000200:1.0:1788880295.552421:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:1.0:1788880295.552423:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:1.0:1788880295.552428:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880295.552433:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880295.552443:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880295.552449:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:0.0:1788880295.552453:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780646528:192.168.204.111@tcp:400: 00000800:00000001:3.0:1788880295.552455:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880295.552459:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880295.552462:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880295.552463:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.552465:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880295.552466:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.552472:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880295.552472:0:201286:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:3.0:1788880295.552472:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 02000000:00000001:1.0:1788880295.552477:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:0.0:1788880295.552479:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c09231ec x1875776780646528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880295.552487:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880295.552488:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880295.552490:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880295.552492:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:1.0:1788880295.552493:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880295.552496:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f77f54b9. 00000100:00001000:1.0:1788880295.552500:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=123 v=5 (5 0 0 0) 02000000:00000001:0.0:1788880295.552501:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880295.552504:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00001000:1.0:1788880295.552506:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=97 v=5 (1 0 0 0) 00000020:00000040:0.0:1788880295.552507:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880295.552512:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:3.0:1788880295.552512:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03900 00000020:00000001:0.0:1788880295.552514:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880295.552516:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880295.552518:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 00010000:00000001:1.0:1788880295.552521:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:3.0:1788880295.552521:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x4b9 [1] + 224 02000000:00000001:0.0:1788880295.552523:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880295.552526:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000800:00000001:3.0:1788880295.552528:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000010:0.0:1788880295.552529:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c09231ec. 00010000:00080000:1.0:1788880295.552529:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880295.552534:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000800:00000001:3.0:1788880295.552534:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.552537:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880295.552540:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880295.552541:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.552548:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:3.0:1788880295.552548:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880295.552551:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880295.552551:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880295.552553:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880295.552557:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880295.552558:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000200:3.0:1788880295.552560:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880295.552561:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880295.552562:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880295.552568:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880295.552572:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880295.552573:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880295.552579:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880295.552589:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880295.552591:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880295.552592:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880295.552595:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880295.552596:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:1.0:1788880295.552604:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780646784:192.168.204.111@tcp:400: 00000800:00000001:3.0:1788880295.552606:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880295.552608:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880295.552610:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880295.552611:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880295.552615:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780646784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880295.552620:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880295.552620:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880295.552622:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880295.552624:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:1.0:1788880295.552624:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b936126e. 02000000:00000001:1.0:1788880295.552629:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000001:0.0:1788880295.552630:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880295.552632:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880295.552634:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000de273b16. 00000020:00000001:1.0:1788880295.552636:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880295.552638:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000200:0.0:1788880295.552640:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f73021f1 00000020:00000001:1.0:1788880295.552641:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000010:0.0:1788880295.552645:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f73021f1. 02000000:00000010:1.0:1788880295.552645:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 00000100:00000001:0.0:1788880295.552649:0:201285:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:1.0:1788880295.552651:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880295.552653:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880295.552655:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000200:0.0:1788880295.552656:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880295.552660:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880295.552663:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880295.552667:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880295.552672:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880295.552673:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880295.552676:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880295.552678:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880295.552681:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880295.552700:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880295.552703:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.552705:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880295.552707:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880295.552708:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880295.552710:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880295.552712:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880295.552715:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=113 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880295.552718:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=113 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880295.552720:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880295.552722:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880295.552723:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880295.552725:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880295.552728:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880295.552729:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880295.552730:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880295.552731:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880295.552735:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880295.552741:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880295.552743:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880295.552744:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880295.552747:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880295.552754:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780646656:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880295.552758:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.552759:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880295.552761:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000094565713 x1875776780646656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880311 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880295.552767:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880295.552768:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880295.552774:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a21494dd. 02000000:00000001:0.0:1788880295.552777:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880295.552778:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880295.552780:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880295.552782:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880295.552783:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:0.0:1788880295.552785:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880295.552786:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880295.552787:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000094565713. 00000100:00000001:0.0:1788880295.552790:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880295.552791:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.552793:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880295.552796:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880295.552797:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880295.552799:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880295.552801:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880295.614352:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880295.614358:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880295.614363:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880295.614366:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880295.614368:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880295.614371:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880295.614375:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880295.614377:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880295.614379:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880295.614381:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880295.614442:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880295.614447:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880295.614450:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880296.638490:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880296.638499:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880296.638505:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880296.638506:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880296.638600:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880296.638607:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880296.638611:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880297.662351:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880297.662357:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880297.662363:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880297.662365:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880297.662430:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880297.662434:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880297.662437:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880298.686338:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880298.686344:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880298.686351:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880298.686352:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880298.686410:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880298.686414:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880298.686418:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880299.710304:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880299.710310:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880299.710327:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880299.710328:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880299.710381:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880299.710385:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880299.710388:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880300.670384:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305931727, at 4305931776 00000100:00000040:2.0:1788880300.670384:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305931727, at 4305931776 00000100:00000040:3.0:1788880300.670384:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305931727, at 4305931776 00000100:00000040:0.0:1788880300.670404:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:3.0:1788880300.670408:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014321469. 00000100:00000001:0.0:1788880300.670410:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880300.670411:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000751d0eed. 00000100:00000010:2.0:1788880300.670418:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e602733. 00000100:00000010:0.0:1788880300.670421:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000802a821. 00000020:00000040:0.0:1788880300.670428:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880300.670433:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880300.670436:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880300.670441:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880300.670450:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000446e4ede. 00000100:00000001:0.0:1788880300.670457:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880300.670468:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880300.670479:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880300.670481:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880300.670487:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000802a821] to pc [ptlrpcd_00_02+2] req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880300.670514:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880300.670516:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880300.670520:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880300.670537:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006f591845. 00000020:00000040:0.0:1788880300.670539:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880300.670542:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880300.670543:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880300.670545:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880300.670546:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880300.670551:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:0.0:1788880300.670554:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000022b14507. 00000100:00000001:3.0:1788880300.670554:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880300.670563:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880300.670566:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880300.670579:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880300.670579:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880300.670585:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880300.670586:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880300.670602:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880300.670606:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880300.670613:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880300.670619:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880300.670621:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:3.0:1788880300.670625:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780646912:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880300.670626:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880300.670636:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:0.0:1788880300.670638:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006f591845] to pc [ptlrpcd_00_03+3] req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880300.670645:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880300.670650:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880300.670653:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880300.670654:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880300.670657:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880300.670660:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880300.670665:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d4e901df. 02000000:00000010:3.0:1788880300.670665:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009b8d16de. 02000000:00000001:3.0:1788880300.670674:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880300.670675:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880300.670675:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880300.670680:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880300.670682:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880300.670684:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880300.670687:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880300.670689:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000be05c49f. 02000000:00000010:0.0:1788880300.670694:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000005d090de9. 00000100:00000001:2.0:1788880300.670694:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:3.0:1788880300.670699:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000100:00000001:0.0:1788880300.670700:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880300.670705:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880300.670711:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000200:3.0:1788880300.670713:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780646912, portal 25 00000100:00000001:3.0:1788880300.670723:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880300.670729:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849370368 : -126822860181248 : ffff8ca7c0783100) 00000100:00000001:0.0:1788880300.670740:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880300.670744:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880300.670749:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880316 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880300.670756:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:0.0:1788880300.670764:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d4e901df] to pc [ptlrpcd_00_00+0] req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880300.670764:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880300.670772:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880300.670777:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:2.0:1788880300.670778:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880300.670791:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880300.670792:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880300.670800:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007fe20b6f. 00000100:00100000:2.0:1788880300.670807:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006f591845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780647040:192.168.204.111@tcp:400: 00000100:00000200:3.0:1788880300.670809:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780646912, offset 0 00000100:00000001:2.0:1788880300.670818:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1788880300.670822:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880300.670829:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880300.670833:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880300.670834:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880300.670840:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880300.670842:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880300.670849:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008c6f5069. 00000400:00000200:3.0:1788880300.670855:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:2.0:1788880300.670859:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880300.670860:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:3.0:1788880300.670865:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:2.0:1788880300.670874:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009252dbe7. 00000400:00000200:3.0:1788880300.670874:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:0.0:1788880300.670875:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11269) 00000400:00000200:3.0:1788880300.670880:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880300.670885:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:2.0:1788880300.670886:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000044b1d7fc. 00000400:00000200:3.0:1788880300.670891:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [124:124:256:4294967295] 192.168.204.111@tcp LPNI seq info [124:124:8:4294967295] 00000100:00000040:0.0:1788880300.670899:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305931727, at 4305931776 00000100:00000200:2.0:1788880300.670899:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780647040, portal 10 00000100:00000001:2.0:1788880300.670908:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880300.670914:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849363200 : -126822860188416 : ffff8ca7c0781500) 00000400:00000200:3.0:1788880300.670917:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:0.0:1788880300.670918:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000455e6647. 00000800:00000200:3.0:1788880300.670933:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880300.670938:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:3.0:1788880300.670942:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000055d7d92. 00000100:00000001:0.0:1788880300.670947:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880300.670947:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880300.670951:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880300.670953:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880300.670959:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:3.0:1788880300.670959:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880300.670965:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880300.670973:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880300.670974:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880300.670982:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:2.0:1788880300.670984:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e1dae22. 00000100:00000200:2.0:1788880300.670989:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780647040, offset 0 00000800:00000200:3.0:1788880300.670998:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000055d7d92 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880300.671005:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:2.0:1788880300.671005:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880300.671009:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880300.671015:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880300.671017:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880300.671019:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880300.671023:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00100000:0.0:1788880300.671024:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780647168:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880300.671027:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880300.671032:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880300.671040:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305936847 00000400:00000200:2.0:1788880300.671041:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880300.671050:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000010:3.0:1788880300.671051:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008ea3b290. 00000400:00000200:2.0:1788880300.671052:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1788880300.671056:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880300.671061:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880300.671062:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788880300.671064:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880300.671064:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 02000000:00000010:0.0:1788880300.671071:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000bef544e8. 00000400:00000200:2.0:1788880300.671072:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [125:125:255:4294967295] 192.168.204.111@tcp LPNI seq info [125:125:7:4294967295] 02000000:00000001:0.0:1788880300.671075:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880300.671077:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305936847 00000400:00000010:0.0:1788880300.671082:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000100:00000010:3.0:1788880300.671086:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000026aaaaf2. 00000400:00000010:0.0:1788880300.671087:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000200:0.0:1788880300.671096:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780647168, portal 10 00000100:00000001:3.0:1788880300.671097:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880300.671107:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880300.671109:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305936847 00000100:00000001:0.0:1788880300.671112:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849369472 : -126822860182144 : ffff8ca7c0782d80) 00000400:00000200:2.0:1788880300.671114:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:3.0:1788880300.671119:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002aba003e. 00000100:00000040:0.0:1788880300.671129:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880300.671129:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880300.671135:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305936847 00000800:00000200:2.0:1788880300.671144:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788880300.671145:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004800b45d. 00000100:00000001:3.0:1788880300.671152:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000010:2.0:1788880300.671158:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000007cb951ea. 00000100:00000001:0.0:1788880300.671162:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880300.671162:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880300.671173:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880300.671173:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880300.671184:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880300.671186:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880300.671187:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f4d84419. 00000800:00000200:2.0:1788880300.671190:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000007cb951ea type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880300.671196:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880300.671199:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880300.671204:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780647168, offset 0 00000800:00000200:1.0:1788880300.671204:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880300.671204:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880300.671207:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 795650 usecs (796 jiffies) 00000800:00000010:1.0:1788880300.671214:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000055d7d92. 00000100:00000001:2.0:1788880300.671217:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305932572 00000100:00000001:3.0:1788880300.671220:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880300.671222:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880300.671224:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880300.671225:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305936847 00000400:00000200:1.0:1788880300.671275:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880300.671286:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880300.671287:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880300.671290:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305932573 00000400:00000200:0.0:1788880300.671293:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880300.671295:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880300.671298:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305936847 00000400:00000200:0.0:1788880300.671300:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880300.671304:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305932573 00000400:00000200:0.0:1788880300.671305:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880300.671309:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880300.671309:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305936847 00000400:00000200:0.0:1788880300.671314:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [126:126:254:4294967295] 192.168.204.111@tcp LPNI seq info [126:126:6:4294967295] 00000100:00000001:2.0:1788880300.671314:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305932573 00000100:00000001:2.0:1788880300.671318:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305936847 00000100:00000001:2.0:1788880300.671321:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880300.671327:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880300.671327:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880300.671332:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880300.671336:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880300.671337:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007fe20b6f 00000100:00000001:2.0:1788880300.671338:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:0.0:1788880300.671340:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000df3afd78. 00000400:00000010:1.0:1788880300.671345:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007fe20b6f. 00000800:00000200:0.0:1788880300.671353:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880300.671353:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880300.671356:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880300.671358:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:0.0:1788880300.671363:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880300.671364:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:1.0:1788880300.671367:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880316 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880300.671369:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:0.0:1788880300.671371:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000df3afd78 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880300.671374:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880300.671384:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880300.671391:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880300.671392:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880300.671392:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880300.671396:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880300.671406:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880300.671408:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 49155 usecs (50 jiffies) 00000100:00000040:1.0:1788880300.671411:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880316 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880300.671420:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305931827 00000100:00000001:0.0:1788880300.671428:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305936847 00000100:00000001:1.0:1788880300.671432:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880300.671433:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305931827 00000100:00000001:0.0:1788880300.671442:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305936847 00000100:00000001:1.0:1788880300.671442:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880300.671447:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305931827 00000100:00000001:0.0:1788880300.671449:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305936847 00000100:00000001:0.0:1788880300.671450:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305931827 00000100:00000001:0.0:1788880300.671451:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305936847 00000100:00000001:0.0:1788880300.671452:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880300.671453:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880300.671457:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880300.671459:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880300.671469:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880300.671470:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880300.671471:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:1.0:1788880300.671473:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880300.671474:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880300.671476:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880300.671479:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000007cb951ea. 00000100:00000001:0.0:1788880300.671480:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880300.671489:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880300.671497:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880300.671501:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e1dae22 00000400:00000010:1.0:1788880300.671503:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e1dae22. 00000100:00000001:1.0:1788880300.671507:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880300.671511:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880300.671519:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880300.671521:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880300.671527:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880300.671537:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880300.671580:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880300.671583:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000df3afd78. 00000400:00000200:1.0:1788880300.671586:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880300.671605:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880300.671608:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f4d84419 00000400:00000010:1.0:1788880300.671610:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f4d84419. 00000100:00000001:1.0:1788880300.671613:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880300.671617:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880300.671625:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880300.671626:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880300.671632:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880300.671634:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880300.673270:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880300.673285:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880300.673288:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880300.673290:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880300.673296:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880300.673313:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b03a00 00000400:00000200:2.0:1788880300.673320:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x4c5 [1] + 224 00000800:00000001:2.0:1788880300.673324:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880300.673333:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880300.673335:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880300.673338:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880300.673343:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880300.673348:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880300.673355:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880300.673371:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880300.673377:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880300.673381:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880300.673411:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880300.673417:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880300.673424:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880300.673427:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880300.673429:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000be05c49f. 00000400:00000200:3.0:1788880300.673435:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000010:3.0:1788880300.673438:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:3.0:1788880300.673441:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880300.673448:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880300.673457:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880300.673464:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880300.673466:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880300.673469:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880300.673472:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880300.673474:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880300.673477:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880300.673481:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880300.673487:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=128 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880300.673492:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=128 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880300.673494:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880300.673496:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880300.673499:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880300.673502:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880300.673505:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880300.673514:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880300.673521:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880300.673525:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880300.673527:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880300.673539:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880300.673547:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780646912:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880300.673552:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880300.673554:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880300.673556:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000802a821 x1875776780646912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880300.673563:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880300.673566:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880300.673569:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009b8d16de. 02000000:00000001:3.0:1788880300.673575:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880300.673579:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880300.673581:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880300.673585:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880300.673604:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000446e4ede. 02000000:00000001:3.0:1788880300.673612:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880300.673615:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880300.673616:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000802a821. 00000100:00000001:3.0:1788880300.673620:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880300.673621:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880300.673623:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880300.673630:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880300.673632:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880300.673634:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880300.673636:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880300.674259:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880300.674306:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880300.674309:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880300.674311:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880300.674318:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880300.674332:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03a80 00000400:00000200:3.0:1788880300.674340:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x4cd [1] + 224 00000800:00000001:3.0:1788880300.674346:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880300.674356:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880300.674359:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880300.674362:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880300.674368:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880300.674373:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880300.674382:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880300.674399:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880300.674406:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880300.674411:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880300.674413:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880300.674416:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880300.674421:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880300.674428:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03b00 00000400:00000200:3.0:1788880300.674435:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x4d5 [1] + 224 00000100:00000001:2.0:1788880300.674440:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880300.674440:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880300.674447:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880300.674451:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880300.674456:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:3.0:1788880300.674457:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:2.0:1788880300.674460:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:3.0:1788880300.674465:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:2.0:1788880300.674466:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009252dbe7. 00000400:00000200:2.0:1788880300.674473:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000044b1d7fc 00000400:00000010:2.0:1788880300.674484:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000044b1d7fc. 00000100:00000001:2.0:1788880300.674488:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880300.674490:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880300.674499:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880300.674499:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880300.674551:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880300.674564:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880300.674587:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880300.674600:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880300.674602:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880300.674606:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880300.674606:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880300.674615:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880300.674616:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880300.674620:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880300.674621:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880300.674624:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880300.674644:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=118 v=5 (5 0 0 0) 00000100:00001000:2.0:1788880300.674648:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=118 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880300.674652:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880300.674655:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880300.674655:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880300.674661:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:2.0:1788880300.674663:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000001:0.0:1788880300.674667:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00010000:00000001:2.0:1788880300.674668:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880300.674672:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880300.674675:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000010:0.0:1788880300.674676:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000100:00000040:2.0:1788880300.674680:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000200:0.0:1788880300.674684:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000100:00000001:2.0:1788880300.674685:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880300.674687:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880300.674690:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880300.674696:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:2.0:1788880300.674697:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880300.674708:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880300.674720:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880300.674732:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880300.674734:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880300.674743:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880300.674746:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880300.674763:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880300.674766:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880300.674775:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880300.674781:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880300.674784:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00100000:2.0:1788880300.674786:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006f591845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780647040:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880300.674787:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880300.674794:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880300.674800:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880300.674804:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=128 v=5 (5 0 0 0) 00000100:00000001:2.0:1788880300.674805:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00001000:0.0:1788880300.674813:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=102 v=5 (1 0 0 0) 00000100:00000040:2.0:1788880300.674813:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006f591845 x1875776780647040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880300.674825:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880300.674828:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000001:2.0:1788880300.674830:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880300.674832:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880300.674836:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008c6f5069. 00010000:00000001:0.0:1788880300.674837:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:2.0:1788880300.674845:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00010000:00080000:0.0:1788880300.674847:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:2.0:1788880300.674850:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880300.674854:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880300.674865:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:0.0:1788880300.674867:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880300.674869:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000022b14507. 00000100:00000001:0.0:1788880300.674871:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880300.674875:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880300.674877:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000001:2.0:1788880300.674885:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880300.674886:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880300.674893:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880300.674894:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880300.674898:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006f591845. 00000100:00000001:0.0:1788880300.674901:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880300.674903:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880300.674904:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880300.674911:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880300.674914:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880300.674920:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:0.0:1788880300.674930:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780647168:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880300.674931:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880300.674937:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880300.674939:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880300.674940:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d4e901df x1875776780647168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880316 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880300.674945:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880300.674948:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880300.674952:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000bef544e8. 00000100:00000001:2.0:1788880300.674955:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1788880300.674956:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880300.674960:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880300.674960:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880300.674963:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880300.674967:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:0.0:1788880300.674968:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880300.674974:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880300.674975:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000005d090de9. 02000000:00000001:0.0:1788880300.674981:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880300.674984:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880300.674985:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d4e901df. 00000100:00000001:0.0:1788880300.674990:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880300.674991:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880300.674993:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880300.674998:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880300.674999:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880300.675003:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880300.675006:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880300.734354:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880300.734362:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880300.734366:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880300.734367:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880300.734438:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880300.734446:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880300.734449:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880301.758312:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880301.758320:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880301.758326:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880301.758327:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880301.758382:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880301.758386:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880301.758389:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880302.782373:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880302.782379:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880302.782384:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880302.782385:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880302.782433:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880302.782436:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880302.782439:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880303.806330:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880303.806337:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880303.806339:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880303.806342:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880303.806345:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880303.806347:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880303.806348:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880303.806350:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880303.806352:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880303.806367:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880303.806408:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880303.806412:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880303.806415:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880304.830365:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880304.830373:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880304.830378:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880304.830379:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880304.830449:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880304.830455:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880304.830458:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880305.790343:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305936847, at 4305936896 00000100:00000040:2.0:1788880305.790343:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305936847, at 4305936896 00000100:00000040:3.0:1788880305.790343:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305936847, at 4305936896 00000100:00000010:3.0:1788880305.790360:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004800b45d. 00000100:00000010:2.0:1788880305.790361:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002aba003e. 00000100:00000010:1.0:1788880305.790362:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000026aaaaf2. 00000100:00000040:0.0:1788880305.790450:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305936847, at 4305936896 00000100:00000010:0.0:1788880305.790454:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008ea3b290. 00000100:00000040:1.0:1788880305.790516:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880305.790520:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880305.790524:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 00000020:00000040:1.0:1788880305.790527:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880305.790531:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880305.790533:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880305.790536:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880305.790542:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:1.0:1788880305.790548:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880305.790556:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880305.790562:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880305.790564:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880305.790568:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_00_01+1] req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880305.790590:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880305.790592:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880305.790599:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880305.790601:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000020:00000040:1.0:1788880305.790604:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880305.790605:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880305.790606:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880305.790608:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880305.790608:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880305.790609:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880305.790611:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:1.0:1788880305.790612:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:1.0:1788880305.790615:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880305.790615:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880305.790617:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880305.790621:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880305.790625:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880305.790628:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880305.790630:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880305.790633:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880305.790634:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880305.790634:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880305.790636:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880305.790637:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca5e4c87] to pc [ptlrpcd_00_02+2] req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880305.790638:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880305.790641:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780647296:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880305.790651:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880305.790656:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880305.790658:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880305.790659:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880305.790660:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880305.790661:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880305.790663:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000010:2.0:1788880305.790664:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d99c594f. 00000100:00000010:1.0:1788880305.790665:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000663051e5. 02000000:00000001:2.0:1788880305.790666:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880305.790667:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880305.790669:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880305.790671:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000010:2.0:1788880305.790671:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009252dbe7. 02000000:00000001:1.0:1788880305.790673:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880305.790677:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000400:00000010:2.0:1788880305.790677:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000044b1d7fc. 00000100:00000001:1.0:1788880305.790681:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880305.790682:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880305.790682:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880305.790683:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780647296, portal 25 00000100:00000040:1.0:1788880305.790685:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788880305.790686:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:2.0:1788880305.790687:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880305.790689:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 00000100:00000001:0.0:1788880305.790691:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880305.790693:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880305.790696:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880321 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880305.790700:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880305.790700:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880305.790703:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880305.790704:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880305.790706:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880305.790709:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880305.790711:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000663051e5] to pc [ptlrpcd_00_03+3] req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880305.790711:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880305.790712:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880305.790715:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:2.0:1788880305.790717:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009543d7d4. 00000100:00000200:2.0:1788880305.790720:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780647296, offset 0 00000100:00000040:0.0:1788880305.790722:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880305.790728:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880305.790729:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880305.790732:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880305.790736:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880305.790738:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880305.790739:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880305.790743:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:1.0:1788880305.790744:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11274) 00000400:00000200:2.0:1788880305.790744:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880305.790745:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880305.790749:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880305.790750:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788880305.790753:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780647424:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880305.790753:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:3.0:1788880305.790755:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:0.0:1788880305.790760:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880305.790761:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880305.790761:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880305.790764:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880305.790766:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880305.790767:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1788880305.790769:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880305.790769:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880305.790771:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788880305.790774:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000fc6edce. 00000100:00000001:3.0:1788880305.790775:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880305.790777:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788880305.790778:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880305.790779:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:0.0:1788880305.790783:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000097339819. 00000100:00000001:3.0:1788880305.790783:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880305.790785:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [127:127:256:4294967295] 192.168.204.111@tcp LPNI seq info [127:127:8:4294967295] 00000400:00000010:0.0:1788880305.790789:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000040:3.0:1788880305.790791:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880305.790793:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780647424, portal 10 00000100:00000001:0.0:1788880305.790797:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880305.790800:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000400:00000200:2.0:1788880305.790805:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880305.790810:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880305.790813:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880305.790816:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880305.790820:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880305.790823:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880305.790826:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880305.790827:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000010:2.0:1788880305.790829:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c52725cf. 02000000:00000001:3.0:1788880305.790831:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880305.790833:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880305.790838:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000096cf0c4a. 00000100:00100000:3.0:1788880305.790838:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780647552:192.168.204.111@tcp:400: 00000800:00000200:2.0:1788880305.790841:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880305.790845:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880305.790849:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000200:0.0:1788880305.790850:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780647424, offset 0 02000000:00000001:3.0:1788880305.790853:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880305.790855:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:3.0:1788880305.790858:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880305.790862:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880305.790862:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c52725cf type 1, nob 320 niov 1 nkiov 1 02000000:00000010:3.0:1788880305.790864:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000005842920. 02000000:00000001:3.0:1788880305.790869:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880305.790872:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:3.0:1788880305.790873:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000be05c49f. 00000100:00000001:2.0:1788880305.790878:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880305.790880:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:3.0:1788880305.790880:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000100:00000200:3.0:1788880305.790885:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780647552, portal 10 00000400:00000200:0.0:1788880305.790886:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880305.790890:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305941967 00000100:00000001:3.0:1788880305.790891:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880305.790894:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880305.790894:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938053248 : -126823771498368 : ffff8ca78a269880) 00000100:00000010:2.0:1788880305.790897:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ea9971ca. 00000400:00000200:0.0:1788880305.790899:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880305.790904:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000040:3.0:1788880305.790905:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880305.790907:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [128:128:255:4294967295] 192.168.204.111@tcp LPNI seq info [128:128:7:4294967295] 00000100:00000001:2.0:1788880305.790910:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305941967 00000100:00000010:2.0:1788880305.790916:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c61ceeab. 00000100:00000001:2.0:1788880305.790922:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880305.790928:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880305.790928:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305941967 00000100:00000001:3.0:1788880305.790928:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:2.0:1788880305.790935:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000007e595eb4. 00000100:00000040:3.0:1788880305.790935:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880305.790939:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880305.790941:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000010:0.0:1788880305.790943:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000005d090de9. 00000100:00000001:2.0:1788880305.790944:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305941967 00000400:00000010:3.0:1788880305.790949:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8336eb2. 00000800:00000200:0.0:1788880305.790950:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:2.0:1788880305.790950:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a3bf26bf. 00000100:00000200:3.0:1788880305.790955:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780647552, offset 0 00000100:00000001:2.0:1788880305.790957:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:0.0:1788880305.790960:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880305.790962:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880305.790964:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000005d090de9 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880305.790968:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880305.790969:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880305.790974:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:0.0:1788880305.790975:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 771690 usecs (772 jiffies) 00000100:00000001:2.0:1788880305.790975:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880305.790975:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880305.790980:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c52725cf. 00000100:00000001:2.0:1788880305.790980:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880305.790981:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305937668 00000400:00000200:1.0:1788880305.790986:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880305.790987:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305941967 00000100:00000001:2.0:1788880305.790987:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:3.0:1788880305.790987:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880305.790991:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880305.790993:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305937668 00000100:00000001:2.0:1788880305.790997:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880305.790997:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880305.791000:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880305.791003:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880305.791004:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305941967 00000100:00000001:2.0:1788880305.791004:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880305.791006:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880305.791009:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880305.791010:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305937668 00000400:00000200:3.0:1788880305.791012:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880305.791013:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305941967 00000100:00000001:0.0:1788880305.791018:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305937668 00000400:00000200:3.0:1788880305.791018:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880305.791022:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305941967 00000100:00000001:0.0:1788880305.791026:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880305.791027:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [129:129:254:4294967295] 192.168.204.111@tcp LPNI seq info [129:129:6:4294967295] 00000100:00000001:0.0:1788880305.791028:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880305.791033:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880305.791036:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880305.791043:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880305.791044:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880305.791045:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880305.791046:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880305.791105:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880305.791111:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880305.791113:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009543d7d4 00000400:00000010:1.0:1788880305.791117:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009543d7d4. 00000800:00000010:3.0:1788880305.791117:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000749ed09e. 00000100:00000001:1.0:1788880305.791122:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:3.0:1788880305.791125:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:1.0:1788880305.791128:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880321 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880305.791135:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880305.791139:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000200:3.0:1788880305.791140:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000749ed09e type 1, nob 320 niov 1 nkiov 1 00000100:00000040:1.0:1788880305.791145:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880321 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880305.791145:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880305.791150:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 47631 usecs (48 jiffies) 00000100:00000001:3.0:1788880305.791154:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305936944 00000100:00000001:3.0:1788880305.791158:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305941967 00000100:00000001:1.0:1788880305.791160:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880305.791163:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305936944 00000100:00000001:1.0:1788880305.791164:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880305.791167:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305941967 00000100:00000001:3.0:1788880305.791170:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305936944 00000100:00000001:3.0:1788880305.791173:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305941967 00000100:00000001:3.0:1788880305.791175:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305936944 00000100:00000001:3.0:1788880305.791177:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305941967 00000100:00000001:3.0:1788880305.791179:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880305.791182:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880305.791184:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880305.791185:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880305.791190:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000005d090de9. 00000100:00000001:3.0:1788880305.791190:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880305.791196:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880305.791202:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880305.791205:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000096cf0c4a 00000400:00000010:1.0:1788880305.791207:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000096cf0c4a. 00000100:00000001:1.0:1788880305.791210:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880305.791212:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880305.791220:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880305.791221:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880305.791228:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880305.791229:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880305.791272:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880305.791274:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000749ed09e. 00000400:00000200:1.0:1788880305.791276:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880305.791280:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880305.791282:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8336eb2 00000400:00000010:1.0:1788880305.791283:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8336eb2. 00000100:00000001:1.0:1788880305.791285:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880305.791288:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880305.791301:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880305.791304:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880305.791309:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880305.791311:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880305.792695:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880305.792704:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880305.792707:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880305.792710:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880305.792717:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880305.792726:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b03b80 00000400:00000200:2.0:1788880305.792733:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x4dd [1] + 224 00000800:00000001:2.0:1788880305.792739:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880305.792745:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880305.792748:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880305.792751:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880305.792761:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880305.792765:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880305.792775:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880305.792792:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880305.792797:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880305.792800:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880305.792829:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880305.792832:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880305.792836:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880305.792837:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880305.792839:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009252dbe7. 00000400:00000200:1.0:1788880305.792852:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000044b1d7fc 00000400:00000010:1.0:1788880305.792853:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000044b1d7fc. 00000100:00000001:1.0:1788880305.792855:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880305.792859:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880305.792866:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880305.792872:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880305.792874:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880305.792876:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880305.792878:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880305.792880:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880305.792882:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880305.792884:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880305.792889:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=133 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880305.792892:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=133 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880305.792894:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880305.792895:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880305.792898:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880305.792900:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880305.792902:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880305.792905:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880305.792921:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880305.792924:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880305.792925:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880305.792928:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880305.792934:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780647296:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880305.792938:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880305.792940:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880305.792942:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780647296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880305.792948:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880305.792950:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880305.792952:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d99c594f. 02000000:00000001:1.0:1788880305.792955:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880305.792956:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880305.792958:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880305.792960:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880305.792961:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:1.0:1788880305.792964:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880305.792966:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880305.792967:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000001:1.0:1788880305.792970:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880305.792971:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880305.792973:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880305.792976:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880305.792977:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880305.792979:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880305.792980:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880305.793204:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880305.793225:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880305.793228:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880305.793229:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880305.793283:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880305.793295:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03c80 00000400:00000200:3.0:1788880305.793301:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x4ed [1] + 224 00000800:00000001:3.0:1788880305.793306:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880305.793309:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880305.793311:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880305.793313:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880305.793318:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880305.793323:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880305.793330:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880305.793349:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880305.793354:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880305.793356:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880305.793357:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880305.793358:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880305.793360:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880305.793363:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03c00 00000400:00000200:3.0:1788880305.793366:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x4e5 [1] + 224 00000800:00000001:3.0:1788880305.793368:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880305.793373:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880305.793374:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880305.793376:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880305.793378:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880305.793381:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880305.793386:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880305.793387:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880305.793391:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880305.793402:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880305.793404:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880305.793405:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000010:0.0:1788880305.793406:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000be05c49f. 00000400:00000200:0.0:1788880305.793412:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000010:0.0:1788880305.793415:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:0.0:1788880305.793419:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:3.0:1788880305.793423:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880305.793425:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880305.793429:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880305.793441:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880305.793444:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880305.793449:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880305.793456:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880305.793459:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880305.793462:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000010:2.0:1788880305.793462:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000097339819. 00000100:00000001:0.0:1788880305.793467:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880305.793469:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000100:00000001:0.0:1788880305.793474:0:201287:0:(client.c:1580:after_reply()) Process entered 00000400:00000010:2.0:1788880305.793475:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 02000000:00000001:0.0:1788880305.793478:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880305.793481:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880305.793482:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880305.793486:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880305.793500:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880305.793505:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880305.793510:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=133 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880305.793523:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=107 v=5 (1 0 0 0) 00000100:00000200:2.0:1788880305.793528:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880305.793530:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880305.793535:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880305.793542:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880305.793551:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00080000:0.0:1788880305.793552:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880305.793554:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880305.793562:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880305.793567:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880305.793570:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00010000:00000001:0.0:1788880305.793573:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880305.793574:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880305.793581:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:2.0:1788880305.793581:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880305.793585:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880305.793588:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00001000:2.0:1788880305.793588:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=123 v=5 (5 0 0 0) 00000100:00000001:0.0:1788880305.793594:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880305.793597:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=123 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880305.793601:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880305.793604:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880305.793608:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:0.0:1788880305.793612:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880305.793612:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880305.793620:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788880305.793626:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880305.793631:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880305.793632:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880305.793636:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880305.793637:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880305.793640:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880305.793651:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880305.793652:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880305.793689:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780647552:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880305.793693:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880305.793697:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880305.793698:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880305.793703:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880305.793703:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880305.793712:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000663051e5 x1875776780647552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880305.793713:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880305.793729:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780647424:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880305.793745:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880305.793748:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880305.793750:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:0.0:1788880305.793753:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:2.0:1788880305.793756:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780647424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880321 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880305.793758:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000005842920. 02000000:00000001:0.0:1788880305.793770:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880305.793772:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000020:00000001:0.0:1788880305.793774:0:201287:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000001:2.0:1788880305.793777:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000020:00000040:0.0:1788880305.793778:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000010:2.0:1788880305.793781:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000fc6edce. 00000020:00000001:0.0:1788880305.793783:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:2.0:1788880305.793789:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000010:0.0:1788880305.793792:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 00000020:00000001:2.0:1788880305.793793:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880305.793796:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880305.793799:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000020:00000001:2.0:1788880305.793802:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:0.0:1788880305.793803:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000010:2.0:1788880305.793806:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 00000100:00000010:0.0:1788880305.793807:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000663051e5. 00000100:00000001:0.0:1788880305.793816:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:2.0:1788880305.793817:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880305.793818:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880305.793822:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880305.793822:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880305.793827:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:0.0:1788880305.793831:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880305.793833:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880305.793833:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880305.793842:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880305.793843:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880305.793848:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880305.793849:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880305.793860:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880305.793861:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880305.793867:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880305.793870:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880305.854369:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880305.854375:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880305.854381:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880305.854383:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880305.854385:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880305.854405:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880305.854407:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880305.854409:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880305.854411:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880305.854412:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880305.854477:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880305.854481:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880305.854484:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880306.878344:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880306.878352:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880306.878357:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880306.878357:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880306.878412:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880306.878417:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880306.878421:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880307.902343:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880307.902348:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880307.902353:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880307.902353:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880307.902410:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880307.902414:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880307.902417:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880308.926489:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880308.926496:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880308.926501:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880308.926502:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880308.926547:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880308.926552:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880308.926555:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880309.950346:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880309.950353:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880309.950359:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880309.950360:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880309.950450:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880309.950456:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880309.950461:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880310.910386:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305941967, at 4305942016 00000100:00000040:2.0:1788880310.910386:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305941967, at 4305942016 00000100:00000040:1.0:1788880310.910387:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305941967, at 4305942016 00000100:00000040:3.0:1788880310.910403:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:1.0:1788880310.910418:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c61ceeab. 00000100:00000001:3.0:1788880310.910418:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880310.910422:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ea9971ca. 00000100:00000010:2.0:1788880310.910428:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000007e595eb4. 00000100:00000010:3.0:1788880310.910433:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880310.910440:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880310.910447:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880310.910452:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880310.910461:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880310.910473:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:3.0:1788880310.910482:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880310.910496:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880310.910509:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880310.910513:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880310.910522:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_00+0] req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880310.910555:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880310.910557:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880310.910561:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880310.910565:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880310.910568:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880310.910571:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880310.910573:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880310.910576:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.910580:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880310.910580:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000055d7d92. 00000100:00000001:3.0:1788880310.910585:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.910590:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880310.910595:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880310.910601:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880310.910610:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880310.910616:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880310.910620:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880310.910625:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:1.0:1788880310.910627:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880310.910628:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880310.910631:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880310.910636:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880310.910638:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_01+1] req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880310.910642:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780647680:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880310.910648:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880310.910659:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880310.910662:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880310.910664:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880310.910665:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880310.910669:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880310.910670:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000010:1.0:1788880310.910675:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000746bfd07. 00000100:00000010:3.0:1788880310.910676:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 02000000:00000001:1.0:1788880310.910677:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880310.910682:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000010:1.0:1788880310.910685:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000100:00000001:3.0:1788880310.910685:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880310.910690:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000010:1.0:1788880310.910693:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 02000000:00000001:3.0:1788880310.910693:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880310.910698:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000200:1.0:1788880310.910702:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780647680, portal 25 00000100:00000001:3.0:1788880310.910704:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.910705:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880310.910707:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000100:00000040:1.0:1788880310.910711:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880326 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880310.910718:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880310.910720:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880310.910721:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880310.910723:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000100:00000200:1.0:1788880310.910725:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780647680, offset 0 00000400:00000200:1.0:1788880310.910733:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880310.910739:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880310.910742:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880310.910748:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788880310.910754:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880310.910755:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880310.910758:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880310.910760:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:0.0:1788880310.910765:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880310.910771:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880310.910782:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880310.910787:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880310.910789:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1788880310.910795:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [130:130:256:4294967295] 192.168.204.111@tcp LPNI seq info [130:130:8:4294967295] 00000100:00000001:0.0:1788880310.910796:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880310.910806:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880310.910811:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880310.910812:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880310.910824:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880310.910824:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880310.910835:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00100000:0.0:1788880310.910836:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780647808:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880310.910846:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:1.0:1788880310.910850:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:0.0:1788880310.910852:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880310.910857:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880310.910862:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:0.0:1788880310.910863:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:1.0:1788880310.910870:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 02000000:00000010:0.0:1788880310.910873:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000fc6edce. 02000000:00000001:0.0:1788880310.910878:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.910886:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880310.910886:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_02+2] req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880310.910887:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000844d9b13. 00000100:00000001:1.0:1788880310.910897:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305947087 00000400:00000010:0.0:1788880310.910901:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000100:00000010:1.0:1788880310.910905:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000100:00000200:0.0:1788880310.910913:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780647808, portal 10 00000100:00000001:0.0:1788880310.910918:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880310.910918:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880310.910926:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000001:3.0:1788880310.910926:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.910930:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305947087 00000100:00000040:3.0:1788880310.910935:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:1.0:1788880310.910939:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000100:00000040:3.0:1788880310.910942:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788880310.910946:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880310.910949:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880310.910957:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305947087 00000100:00000010:1.0:1788880310.910963:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000100:00000001:1.0:1788880310.910971:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880310.910972:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880310.910981:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305947087 00000100:00000040:0.0:1788880310.910984:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880310.910989:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ff62fd67. 00000100:00000001:1.0:1788880310.910992:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305941967 00000100:00000200:0.0:1788880310.910993:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780647808, offset 0 00000100:00000001:1.0:1788880310.910999:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880310.911007:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880310.911008:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880310.911022:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.911026:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880310.911032:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880310.911039:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880310.911040:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880310.911044:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880310.911049:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880310.911051:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880310.911055:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880310.911059:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880310.911060:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880310.911067:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880310.911067:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880310.911080:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [131:131:255:4294967295] 192.168.204.111@tcp LPNI seq info [131:131:7:4294967295] 00000400:00000200:0.0:1788880310.911093:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880310.911099:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880310.911105:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000079c96644. 00000800:00000200:0.0:1788880310.911113:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:3.0:1788880310.911119:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11279) 00000800:00000200:0.0:1788880310.911121:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880310.911127:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000079c96644 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880310.911133:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880310.911138:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 749100 usecs (750 jiffies) 00000800:00000200:1.0:1788880310.911141:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880310.911142:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305942766 00000800:00000010:1.0:1788880310.911147:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000001:0.0:1788880310.911149:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305947087 00000100:00000001:0.0:1788880310.911156:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305942766 00000400:00000200:1.0:1788880310.911156:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880310.911163:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305947087 00000400:00000200:1.0:1788880310.911168:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880310.911171:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305942766 00000400:00000200:1.0:1788880310.911176:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000100:00000001:0.0:1788880310.911177:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305947087 00000400:00000010:1.0:1788880310.911182:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:0.0:1788880310.911183:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305942766 00000100:00000001:0.0:1788880310.911188:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305947087 00000100:00000001:1.0:1788880310.911190:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880310.911191:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880310.911198:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880310.911200:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880326 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880310.911203:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880310.911210:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880310.911224:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880310.911226:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788880310.911256:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880326 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880310.911261:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880310.911271:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880310.911273:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880310.911275:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880310.911277:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880310.911280:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.911284:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880310.911300:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880310.911304:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880310.911316:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880310.911323:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880310.911333:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880310.911334:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000010:1.0:1788880310.911338:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000079c96644. 00000100:00000001:0.0:1788880310.911339:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880310.911346:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880310.911347:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:0.0:1788880310.911349:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880310.911355:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880310.911359:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00100000:0.0:1788880310.911365:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780647936:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880310.911368:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ff62fd67 00000100:00000001:0.0:1788880310.911375:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:1.0:1788880310.911376:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ff62fd67. 02000000:00000001:0.0:1788880310.911380:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880310.911382:0:201279:0:(events.c:34:request_out_callback()) Process entered 02000000:00000001:0.0:1788880310.911384:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880310.911387:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880310.911391:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000bef544e8. 00000100:00000200:1.0:1788880310.911391:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880310.911410:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880310.911417:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000002cd3292f. 00000100:00000001:1.0:1788880310.911425:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000010:0.0:1788880310.911428:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e73139ca. 00000100:00000040:1.0:1788880310.911432:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880310.911435:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780647936, portal 10 00000100:00000001:0.0:1788880310.911444:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880310.911447:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000100:00000001:1.0:1788880310.911450:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.911455:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000040:0.0:1788880310.911459:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880310.911473:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880310.911477:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880310.911480:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002aa6ba25. 00000100:00000200:0.0:1788880310.911483:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780647936, offset 0 00000400:00000200:0.0:1788880310.911489:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880310.911496:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880310.911499:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880310.911501:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880310.911503:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880310.911505:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880310.911508:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [132:132:256:4294967295] 192.168.204.111@tcp LPNI seq info [132:132:8:4294967295] 00000400:00000200:0.0:1788880310.911521:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880310.911527:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880310.911531:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000079c96644. 00000800:00000200:0.0:1788880310.911535:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880310.911540:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880310.911542:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000079c96644 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880310.911562:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880310.911566:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 46158 usecs (47 jiffies) 00000100:00000001:0.0:1788880310.911568:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305942064 00000100:00000001:0.0:1788880310.911570:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305947087 00000100:00000001:0.0:1788880310.911572:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305942064 00000100:00000001:0.0:1788880310.911573:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305947087 00000100:00000001:0.0:1788880310.911575:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305942064 00000100:00000001:0.0:1788880310.911577:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305947087 00000100:00000001:0.0:1788880310.911580:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305942064 00000100:00000001:0.0:1788880310.911581:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305947087 00000100:00000001:0.0:1788880310.911583:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880310.911586:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880310.911589:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880310.911591:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880310.911596:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880310.911597:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880310.911599:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880310.911600:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880310.911602:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880310.911605:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880310.911635:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880310.911639:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000079c96644. 00000400:00000200:3.0:1788880310.911644:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880310.911655:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880310.911658:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002aa6ba25 00000400:00000010:3.0:1788880310.911660:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002aa6ba25. 00000100:00000001:3.0:1788880310.911663:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880310.911667:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880310.911676:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880310.911678:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880310.911685:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880310.911687:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880310.911933:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880310.911937:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880310.912614:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880310.912623:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880310.912627:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880310.912629:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880310.912768:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880310.912782:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b03d00 00000400:00000200:3.0:1788880310.912791:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x4f5 [1] + 224 00000800:00000001:3.0:1788880310.912841:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880310.912847:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880310.912849:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880310.912853:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880310.912858:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880310.912863:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880310.912887:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880310.912908:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880310.912915:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880310.912918:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880310.913045:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880310.913048:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880310.913054:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880310.913057:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880310.913059:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000400:00000200:1.0:1788880310.913062:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:1.0:1788880310.913064:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:1.0:1788880310.913066:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880310.913070:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880310.913079:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880310.913085:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880310.913086:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880310.913088:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880310.913092:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880310.913095:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880310.913097:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880310.913100:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880310.913107:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=138 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880310.913117:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=138 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880310.913119:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880310.913120:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880310.913122:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880310.913125:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.913128:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880310.913133:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880310.913140:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880310.913143:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880310.913145:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880310.913148:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880310.913156:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780647680:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880310.913161:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880310.913163:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880310.913165:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780647680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880310.913172:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880310.913175:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880310.913177:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000746bfd07. 02000000:00000001:1.0:1788880310.913180:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880310.913182:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880310.913184:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880310.913186:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880310.913188:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 02000000:00000001:1.0:1788880310.913192:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880310.913193:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880310.913194:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:1.0:1788880310.913197:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880310.913198:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880310.913199:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880310.913203:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880310.913209:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.913210:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880310.913212:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880310.913318:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880310.913322:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880310.913333:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880310.913339:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03d80 00000400:00000200:3.0:1788880310.913345:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x4fd [1] + 224 00000800:00000001:3.0:1788880310.913349:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880310.913352:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880310.913354:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880310.913357:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880310.913361:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880310.913365:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880310.913374:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880310.913404:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880310.913409:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880310.913412:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880310.913413:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880310.913414:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880310.913417:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880310.913421:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03e00 00000400:00000200:3.0:1788880310.913424:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x505 [1] + 224 00000800:00000001:3.0:1788880310.913427:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880310.913430:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880310.913431:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880310.913433:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880310.913438:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880310.913442:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880310.913443:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880310.913449:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880310.913453:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880310.913460:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880310.913465:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880310.913474:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000010:0.0:1788880310.913479:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000844d9b13. 00000400:00000200:0.0:1788880310.913487:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000400:00000010:0.0:1788880310.913489:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:0.0:1788880310.913494:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880310.913501:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880310.913511:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880310.913519:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880310.913520:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880310.913523:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880310.913526:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880310.913528:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880310.913530:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880310.913533:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880310.913541:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=128 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880310.913545:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=128 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880310.913547:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880310.913549:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880310.913551:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880310.913555:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880310.913558:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880310.913560:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880310.913562:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880310.913563:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880310.913567:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880310.913575:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880310.913578:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880310.913580:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880310.913585:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880310.913594:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780647808:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880310.913596:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880310.913600:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880310.913610:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880310.913613:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880310.913616:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780647808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880310.913624:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880310.913626:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880310.913629:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000fc6edce. 02000000:00000001:0.0:1788880310.913632:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880310.913634:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880310.913636:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880310.913638:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880310.913640:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000055d7d92. 02000000:00000001:0.0:1788880310.913645:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880310.913647:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880310.913648:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:0.0:1788880310.913652:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880310.913653:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880310.913656:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880310.913662:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880310.913664:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880310.913668:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880310.913669:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880310.913674:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880310.913675:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880310.913701:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880310.913703:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880310.913704:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000002cd3292f. 00000400:00000200:1.0:1788880310.913707:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e73139ca 00000400:00000010:1.0:1788880310.913708:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e73139ca. 00000100:00000001:1.0:1788880310.913711:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880310.913715:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880310.913720:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880310.913724:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880310.913725:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880310.913726:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880310.913728:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880310.913729:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880310.913730:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880310.913731:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880310.913736:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=138 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880310.913738:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=112 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880310.913740:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880310.913741:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880310.913747:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880310.913749:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880310.913754:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.913756:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880310.913757:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880310.913759:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880310.913760:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880310.913763:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880310.913776:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880310.913779:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880310.913780:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880310.913783:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880310.913790:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780647936:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880310.913794:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880310.913797:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880310.913799:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780647936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880326 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880310.913805:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880310.913807:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880310.913809:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000bef544e8. 02000000:00000001:1.0:1788880310.913812:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880310.913814:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880310.913814:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880310.913816:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880310.913817:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 02000000:00000001:1.0:1788880310.913820:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880310.913821:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880310.913822:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:1.0:1788880310.913824:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880310.913825:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880310.913826:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880310.913829:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880310.913830:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880310.913832:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880310.913833:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880310.974380:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880310.974390:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880310.974411:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880310.974412:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880310.974512:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880310.974519:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880310.974524:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880311.998355:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880311.998361:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880311.998366:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880311.998368:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880311.999397:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880311.999402:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880311.999405:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880313.022344:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880313.022351:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880313.022368:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880313.022370:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880313.022451:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880313.022459:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880313.022463:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880314.046359:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880314.046368:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880314.046371:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880314.046375:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880314.046379:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880314.046382:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880314.046384:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880314.046386:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880314.046391:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880314.046393:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880314.046479:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880314.046488:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880314.046493:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880315.070364:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880315.070371:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880315.070377:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880315.070379:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880315.070476:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880315.070480:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880315.070483:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880316.030347:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305947087, at 4305947136 00000100:00000040:0.0:1788880316.030352:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305947087, at 4305947136 00000100:00000040:3.0:1788880316.030366:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:1.0:1788880316.030367:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000010:0.0:1788880316.030368:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000001:3.0:1788880316.030380:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880316.030391:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000020:00000040:3.0:1788880316.030396:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880316.030401:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880316.030405:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880316.030411:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880316.030419:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5b428a0. 00000100:00000001:3.0:1788880316.030426:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880316.030436:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880316.030446:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880316.030449:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880316.030455:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_03+3] req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880316.030490:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305947087, at 4305947136 00000100:00000001:3.0:1788880316.030494:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880316.030497:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000040:3.0:1788880316.030497:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880316.030508:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880316.030511:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:3.0:1788880316.030512:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000100:00000001:1.0:1788880316.030517:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:3.0:1788880316.030518:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880316.030520:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880316.030524:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880316.030524:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880316.030532:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:1.0:1788880316.030533:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880316.030533:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880316.030542:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880316.030544:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880316.030547:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880316.030551:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880316.030557:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880316.030558:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880316.030565:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780648064:192.168.204.111@tcp:400: 02000000:00000010:3.0:1788880316.030565:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:3.0:1788880316.030573:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880316.030576:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880316.030582:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880316.030590:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880316.030591:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880316.030597:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_00+0] req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880316.030600:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880316.030607:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880316.030613:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880316.030619:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000dc03ef0a. 02000000:00000001:1.0:1788880316.030622:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880316.030626:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880316.030628:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000100:00000040:3.0:1788880316.030630:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880316.030635:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880316.030640:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000400:00000010:1.0:1788880316.030641:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000020:00000040:3.0:1788880316.030645:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880316.030646:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880316.030650:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:1.0:1788880316.030651:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780648064, portal 25 00000100:00000001:0.0:1788880316.030653:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880316.030655:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880316.030660:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000100:00000001:3.0:1788880316.030661:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880316.030664:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880316.030666:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880316.030666:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:1.0:1788880316.030672:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880332 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880316.030672:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880316.030673:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880316.030685:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:0.0:1788880316.030696:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880316.030697:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880316.030699:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880316.030699:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880316.030703:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880316.030711:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880316.030711:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880316.030715:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880316.030717:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880316.030718:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000100:00000200:1.0:1788880316.030729:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780648064, offset 0 00000100:00100000:0.0:1788880316.030733:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780648192:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880316.030744:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880316.030746:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880316.030748:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880316.030749:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880316.030752:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880316.030756:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880316.030762:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_01+1] req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880316.030764:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880316.030770:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000058b4ee38. 00000400:00000200:1.0:1788880316.030770:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880316.030779:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880316.030782:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:0.0:1788880316.030794:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000844d9b13. 00000400:00000200:1.0:1788880316.030795:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:0.0:1788880316.030807:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000400:00000200:1.0:1788880316.030809:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:0.0:1788880316.030814:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780648192, portal 10 00000400:00000200:1.0:1788880316.030814:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880316.030820:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880316.030823:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880316.030824:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000400:00000200:1.0:1788880316.030824:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [133:133:256:4294967295] 192.168.204.111@tcp LPNI seq info [133:133:8:4294967295] 00000100:00000040:3.0:1788880316.030827:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880316.030839:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788880316.030845:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880316.030850:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11284) 00000100:00000001:2.0:1788880316.030855:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880316.030861:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880316.030864:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880316.030865:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880316.030866:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880316.030872:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880316.030876:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880316.030876:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880316.030881:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000fbbca8a. 00000100:00000001:2.0:1788880316.030883:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000010:1.0:1788880316.030884:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:2.0:1788880316.030886:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880316.030887:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780648192, offset 0 00000100:00000040:3.0:1788880316.030892:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305947087, at 4305947136 00000800:00000200:1.0:1788880316.030893:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:2.0:1788880316.030900:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788880316.030901:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880316.030908:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:2.0:1788880316.030910:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:3.0:1788880316.030911:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a3bf26bf. 00000800:00000200:1.0:1788880316.030913:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880316.030915:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880316.030916:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880316.030923:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:2.0:1788880316.030923:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780648320:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880316.030929:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880316.030930:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880316.030933:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880316.030934:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:2.0:1788880316.030934:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880316.030939:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880316.030939:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305952207 02000000:00000001:2.0:1788880316.030940:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880316.030944:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:1.0:1788880316.030945:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 02000000:00000010:2.0:1788880316.030949:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005bd099ee. 00000400:00000200:0.0:1788880316.030950:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [134:134:255:4294967295] 192.168.204.111@tcp LPNI seq info [134:134:7:4294967295] 02000000:00000001:2.0:1788880316.030953:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880316.030956:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000010:2.0:1788880316.030959:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000096e78f73. 00000100:00000001:1.0:1788880316.030961:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305952207 00000400:00000010:2.0:1788880316.030964:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000032b7316b. 00000100:00000010:1.0:1788880316.030967:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000400:00000200:0.0:1788880316.030969:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:2.0:1788880316.030969:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780648320, portal 10 00000100:00000001:1.0:1788880316.030971:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:2.0:1788880316.030973:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880316.030975:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000100:00000001:1.0:1788880316.030976:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305952207 00000800:00000200:0.0:1788880316.030980:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880316.030982:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000100:00000040:2.0:1788880316.030986:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:0.0:1788880316.030987:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000005d090de9. 00000100:00000001:1.0:1788880316.030989:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880316.030994:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305952207 00000100:00000001:2.0:1788880316.030994:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880316.030997:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:2.0:1788880316.030998:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880316.030999:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000400:00000010:2.0:1788880316.031001:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c013629f. 00000100:00000200:2.0:1788880316.031003:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780648320, offset 0 00000100:00000001:1.0:1788880316.031004:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:0.0:1788880316.031008:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880316.031009:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880316.031010:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880316.031014:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880316.031015:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000005d090de9 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880316.031017:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880316.031019:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880316.031019:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880316.031022:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880316.031026:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880316.031028:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880316.031028:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880316.031030:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880316.031030:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880316.031033:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880316.031034:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 727800 usecs (728 jiffies) 00000400:00000200:2.0:1788880316.031034:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880316.031035:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880316.031037:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880316.031040:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [135:135:254:4294967295] 192.168.204.111@tcp LPNI seq info [135:135:6:4294967295] 00000100:00000001:1.0:1788880316.031041:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880316.031042:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305947864 00000100:00000001:0.0:1788880316.031046:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305952207 00000100:00000001:0.0:1788880316.031052:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305947864 00000800:00000200:3.0:1788880316.031053:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880316.031062:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000001:0.0:1788880316.031063:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305952207 00000400:00000200:2.0:1788880316.031063:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:3.0:1788880316.031071:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880316.031072:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305947864 00000800:00000200:2.0:1788880316.031078:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880316.031082:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305952207 00000400:00000200:3.0:1788880316.031082:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000010:2.0:1788880316.031088:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000100:00000001:0.0:1788880316.031093:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305947864 00000400:00000200:3.0:1788880316.031094:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000400:00000010:3.0:1788880316.031103:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:0.0:1788880316.031104:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305952207 00000800:00000200:2.0:1788880316.031104:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880316.031112:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880316.031112:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:2.0:1788880316.031120:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:3.0:1788880316.031122:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880332 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880316.031124:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880316.031128:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880316.031138:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880316.031141:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880316.031149:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880316.031153:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 44782 usecs (45 jiffies) 00000100:00000001:0.0:1788880316.031160:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880316.031163:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880316.031163:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880316.031165:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305947181 00000100:00000040:3.0:1788880316.031172:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880332 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880316.031173:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880316.031173:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305952207 00000100:00000001:0.0:1788880316.031182:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880316.031185:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305947181 00000100:00000001:0.0:1788880316.031187:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880316.031194:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305952207 00000100:00000001:0.0:1788880316.031200:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880316.031208:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305947181 00000100:00000001:3.0:1788880316.031215:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880316.031216:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305952207 00000100:00000001:3.0:1788880316.031217:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880316.031222:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305947181 00000100:00000001:2.0:1788880316.031226:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305952207 00000100:00000001:2.0:1788880316.031228:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880316.031231:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880316.031259:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880316.031261:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880316.031264:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880316.031266:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:3.0:1788880316.031266:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880316.031274:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000005d090de9. 00000100:00000001:2.0:1788880316.031276:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880316.031280:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880316.031283:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880316.031286:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880316.031358:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880316.031362:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880316.031365:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000fbbca8a 00000400:00000010:3.0:1788880316.031366:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000fbbca8a. 00000100:00000001:3.0:1788880316.031369:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880316.031371:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880316.031377:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880316.031378:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880316.031382:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880316.031383:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880316.031394:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880316.031396:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000400:00000200:3.0:1788880316.031397:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880316.031450:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880316.031453:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c013629f 00000400:00000010:3.0:1788880316.031455:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c013629f. 00000100:00000001:3.0:1788880316.031457:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880316.031459:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880316.031464:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880316.031466:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880316.031470:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880316.031471:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880316.033015:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880316.033052:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880316.033055:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880316.033057:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880316.033065:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880316.033077:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b03e80 00000400:00000200:3.0:1788880316.033084:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x50d [1] + 224 00000800:00000001:3.0:1788880316.033090:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880316.033092:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880316.033094:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880316.033096:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880316.033100:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880316.033104:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880316.033111:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880316.033125:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880316.033130:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880316.033131:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880316.033132:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880316.033133:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880316.033142:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880316.033147:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03f00 00000400:00000200:3.0:1788880316.033150:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x515 [1] + 224 00000800:00000001:3.0:1788880316.033154:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880316.033156:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880316.033157:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880316.033162:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880316.033164:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880316.033167:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880316.033170:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880316.033173:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880316.033174:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880316.033182:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:3.0:1788880316.033186:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880316.033187:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000100:00000001:3.0:1788880316.033206:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:1.0:1788880316.033209:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:1.0:1788880316.033213:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:1.0:1788880316.033218:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:3.0:1788880316.033220:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880316.033224:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880316.033225:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880316.033229:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880316.033256:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880316.033263:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880316.033264:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880316.033270:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:1.0:1788880316.033270:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880316.033275:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:0.0:1788880316.033276:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880316.033284:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000844d9b13. 00000400:00000200:3.0:1788880316.033290:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b03f80 00000400:00000200:0.0:1788880316.033293:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000400:00000010:0.0:1788880316.033311:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:1.0:1788880316.033314:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880316.033316:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:3.0:1788880316.033321:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x51d [1] + 224 00000100:00000200:0.0:1788880316.033327:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880316.033328:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880316.033330:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880316.033336:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880316.033337:0:201288:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:3.0:1788880316.033343:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1788880316.033345:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880316.033351:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000200:0.0:1788880316.033356:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880316.033356:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880316.033359:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880316.033363:0:201278:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:1.0:1788880316.033369:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880316.033373:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880316.033385:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880316.033386:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=144 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880316.033405:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=144 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880316.033425:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880316.033431:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880316.033442:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880316.033447:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880316.033460:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880316.033462:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880316.033465:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880316.033466:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880316.033469:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:1.0:1788880316.033469:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880316.033473:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880316.033477:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880316.033481:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880316.033486:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:0.0:1788880316.033487:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880316.033497:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=134 v=5 (5 0 0 0) 00000800:00000001:3.0:1788880316.033499:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880316.033500:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00001000:0.0:1788880316.033503:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=134 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880316.033505:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880316.033517:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880316.033517:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880316.033517:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880316.033524:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880316.033525:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880316.033527:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00010000:00000001:0.0:1788880316.033530:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000001:2.0:1788880316.033532:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00010000:00000001:0.0:1788880316.033535:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880316.033535:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880316.033537:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000096e78f73. 00000100:00000001:0.0:1788880316.033543:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000200:2.0:1788880316.033546:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000032b7316b 00000100:00000040:0.0:1788880316.033547:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000010:2.0:1788880316.033556:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000032b7316b. 00000100:00000001:0.0:1788880316.033557:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880316.033561:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880316.033564:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780648064:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880316.033565:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:0.0:1788880316.033571:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880316.033575:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880316.033576:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880316.033584:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880316.033597:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780648064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880316.033615:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880316.033621:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880316.033624:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880316.033627:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:0.0:1788880316.033634:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880316.033636:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880316.033643:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000dc03ef0a. 02000000:00000001:1.0:1788880316.033655:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880316.033659:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000200:2.0:1788880316.033663:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:1.0:1788880316.033664:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00100000:0.0:1788880316.033675:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780648192:192.168.204.111@tcp:400: 00000020:00000001:1.0:1788880316.033675:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880316.033678:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5b428a0. 02000000:00000001:1.0:1788880316.033687:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880316.033697:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880316.033698:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880316.033703:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000001:0.0:1788880316.033710:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880316.033714:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880316.033716:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880316.033717:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880316.033718:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780648192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880316.033725:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880316.033726:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880316.033738:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788880316.033739:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880316.033742:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788880316.033743:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:0.0:1788880316.033744:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880316.033744:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880316.033747:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880316.033748:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880316.033751:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880316.033752:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880316.033757:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000058b4ee38. 02000000:00000001:2.0:1788880316.033760:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880316.033764:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880316.033766:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880316.033767:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880316.033770:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00001000:2.0:1788880316.033770:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=144 v=5 (5 0 0 0) 02000000:00000010:0.0:1788880316.033772:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 02000000:00000001:0.0:1788880316.033778:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880316.033783:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00001000:2.0:1788880316.033783:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=118 v=5 (1 0 0 0) 00000100:00000010:0.0:1788880316.033786:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:0.0:1788880316.033794:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880316.033797:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880316.033798:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880316.033803:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880316.033806:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880316.033810:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880316.033815:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00080000:2.0:1788880316.033815:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880316.033818:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880316.033827:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880316.033832:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880316.033835:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880316.033839:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880316.033841:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880316.033842:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880316.033843:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880316.033846:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880316.033851:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880316.033853:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880316.033854:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880316.033856:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880316.033860:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780648320:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880316.033864:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880316.033872:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880316.033874:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780648320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880332 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880316.033885:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880316.033887:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880316.033889:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005bd099ee. 02000000:00000001:2.0:1788880316.033892:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880316.033896:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880316.033897:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880316.033899:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880316.033900:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 02000000:00000001:2.0:1788880316.033904:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880316.033905:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880316.033906:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:2.0:1788880316.033908:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880316.033909:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880316.033910:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880316.033916:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880316.033917:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880316.033919:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880316.033920:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880316.094358:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880316.094365:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880316.094372:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880316.094375:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880316.094378:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880316.094381:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880316.094384:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880316.094386:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880316.094387:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880316.094389:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880316.094470:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880316.094478:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880316.094482:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880317.118387:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880317.118397:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880317.118405:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880317.118406:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880317.118483:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880317.118488:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880317.118490:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880318.142353:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880318.142360:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880318.142366:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880318.142367:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880318.142420:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880318.142422:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880318.142425:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880319.166347:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880319.166353:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880319.166359:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880319.166361:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880319.166435:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880319.166440:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880319.166443:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880320.190354:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880320.190360:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880320.190368:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880320.190369:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880320.190425:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880320.190430:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880320.190433:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880321.150395:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305952207, at 4305952256 00000100:00000040:1.0:1788880321.150395:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305952207, at 4305952256 00000100:00000040:2.0:1788880321.150395:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305952207, at 4305952256 00000100:00000040:3.0:1788880321.150399:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305952207, at 4305952256 00000100:00000010:2.0:1788880321.150418:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:0.0:1788880321.150426:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000010:1.0:1788880321.150429:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000010:3.0:1788880321.150435:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000040:2.0:1788880321.150458:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880321.150465:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880321.150474:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000020:00000040:2.0:1788880321.150490:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880321.150494:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880321.150496:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880321.150500:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880321.150507:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 00000100:00000001:2.0:1788880321.150513:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880321.150520:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880321.150530:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880321.150533:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880321.150539:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_02+2] req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880321.150573:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880321.150575:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880321.150579:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880321.150581:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000020:00000040:2.0:1788880321.150584:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880321.150586:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880321.150588:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880321.150590:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880321.150594:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c54a5c6f. 00000100:00000001:1.0:1788880321.150597:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880321.150598:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880321.150604:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880321.150607:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880321.150610:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880321.150611:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880321.150617:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880321.150619:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880321.150620:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880321.150624:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880321.150632:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_03+3] req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880321.150637:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880321.150641:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880321.150647:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880321.150648:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880321.150651:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880321.150651:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880321.150657:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880321.150663:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780648448:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880321.150663:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880321.150664:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880321.150671:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002084a2b4. 00000100:00000001:3.0:1788880321.150671:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880321.150673:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880321.150678:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:2.0:1788880321.150679:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880321.150681:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880321.150686:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880321.150687:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880321.150688:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880321.150691:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:2.0:1788880321.150692:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880321.150695:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880321.150699:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 02000000:00000010:1.0:1788880321.150707:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000760cd5b8. 00000100:00000001:2.0:1788880321.150710:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880321.150715:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880321.150727:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:1.0:1788880321.150729:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000100:00000001:3.0:1788880321.150743:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:1.0:1788880321.150745:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000100:00000001:3.0:1788880321.150750:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880321.150751:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000200:1.0:1788880321.150753:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780648448, portal 25 00000100:00000001:1.0:1788880321.150758:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880321.150758:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880321.150760:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880321.150762:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 02000000:00000001:3.0:1788880321.150765:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880321.150766:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002084a2b4] to pc [ptlrpcd_00_00+0] req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880321.150772:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880337 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880321.150777:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880321.150795:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780648576:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880321.150803:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880321.150814:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880321.150815:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:1.0:1788880321.150820:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d71e0867. 00000100:00000200:1.0:1788880321.150824:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780648448, offset 0 00000100:00000001:2.0:1788880321.150824:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880321.150824:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880321.150833:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880321.150834:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880321.150843:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880321.150844:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880321.150847:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880321.150852:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880321.150853:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880321.150858:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:3.0:1788880321.150858:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008e83276d. 00000100:00000040:2.0:1788880321.150861:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11289) 00000400:00000200:1.0:1788880321.150866:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880321.150867:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880321.150876:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880321.150877:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:3.0:1788880321.150880:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000977bf705. 00000100:00000001:2.0:1788880321.150883:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880321.150888:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880321.150891:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880321.150893:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007839347f. 00000400:00000200:1.0:1788880321.150898:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:3.0:1788880321.150904:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780648576, portal 10 00000100:00000001:0.0:1788880321.150906:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1788880321.150908:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880321.150912:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880321.150913:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880321.150919:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 02000000:00000001:0.0:1788880321.150922:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880321.150927:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880321.150927:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [136:136:256:4294967295] 192.168.204.111@tcp LPNI seq info [136:136:8:4294967295] 00000100:00000040:3.0:1788880321.150933:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880321.150935:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880321.150941:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780648704:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880321.150950:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880321.150957:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880321.150965:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880321.150971:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880321.150976:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880321.150984:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000fa0f7569. 00000800:00000200:1.0:1788880321.150994:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880321.150995:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880321.150996:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880321.151000:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000100:00000040:3.0:1788880321.151004:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880321.151011:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cd783db0. 00000800:00000200:1.0:1788880321.151012:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:0.0:1788880321.151014:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000844d9b13. 00000100:00000200:3.0:1788880321.151026:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780648576, offset 0 00000400:00000010:0.0:1788880321.151029:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000800:00000200:1.0:1788880321.151033:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:0.0:1788880321.151040:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780648704, portal 10 00000800:00000200:1.0:1788880321.151040:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880321.151051:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788880321.151051:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880321.151055:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938429184 : -126823771122432 : ffff8ca78a2c5500) 00000100:00000001:1.0:1788880321.151061:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880321.151067:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305957327 00000400:00000200:3.0:1788880321.151070:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:0.0:1788880321.151071:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880321.151074:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000400:00000200:3.0:1788880321.151087:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880321.151088:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1788880321.151097:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305957327 00000400:00000200:3.0:1788880321.151098:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:1.0:1788880321.151105:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000100:00000001:0.0:1788880321.151107:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788880321.151109:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880321.151114:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:0.0:1788880321.151119:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880321.151121:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880321.151125:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305957327 00000400:00000010:0.0:1788880321.151127:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000100:00000010:1.0:1788880321.151133:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000400:00000200:3.0:1788880321.151133:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [137:137:255:4294967295] 192.168.204.111@tcp LPNI seq info [137:137:7:4294967295] 00000100:00000200:0.0:1788880321.151137:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780648704, offset 0 00000100:00000001:1.0:1788880321.151140:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880321.151151:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305957327 00000400:00000200:0.0:1788880321.151156:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880321.151158:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000100:00000001:1.0:1788880321.151168:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788880321.151168:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880321.151184:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1788880321.151187:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880321.151188:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:0.0:1788880321.151189:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880321.151194:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880321.151208:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:0.0:1788880321.151211:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:3.0:1788880321.151213:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000100:00000001:1.0:1788880321.151214:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880321.151216:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880321.151217:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880321.151226:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880321.151227:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880321.151272:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880321.151274:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880321.151283:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880321.151286:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [138:138:254:4294967295] 192.168.204.111@tcp LPNI seq info [138:138:6:4294967295] 00000100:00000001:1.0:1788880321.151288:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880321.151293:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880321.151297:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880321.151300:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880321.151304:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880321.151305:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880321.151314:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880321.151314:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d71e0867 00000400:00000010:2.0:1788880321.151319:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d71e0867. 00000800:00000010:0.0:1788880321.151321:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000022b14507. 00000100:00000001:2.0:1788880321.151325:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:0.0:1788880321.151331:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880321.151334:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880337 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880321.151341:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880321.151343:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880321.151350:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000022b14507 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880321.151361:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880321.151361:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000200:3.0:1788880321.151365:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:0.0:1788880321.151371:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 43497 usecs (44 jiffies) 00000100:00000040:2.0:1788880321.151372:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880337 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880321.151374:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880321.151378:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305952301 00000100:00000001:3.0:1788880321.151386:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880321.151387:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305957327 00000100:00000001:0.0:1788880321.151396:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305952301 00000100:00000040:3.0:1788880321.151397:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 707682 usecs (708 jiffies) 00000100:00000001:2.0:1788880321.151401:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880321.151405:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305957327 00000100:00000001:3.0:1788880321.151405:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305952965 00000100:00000001:2.0:1788880321.151407:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880321.151413:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305957327 00000100:00000001:3.0:1788880321.151422:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305952965 00000100:00000001:0.0:1788880321.151424:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305952301 00000100:00000001:3.0:1788880321.151427:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305957327 00000100:00000001:0.0:1788880321.151430:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305957327 00000800:00000200:2.0:1788880321.151431:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880321.151432:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305952965 00000800:00000010:2.0:1788880321.151442:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000022b14507. 00000100:00000001:0.0:1788880321.151443:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305952301 00000100:00000001:3.0:1788880321.151443:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305957327 00000100:00000001:3.0:1788880321.151454:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305952965 00000100:00000001:0.0:1788880321.151456:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305957327 00000400:00000200:2.0:1788880321.151456:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880321.151466:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880321.151466:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305957327 00000400:00000200:2.0:1788880321.151467:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880321.151473:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880321.151474:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000400:00000010:2.0:1788880321.151479:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:0.0:1788880321.151481:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880321.151483:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880321.151486:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880321.151491:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788880321.151493:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880321.151499:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880321.151500:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880321.151503:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880321.151510:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880321.151515:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880321.151521:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880321.151522:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880321.151523:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880321.151526:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880321.151530:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880321.151531:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880321.151534:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880321.151538:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880321.151549:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880321.151553:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880321.151554:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880321.151557:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880321.151558:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880321.151564:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880321.151596:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880321.151600:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000400:00000200:2.0:1788880321.151604:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880321.151608:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880321.151610:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cd783db0 00000400:00000010:2.0:1788880321.151611:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cd783db0. 00000100:00000001:2.0:1788880321.151614:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880321.151616:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880321.151623:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880321.151625:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880321.151630:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880321.151632:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880321.152677:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880321.152689:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880321.152693:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880321.152766:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880321.152777:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880321.152801:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b04000 00000400:00000200:3.0:1788880321.152883:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x525 [1] + 224 00000800:00000001:3.0:1788880321.152891:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880321.152896:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880321.152898:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880321.152902:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880321.152909:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880321.152916:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880321.152925:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880321.152980:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880321.152988:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880321.152991:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880321.152995:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880321.153000:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880321.153001:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880321.153005:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880321.153010:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:3.0:1788880321.153014:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:1.0:1788880321.153015:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880321.153022:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000400:00000200:1.0:1788880321.153028:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000200:3.0:1788880321.153029:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04100 00000400:00000010:1.0:1788880321.153032:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:1.0:1788880321.153036:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880321.153052:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880321.153065:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880321.153071:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880321.153073:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880321.153076:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880321.153078:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880321.153081:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880321.153083:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880321.153086:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880321.153099:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=149 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880321.153104:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=149 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880321.153107:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880321.153109:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880321.153112:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x535 [1] + 224 00010000:00000001:1.0:1788880321.153117:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:3.0:1788880321.153122:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:1.0:1788880321.153123:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880321.153129:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880321.153134:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880321.153142:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880321.153143:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880321.153148:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880321.153150:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880321.153154:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880321.153164:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880321.153172:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780648448:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880321.153194:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880321.153207:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:3.0:1788880321.153213:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:1.0:1788880321.153217:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780648448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880321.153221:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880321.153230:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880321.153268:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880321.153271:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880321.153276:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880321.153279:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000760cd5b8. 02000000:00000001:1.0:1788880321.153284:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880321.153289:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880321.153295:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880321.153298:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880321.153301:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 00000100:00000001:3.0:1788880321.153301:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880321.153309:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880321.153313:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000800:00000001:3.0:1788880321.153313:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000010:1.0:1788880321.153318:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000800:00000001:3.0:1788880321.153321:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880321.153323:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880321.153324:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000800:00000001:3.0:1788880321.153326:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880321.153328:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880321.153331:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880321.153332:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880321.153334:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880321.153340:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880321.153340:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880321.153347:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000844d9b13. 00000400:00000200:3.0:1788880321.153351:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880321.153356:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880321.153359:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000100:00000001:1.0:1788880321.153360:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880321.153369:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:1.0:1788880321.153369:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880321.153369:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04080 00000100:00000001:1.0:1788880321.153374:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880321.153380:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:3.0:1788880321.153387:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x52d [1] + 224 00000100:00000200:0.0:1788880321.153393:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880321.153394:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880321.153404:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880321.153409:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:0.0:1788880321.153411:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880321.153425:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880321.153439:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880321.153441:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880321.153442:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880321.153447:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000200:3.0:1788880321.153450:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880321.153451:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880321.153456:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880321.153460:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880321.153469:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880321.153478:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=149 v=5 (5 0 0 0) 00000100:00000040:3.0:1788880321.153483:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880321.153501:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=123 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880321.153513:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880321.153515:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880321.153518:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880321.153524:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880321.153527:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880321.153534:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:0.0:1788880321.153536:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880321.153540:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880321.153542:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880321.153546:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880321.153554:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880321.153558:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880321.153563:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880321.153571:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880321.153573:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880321.153575:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880321.153576:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880321.153579:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880321.153582:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:0.0:1788880321.153585:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880321.153586:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880321.153590:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000977bf705. 00000400:00000200:1.0:1788880321.153595:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007839347f 00000100:00100000:0.0:1788880321.153598:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780648704:192.168.204.111@tcp:400: 00000400:00000010:1.0:1788880321.153600:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007839347f. 00000100:00000001:1.0:1788880321.153606:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880321.153607:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880321.153610:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880321.153615:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002084a2b4 x1875776780648704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880321.153615:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880321.153634:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880321.153636:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880321.153639:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000fa0f7569. 02000000:00000001:0.0:1788880321.153643:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880321.153644:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880321.153645:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880321.153647:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880321.153648:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 00000100:00000200:1.0:1788880321.153654:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880321.153660:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880321.153663:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880321.153667:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002084a2b4. 00000100:00000001:1.0:1788880321.153668:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880321.153671:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880321.153673:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880321.153675:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880321.153677:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880321.153681:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880321.153681:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788880321.153684:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880321.153685:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880321.153688:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880321.153689:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880321.153691:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880321.153692:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880321.153695:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880321.153701:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=139 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880321.153705:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=139 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880321.153707:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880321.153709:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880321.153710:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880321.153712:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880321.153715:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880321.153716:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880321.153718:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880321.153719:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880321.153722:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880321.153729:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880321.153731:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880321.153732:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880321.153735:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880321.153741:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780648576:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880321.153745:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880321.153747:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880321.153749:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780648576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880337 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880321.153754:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880321.153756:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880321.153757:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008e83276d. 02000000:00000001:1.0:1788880321.153760:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880321.153761:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880321.153767:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880321.153769:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880321.153771:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c54a5c6f. 02000000:00000001:1.0:1788880321.153773:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880321.153774:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880321.153775:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000001:1.0:1788880321.153777:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880321.153779:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880321.153780:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880321.153784:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880321.153785:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880321.153787:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880321.153788:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880321.214413:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880321.214421:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880321.214428:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880321.214429:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880321.214778:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880321.214786:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880321.214791:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880322.238393:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880322.238403:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880322.238412:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880322.238413:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880322.238483:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880322.238490:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880322.238511:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880323.262337:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880323.262343:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880323.262345:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880323.262347:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880323.262351:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880323.262353:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880323.262354:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880323.262355:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880323.262358:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880323.262359:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880323.262426:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880323.262431:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880323.262435:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880324.286314:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880324.286322:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880324.286330:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880324.286331:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880324.286396:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880324.286400:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880324.286403:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880325.310334:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880325.310340:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880325.310345:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880325.310364:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880325.310366:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880325.310368:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880325.310370:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880325.310371:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880325.310372:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880325.310374:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880325.310425:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880325.310429:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880325.310432:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880326.270310:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305957327, at 4305957376 00000100:00000040:2.0:1788880326.270319:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305957327, at 4305957376 00000100:00000010:1.0:1788880326.270324:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000010:2.0:1788880326.270326:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000040:0.0:1788880326.270351:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305957327, at 4305957376 00000100:00000010:0.0:1788880326.270360:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000040:2.0:1788880326.270368:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880326.270373:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305957327, at 4305957376 00000100:00000001:2.0:1788880326.270374:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880326.270384:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880326.270390:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:3.0:1788880326.270390:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000001:2.0:1788880326.270400:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880326.270403:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880326.270407:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880326.270413:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 00000100:00000001:2.0:1788880326.270419:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880326.270437:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880326.270445:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880326.270447:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880326.270451:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_01+1] req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880326.270472:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880326.270473:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880326.270476:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880326.270477:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000020:00000040:2.0:1788880326.270479:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880326.270480:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880326.270481:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880326.270483:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880326.270486:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 00000100:00000001:2.0:1788880326.270489:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880326.270494:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880326.270503:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880326.270503:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880326.270508:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880326.270509:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880326.270514:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880326.270520:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:0.0:1788880326.270522:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880326.270523:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880326.270531:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fd6b7ee1] to pc [ptlrpcd_00_02+2] req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880326.270538:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880326.270541:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880326.270548:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880326.270550:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880326.270553:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880326.270558:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780648832:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880326.270559:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880326.270563:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880326.270566:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880326.270567:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880326.270570:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:2.0:1788880326.270571:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 02000000:00000001:0.0:1788880326.270572:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000020:00000040:2.0:1788880326.270575:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880326.270576:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880326.270576:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880326.270580:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880326.270581:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880326.270583:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:2.0:1788880326.270584:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:1.0:1788880326.270588:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880326.270588:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880326.270591:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000fa0f7569. 00000100:00000001:3.0:1788880326.270591:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880326.270592:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 00000100:00000001:3.0:1788880326.270599:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880326.270601:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880326.270601:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880326.270612:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:0.0:1788880326.270613:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000844d9b13. 00000100:00000001:1.0:1788880326.270615:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880326.270616:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880326.270620:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880326.270623:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880326.270624:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000100:00000001:2.0:1788880326.270627:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:1.0:1788880326.270628:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880326.270630:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880326.270633:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780648832, portal 25 00000100:00100000:1.0:1788880326.270633:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780648960:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880326.270637:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_03+3] req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880326.270639:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880326.270642:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880326.270643:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 02000000:00000001:1.0:1788880326.270646:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880326.270649:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880326.270652:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880326.270655:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880342 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880326.270656:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009e1ad299. 02000000:00000001:1.0:1788880326.270659:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880326.270662:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880326.270663:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000400:00000010:1.0:1788880326.270669:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000100:00000040:2.0:1788880326.270669:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:1.0:1788880326.270675:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780648960, portal 10 00000100:00000001:3.0:1788880326.270680:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880326.270681:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:2.0:1788880326.270681:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880326.270682:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880326.270685:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000100:00000001:3.0:1788880326.270686:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880326.270694:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880326.270694:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880326.270697:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880326.270703:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000115221d8. 00000100:00000040:2.0:1788880326.270706:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11295) 00000100:00000040:3.0:1788880326.270710:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880326.270713:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780648832, offset 0 00000400:00000200:0.0:1788880326.270744:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880326.270756:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880326.270758:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880326.270758:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880326.270767:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788880326.270770:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:1.0:1788880326.270772:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880326.270772:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880326.270779:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880326.270783:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:1.0:1788880326.270783:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000062e8e6f4. 00000100:00100000:3.0:1788880326.270787:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780649088:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880326.270791:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780648960, offset 0 00000400:00000200:0.0:1788880326.270794:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880326.270801:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880326.270806:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880326.270809:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880326.270812:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880326.270814:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880326.270817:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880326.270821:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880326.270824:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [139:139:256:4294967295] 192.168.204.111@tcp LPNI seq info [139:139:8:4294967295] 02000000:00000010:3.0:1788880326.270830:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000c9a81c5. 02000000:00000001:3.0:1788880326.270835:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880326.270844:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:3.0:1788880326.270845:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007e36d068. 00000400:00000010:3.0:1788880326.270854:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000400:00000200:1.0:1788880326.270861:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:0.0:1788880326.270862:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:3.0:1788880326.270863:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780649088, portal 10 00000100:00000001:3.0:1788880326.270873:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880326.270874:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:0.0:1788880326.270875:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000df3afd78. 00000400:00000200:1.0:1788880326.270880:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788880326.270885:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880326.270885:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880326.270893:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938435456 : -126823771116160 : ffff8ca78a2c6d80) 00000400:00000200:1.0:1788880326.270897:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788880326.270902:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:3.0:1788880326.270908:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880326.270909:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000df3afd78 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880326.270912:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [140:140:255:4294967295] 192.168.204.111@tcp LPNI seq info [140:140:7:4294967295] 00000100:00000001:0.0:1788880326.270934:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880326.270938:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880326.270941:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305962447 00000100:00000040:3.0:1788880326.270949:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880326.270953:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eab8d071. 00000100:00000010:0.0:1788880326.270954:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006bd48548. 00000100:00000200:3.0:1788880326.270964:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780649088, offset 0 00000400:00000200:1.0:1788880326.270965:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880326.270966:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880326.270981:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305962447 00000400:00000200:3.0:1788880326.270984:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880326.270990:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880326.270993:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e6f9f09c. 00000800:00000010:1.0:1788880326.270998:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000001:0.0:1788880326.270999:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:3.0:1788880326.271003:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880326.271013:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880326.271015:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305962447 00000800:00000200:1.0:1788880326.271015:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880326.271023:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:0.0:1788880326.271026:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000535177f1. 00000800:00000200:1.0:1788880326.271032:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880326.271032:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880326.271035:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:3.0:1788880326.271040:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880326.271042:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880326.271043:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305962447 00000100:00000001:1.0:1788880326.271050:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880326.271053:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:0.0:1788880326.271054:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e64ede4. 00000100:00000040:1.0:1788880326.271055:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 688658 usecs (689 jiffies) 00000400:00000200:3.0:1788880326.271055:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [141:141:254:4294967295] 192.168.204.111@tcp LPNI seq info [141:141:6:4294967295] 00000800:00000010:2.0:1788880326.271061:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000df3afd78. 00000100:00000001:1.0:1788880326.271065:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305958065 00000400:00000200:2.0:1788880326.271073:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880326.271074:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880326.271076:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305962447 00000100:00000001:1.0:1788880326.271083:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305958065 00000100:00000001:0.0:1788880326.271086:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880326.271088:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305962447 00000400:00000200:2.0:1788880326.271095:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880326.271096:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880326.271098:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305958065 00000400:00000200:3.0:1788880326.271112:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880326.271120:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305962447 00000100:00000001:0.0:1788880326.271121:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880326.271132:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305958065 00000100:00000001:0.0:1788880326.271133:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880326.271135:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000115221d8 00000800:00000200:3.0:1788880326.271136:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880326.271147:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305962447 00000400:00000010:2.0:1788880326.271147:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000115221d8. 00000800:00000010:3.0:1788880326.271148:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000100:00000001:0.0:1788880326.271152:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880326.271157:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880326.271159:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880326.271160:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1788880326.271160:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880326.271171:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880326.271171:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880326.271174:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880342 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880326.271175:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880326.271176:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880326.271181:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880326.271182:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880326.271184:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:3.0:1788880326.271193:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880326.271194:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880326.271206:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880326.271206:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880326.271209:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880326.271213:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880326.271213:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880342 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880326.271217:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880326.271219:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880326.271224:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880326.271225:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880326.271228:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880326.271263:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880326.271265:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:2.0:1788880326.271272:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880326.271275:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880326.271277:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000062e8e6f4 00000400:00000010:2.0:1788880326.271278:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000062e8e6f4. 00000100:00000001:2.0:1788880326.271280:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880326.271283:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880326.271288:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880326.271290:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880326.271294:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880326.271296:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880326.271366:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880326.271370:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 42280 usecs (43 jiffies) 00000100:00000001:3.0:1788880326.271373:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305957420 00000100:00000001:3.0:1788880326.271375:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305962447 00000800:00000200:2.0:1788880326.271400:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880326.271403:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000400:00000200:2.0:1788880326.271407:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880326.271410:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880326.271413:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eab8d071 00000400:00000010:2.0:1788880326.271415:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eab8d071. 00000100:00000001:2.0:1788880326.271417:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880326.271420:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880326.271424:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305957420 00000100:00000001:3.0:1788880326.271427:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305962447 00000100:00000001:3.0:1788880326.271434:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305957420 00000100:00000001:3.0:1788880326.271441:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305962447 00000100:00000001:3.0:1788880326.271448:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305957420 00000100:00000001:2.0:1788880326.271461:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880326.271463:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880326.271466:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880326.271467:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880326.271507:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305962447 00000100:00000001:3.0:1788880326.271509:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880326.271511:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880326.271514:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880326.271516:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880326.271523:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880326.271525:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880326.271527:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880326.271528:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880326.271529:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880326.271531:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880326.272681:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880326.272693:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880326.272696:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880326.272699:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880326.272708:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880326.272756:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b04180 00000400:00000200:3.0:1788880326.272766:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x53d [1] + 224 00000800:00000001:3.0:1788880326.272774:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880326.272782:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880326.272784:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880326.272788:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880326.272796:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880326.272801:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880326.272812:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880326.272834:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880326.272842:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880326.272845:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880326.272877:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880326.272881:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880326.272886:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880326.272888:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880326.272890:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000844d9b13. 00000400:00000200:0.0:1788880326.272894:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000400:00000010:0.0:1788880326.272895:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:0.0:1788880326.272897:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880326.272902:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880326.272909:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880326.272914:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880326.272915:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880326.272917:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880326.272919:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880326.272920:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880326.272922:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880326.272923:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880326.272928:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=154 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880326.272931:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=154 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880326.272933:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880326.272933:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880326.272935:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880326.272937:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880326.272946:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880326.272948:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880326.272953:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880326.272954:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880326.272956:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880326.272958:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880326.272962:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780648832:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880326.272965:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880326.272966:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880326.272967:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780648832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880326.272971:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880326.272973:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880326.272974:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000fa0f7569. 02000000:00000001:0.0:1788880326.272976:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880326.272977:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880326.272978:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880326.272980:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880326.272981:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 02000000:00000001:0.0:1788880326.272984:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880326.272985:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880326.272987:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:0.0:1788880326.272990:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880326.272991:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880326.272993:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880326.272998:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880326.272999:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880326.273001:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880326.273003:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880326.273169:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880326.273179:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880326.273182:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880326.273184:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880326.273190:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880326.273202:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04280 00000400:00000200:2.0:1788880326.273209:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x54d [1] + 224 00000800:00000001:2.0:1788880326.273215:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880326.273224:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880326.273227:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880326.273230:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880326.273256:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880326.273259:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880326.273267:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880326.273279:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880326.273287:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880326.273292:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880326.273295:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880326.273298:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880326.273306:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880326.273311:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880326.273319:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:3.0:1788880326.273319:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880326.273330:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04200 00000400:00000001:3.0:1788880326.273333:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880326.273376:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007e36d068. 00000400:00000200:3.0:1788880326.273380:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880326.273382:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880326.273386:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880326.273390:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x541 [1] + 224 00000100:00000200:3.0:1788880326.273396:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880326.273398:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1788880326.273409:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880326.273413:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880326.273422:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880326.273425:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880326.273429:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880326.273433:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880326.273435:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880326.273439:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880326.273440:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880326.273442:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880326.273446:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880326.273449:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880326.273452:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880326.273463:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880326.273474:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=154 v=5 (5 0 0 0) 00000100:00001000:3.0:1788880326.273480:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=128 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880326.273482:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880326.273484:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880326.273487:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880326.273490:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788880326.273492:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00080000:3.0:1788880326.273497:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880326.273498:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880326.273512:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880326.273517:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880326.273519:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880326.273522:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880326.273524:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880326.273528:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880326.273534:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880326.273535:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880326.273539:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880326.273542:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000001:1.0:1788880326.273545:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880326.273546:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880326.273548:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880326.273549:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000400:00000200:1.0:1788880326.273552:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:1.0:1788880326.273553:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:1.0:1788880326.273562:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1788880326.273567:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880326.273568:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880326.273637:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780649088:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880326.273643:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880326.273648:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880326.273656:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880326.273661:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880326.273662:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000175072aa x1875776780649088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880326.273664:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880326.273667:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880326.273668:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880326.273669:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880326.273671:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880326.273672:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880326.273680:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=144 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880326.273691:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=144 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880326.273691:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880326.273696:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880326.273697:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880326.273701:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880326.273701:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000c9a81c5. 00010000:00000001:1.0:1788880326.273708:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:3.0:1788880326.273709:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00010000:00000001:1.0:1788880326.273712:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880326.273713:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880326.273718:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880326.273720:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000020:00000001:3.0:1788880326.273722:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880326.273725:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 02000000:00000001:3.0:1788880326.273727:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880326.273731:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880326.273733:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000175072aa. 00000100:00000001:3.0:1788880326.273737:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880326.273741:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880326.273742:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880326.273745:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880326.273748:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880326.273752:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880326.273753:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880326.273757:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880326.273761:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880326.273761:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880326.273765:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880326.273776:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880326.273780:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880326.273783:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880326.273787:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880326.273795:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780648960:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880326.273800:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880326.273802:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880326.273804:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000fd6b7ee1 x1875776780648960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880342 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880326.273810:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880326.273812:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880326.273814:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009e1ad299. 02000000:00000001:1.0:1788880326.273817:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880326.273818:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880326.273820:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880326.273822:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880326.273824:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 02000000:00000001:1.0:1788880326.273827:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880326.273828:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880326.273830:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000fd6b7ee1. 00000100:00000001:1.0:1788880326.273833:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880326.273834:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880326.273836:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880326.273840:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880326.273842:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880326.273843:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880326.273845:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880326.334318:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880326.334326:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880326.334331:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880326.334332:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880326.334379:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880326.334383:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880326.334386:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880327.358359:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880327.358367:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880327.358373:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880327.358374:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880327.358467:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880327.358475:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880327.358480:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880328.383146:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880328.383153:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880328.383159:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880328.383160:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880328.383263:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880328.383267:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880328.383270:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880329.406353:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880329.406368:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880329.406376:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880329.406378:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880329.406426:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880329.406430:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880329.406432:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880330.430359:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880330.430366:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880330.430371:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880330.430372:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880330.430421:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880330.430425:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880330.430427:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880331.390362:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305962447, at 4305962496 00000100:00000040:2.0:1788880331.390362:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880331.390362:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305962447, at 4305962496 00000100:00000040:0.0:1788880331.390369:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305962447, at 4305962496 00000100:00000001:2.0:1788880331.390380:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880331.390385:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e64ede4. 00000100:00000010:1.0:1788880331.390387:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e6f9f09c. 00000100:00000010:0.0:1788880331.390389:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006bd48548. 00000100:00000010:2.0:1788880331.390393:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000078d5a15d. 00000020:00000040:2.0:1788880331.390399:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880331.390405:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880331.390410:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880331.390416:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880331.390425:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e53a8f34. 00000100:00000001:2.0:1788880331.390435:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880331.390507:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880331.390519:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880331.390522:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880331.390538:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000078d5a15d] to pc [ptlrpcd_00_00+0] req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880331.390567:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880331.390570:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880331.390574:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880331.390577:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880331.390580:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880331.390587:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880331.390593:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880331.390599:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880331.390605:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880331.390615:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880331.390616:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880331.390621:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880331.390624:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880331.390628:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880331.390632:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780649216:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880331.390637:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880331.390637:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880331.390640:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880331.390641:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880331.390646:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880331.390647:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880331.390647:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000010:2.0:1788880331.390652:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 02000000:00000001:1.0:1788880331.390658:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880331.390659:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880331.390663:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880331.390667:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000010:1.0:1788880331.390671:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000bd87bd8d. 02000000:00000001:1.0:1788880331.390675:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880331.390682:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880331.390683:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880331.390685:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000100:00000040:2.0:1788880331.390688:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_00_01+1] req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880331.390696:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000100:00000001:2.0:1788880331.390705:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880331.390706:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780649216, portal 25 00000100:00000040:2.0:1788880331.390710:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880331.390711:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880331.390714:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938426496 : -126823771125120 : ffff8ca78a2c4a80) 00000100:00000001:2.0:1788880331.390717:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880331.390721:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000100:00000040:1.0:1788880331.390723:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880347 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:2.0:1788880331.390725:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880331.390730:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880331.390730:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880331.390735:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880331.390736:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880331.390737:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:2.0:1788880331.390738:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880331.390743:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880331.390744:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:2.0:1788880331.390745:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 00000400:00000010:1.0:1788880331.390747:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000100:00000001:2.0:1788880331.390751:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880331.390752:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780649216, offset 0 00000100:00000040:0.0:1788880331.390753:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880331.390760:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:1.0:1788880331.390774:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880331.390795:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880331.390806:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1788880331.390806:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880331.390808:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880331.390810:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880331.390811:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880331.390813:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880331.390818:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880331.390822:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1788880331.390822:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d838cab4] to pc [ptlrpcd_00_02+2] req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880331.390823:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880331.390829:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880331.390836:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780649344:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880331.390842:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880331.390844:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880331.390848:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1788880331.390850:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [142:142:256:4294967295] 192.168.204.111@tcp LPNI seq info [142:142:8:4294967295] 02000000:00000001:0.0:1788880331.390851:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880331.390855:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880331.390861:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009a0ddd94. 02000000:00000001:0.0:1788880331.390864:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880331.390864:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880331.390871:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:1.0:1788880331.390873:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:0.0:1788880331.390877:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000844d9b13. 00000100:00000040:2.0:1788880331.390878:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000800:00000200:1.0:1788880331.390885:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880331.390886:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000100:00000040:2.0:1788880331.390887:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11300) 00000800:00000010:1.0:1788880331.390893:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000200:0.0:1788880331.390894:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780649344, portal 10 00000100:00000001:0.0:1788880331.390899:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880331.390900:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000800:00000200:1.0:1788880331.390904:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:2.0:1788880331.390905:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305962447, at 4305962496 00000100:00000010:2.0:1788880331.390913:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000535177f1. 00000100:00000040:0.0:1788880331.390914:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880331.390919:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880331.390927:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880331.390929:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880331.390935:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880331.390938:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880331.390939:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000200:0.0:1788880331.390944:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780649344, offset 0 00000100:00000001:1.0:1788880331.390946:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305967567 00000100:00000010:1.0:1788880331.390951:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000400:00000200:0.0:1788880331.390952:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880331.390960:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880331.390963:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880331.390964:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880331.390969:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305967567 00000100:00000010:1.0:1788880331.390972:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000100:00000001:1.0:1788880331.390975:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880331.390977:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305967567 00000100:00000010:1.0:1788880331.390980:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000400:00000200:0.0:1788880331.390984:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880331.390985:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:0.0:1788880331.390989:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880331.390991:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305967567 00000400:00000200:0.0:1788880331.390995:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:1.0:1788880331.390995:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000400:00000200:0.0:1788880331.391001:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [143:143:255:4294967295] 192.168.204.111@tcp LPNI seq info [143:143:7:4294967295] 00000100:00000001:1.0:1788880331.391001:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880331.391018:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880331.391022:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880331.391027:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880331.391028:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880331.391033:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880331.391039:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880331.391044:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000df3afd78. 00000100:00000001:1.0:1788880331.391044:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880331.391048:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:0.0:1788880331.391053:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880331.391054:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880331.391057:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880331.391059:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880331.391061:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880331.391063:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880331.391065:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000df3afd78 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880331.391070:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880331.391076:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 670638 usecs (671 jiffies) 00000800:00000200:2.0:1788880331.391077:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880331.391080:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305963167 00000800:00000010:2.0:1788880331.391087:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000100:00000001:0.0:1788880331.391088:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305967567 00000400:00000200:2.0:1788880331.391094:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880331.391099:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305963167 00000400:00000200:2.0:1788880331.391101:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880331.391106:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305967567 00000100:00000001:0.0:1788880331.391109:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305963167 00000400:00000200:2.0:1788880331.391109:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000100:00000001:0.0:1788880331.391114:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305967567 00000400:00000010:2.0:1788880331.391114:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:0.0:1788880331.391119:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305963167 00000100:00000001:0.0:1788880331.391121:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305967567 00000100:00000001:2.0:1788880331.391122:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880331.391124:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788880331.391128:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880347 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880331.391129:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880331.391133:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880331.391139:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880331.391147:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880331.391147:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880331.391150:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:2.0:1788880331.391154:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880347 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880331.391155:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880331.391158:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880331.391161:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880331.391166:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880331.391168:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880331.391171:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880331.391187:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880331.391190:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000df3afd78. 00000400:00000200:2.0:1788880331.391193:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880331.391196:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880331.391199:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880331.391199:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000100:00000001:0.0:1788880331.391203:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880331.391205:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:2.0:1788880331.391209:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000040:0.0:1788880331.391212:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880331.391214:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880331.391221:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880331.391229:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880331.391258:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880331.391264:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:0.0:1788880331.391265:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880331.391269:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880331.391270:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880331.391274:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880331.391282:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780649472:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880331.391283:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880331.391287:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880331.391289:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880331.391293:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880331.391297:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880331.391300:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880331.391304:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000027d4cece. 02000000:00000001:0.0:1788880331.391307:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880331.391310:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008e3802a2. 00000100:00000001:3.0:1788880331.391315:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788880331.391316:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a09430ba. 00000100:00000200:0.0:1788880331.391320:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780649472, portal 10 00000100:00000001:0.0:1788880331.391323:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880331.391325:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000100:00000040:0.0:1788880331.391332:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880331.391339:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880331.391345:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880331.391351:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000200:0.0:1788880331.391354:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780649472, offset 0 00000400:00000200:0.0:1788880331.391359:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880331.391359:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880331.391371:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880331.391375:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880331.391378:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880331.391388:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880331.391391:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880331.391393:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [144:144:256:4294967295] 192.168.204.111@tcp LPNI seq info [144:144:8:4294967295] 00000400:00000200:0.0:1788880331.391402:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880331.391407:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880331.391411:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000022b14507. 00000800:00000200:0.0:1788880331.391415:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880331.391419:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880331.391422:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000022b14507 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880331.391441:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880331.391445:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 41138 usecs (42 jiffies) 00000100:00000001:0.0:1788880331.391447:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305962539 00000100:00000001:0.0:1788880331.391448:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305967567 00000100:00000001:0.0:1788880331.391450:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305962539 00000100:00000001:0.0:1788880331.391452:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305967567 00000100:00000001:0.0:1788880331.391454:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305962539 00000100:00000001:0.0:1788880331.391456:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305967567 00000100:00000001:0.0:1788880331.391458:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305962539 00000100:00000001:0.0:1788880331.391460:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305967567 00000100:00000001:0.0:1788880331.391462:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880331.391465:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880331.391468:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880331.391470:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880331.391474:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880331.391475:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880331.391478:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880331.391479:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880331.391481:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880331.391485:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880331.391558:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880331.391562:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000022b14507. 00000400:00000200:1.0:1788880331.391565:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880331.391571:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880331.391574:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:1.0:1788880331.391575:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:1.0:1788880331.391578:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880331.391583:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880331.391593:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880331.391596:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880331.391603:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880331.391610:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880331.392949:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880331.392966:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880331.392970:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880331.392972:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880331.392982:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880331.392997:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04380 00000400:00000200:2.0:1788880331.393020:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x55d [1] + 224 00000800:00000001:2.0:1788880331.393027:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880331.393031:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880331.393035:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880331.393038:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880331.393045:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880331.393051:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880331.393061:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880331.393085:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880331.393094:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880331.393097:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880331.393099:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880331.393101:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880331.393104:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880331.393110:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b04300 00000400:00000200:2.0:1788880331.393130:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x555 [1] + 224 00000800:00000001:2.0:1788880331.393135:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880331.393143:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880331.393146:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880331.393148:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880331.393151:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880331.393155:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880331.393161:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880331.393181:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880331.393185:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880331.393188:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880331.393258:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880331.393262:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880331.393271:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880331.393274:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880331.393277:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000844d9b13. 00000400:00000200:0.0:1788880331.393281:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000400:00000010:0.0:1788880331.393283:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:0.0:1788880331.393286:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880331.393291:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880331.393300:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880331.393305:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880331.393308:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880331.393311:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880331.393314:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880331.393317:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880331.393319:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880331.393322:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880331.393333:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=149 v=5 (5 0 0 0) 00000100:00001000:0.0:1788880331.393337:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=149 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880331.393340:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880331.393341:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880331.393345:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880331.393348:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880331.393353:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880331.393355:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880331.393357:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880331.393359:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880331.393363:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880331.393370:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880331.393373:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880331.393375:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880331.393379:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880331.393394:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780649344:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880331.393400:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880331.393402:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880331.393405:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780649344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880331.393412:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880331.393414:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880331.393417:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009a0ddd94. 02000000:00000001:0.0:1788880331.393421:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880331.393425:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880331.393426:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880331.393429:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880331.393431:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 02000000:00000001:0.0:1788880331.393435:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880331.393437:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880331.393439:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:0.0:1788880331.393443:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880331.393444:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880331.393447:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880331.393453:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880331.393454:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880331.393457:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880331.393459:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880331.393521:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880331.393525:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880331.393530:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880331.393532:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880331.393534:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000400:00000200:1.0:1788880331.393538:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:1.0:1788880331.393540:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:1.0:1788880331.393544:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880331.393548:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880331.393556:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880331.393564:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880331.393565:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880331.393567:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880331.393568:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880331.393569:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880331.393571:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880331.393572:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880331.393575:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=9 v=5 (5 5 0 0) 00000800:00000001:2.0:1788880331.393577:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:1.0:1788880331.393579:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=9 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880331.393582:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880331.393584:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880331.393586:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880331.393588:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880331.393589:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880331.393592:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880331.393646:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880331.393649:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880331.393653:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880331.393662:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880331.393672:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880331.393675:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880331.393677:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880331.393677:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04400 00000100:00000040:1.0:1788880331.393686:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880331.393689:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x565 [1] + 224 00000800:00000001:2.0:1788880331.393699:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:1.0:1788880331.393703:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780649216:192.168.204.111@tcp:400: 00000800:00000001:2.0:1788880331.393707:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880331.393711:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880331.393718:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880331.393719:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880331.393723:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880331.393725:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880331.393732:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880331.393741:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000078d5a15d x1875776780649216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880331.393744:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880331.393762:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880331.393765:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880331.393767:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000bd87bd8d. 02000000:00000001:1.0:1788880331.393771:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880331.393772:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880331.393773:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880331.393776:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880331.393776:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:1.0:1788880331.393778:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e53a8f34. 02000000:00000001:1.0:1788880331.393780:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880331.393782:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880331.393783:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000078d5a15d. 00000100:00000001:1.0:1788880331.393788:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880331.393789:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880331.393791:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880331.393792:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880331.393796:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000001:2.0:1788880331.393797:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880331.393799:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880331.393803:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880331.393805:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880331.393878:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880331.393885:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880331.393899:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880331.393901:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880331.393904:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008e3802a2. 00000400:00000200:0.0:1788880331.393911:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a09430ba 00000400:00000010:0.0:1788880331.393913:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a09430ba. 00000100:00000001:0.0:1788880331.393917:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880331.393928:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880331.393939:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880331.393956:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880331.393958:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880331.393961:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880331.393964:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880331.393966:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880331.393969:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880331.393972:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880331.393980:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=9 v=5 (5 5 0 0) 00000100:00001000:0.0:1788880331.393983:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=133 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880331.393986:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880331.393987:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880331.393990:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880331.393994:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880331.394001:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880331.394015:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880331.394016:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880331.394019:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880331.394020:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880331.394023:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880331.394030:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880331.394033:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880331.394035:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880331.394047:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880331.394055:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780649472:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880331.394060:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880331.394065:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880331.394068:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d838cab4 x1875776780649472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880347 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880331.394085:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880331.394087:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880331.394090:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000027d4cece. 02000000:00000001:0.0:1788880331.394093:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880331.394095:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880331.394097:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880331.394099:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880331.394101:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 02000000:00000001:0.0:1788880331.394105:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880331.394107:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880331.394108:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d838cab4. 00000100:00000001:0.0:1788880331.394112:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880331.394113:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880331.394115:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880331.394122:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880331.394124:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880331.394126:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880331.394128:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880331.454405:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880331.454424:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880331.454433:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880331.454435:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880331.454506:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880331.454511:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880331.454514:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880332.478354:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880332.478360:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880332.478366:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880332.478367:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880332.478450:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880332.478454:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880332.478457:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880333.502349:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880333.502357:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880333.502359:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880333.502362:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880333.502367:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880333.502370:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880333.502388:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880333.502390:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880333.502395:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880333.502396:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880333.502459:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880333.502463:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880333.502466:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880334.526369:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880334.526377:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880334.526383:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880334.526383:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880334.526465:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880334.526472:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880334.526477:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880335.551009:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880335.551020:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880335.551027:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880335.551030:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880335.551033:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880335.551037:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880335.551040:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880335.551042:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880335.551044:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880335.551046:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880335.551141:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880335.551147:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880335.551150:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880336.510379:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305967567, at 4305967616 00000100:00000040:0.0:1788880336.510390:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880336.510393:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305967567, at 4305967616 00000100:00000010:1.0:1788880336.510396:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000010:2.0:1788880336.510409:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000001:0.0:1788880336.510410:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880336.510423:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d4e901df. 00000020:00000040:0.0:1788880336.510432:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880336.510441:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880336.510447:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880336.510455:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880336.510467:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000005d090de9. 00000100:00000040:3.0:1788880336.510472:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305967567, at 4305967616 00000100:00000001:0.0:1788880336.510477:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880336.510484:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000040:0.0:1788880336.510495:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880336.510506:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880336.510508:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880336.510513:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d4e901df] to pc [ptlrpcd_00_03+3] req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880336.510570:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880336.510573:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880336.510577:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880336.510580:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c2a263b1. 00000020:00000040:0.0:1788880336.510583:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880336.510585:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880336.510587:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880336.510588:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880336.510592:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000079c96644. 00000100:00000001:0.0:1788880336.510595:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880336.510599:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880336.510607:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880336.510608:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880336.510613:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c2a263b1] to pc [ptlrpcd_00_00+0] req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880336.510631:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788880336.510639:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:1.0:1788880336.510642:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880336.510648:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880336.510652:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880336.510654:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880336.510654:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880336.510657:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880336.510661:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880336.510667:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880336.510669:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880336.510672:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:0.0:1788880336.510675:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000b4a4221. 00000100:00000001:1.0:1788880336.510676:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880336.510682:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880336.510687:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:0.0:1788880336.510689:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880336.510690:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880336.510697:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880336.510700:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880336.510704:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880336.510707:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880336.510707:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880336.510712:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000022b14507. 00000100:00000001:0.0:1788880336.510723:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880336.510742:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880336.510770:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880336.510771:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880336.510779:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880336.510779:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880336.510779:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880336.510780:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880336.510786:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880336.510790:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880336.510792:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880336.510792:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880336.510801:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880336.510801:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880336.510805:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000b4a4221] to pc [ptlrpcd_00_01+1] req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880336.510809:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880336.510820:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780649600:192.168.204.111@tcp:400: 02000000:00000001:2.0:1788880336.510823:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880336.510842:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780649728:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880336.510848:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880336.510853:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880336.510855:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880336.510860:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880336.510861:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880336.510862:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880336.510871:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880336.510872:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:2.0:1788880336.510873:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:1.0:1788880336.510875:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880336.510876:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880336.510884:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11305) 02000000:00000001:1.0:1788880336.510886:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880336.510886:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:2.0:1788880336.510894:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ace1117b. 00000100:00000001:3.0:1788880336.510894:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880336.510900:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880336.510902:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880336.510914:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009e1ad299. 00000100:00000040:0.0:1788880336.510916:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305967567, at 4305967616 00000400:00000010:2.0:1788880336.510921:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000096e78f73. 02000000:00000001:1.0:1788880336.510922:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880336.510929:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000400:00000010:2.0:1788880336.510935:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000032b7316b. 00000400:00000010:1.0:1788880336.510937:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000100:00000001:3.0:1788880336.510944:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880336.510948:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880336.510950:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780649728, portal 10 00000400:00000010:1.0:1788880336.510951:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 02000000:00000001:3.0:1788880336.510953:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880336.510957:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:3.0:1788880336.510958:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788880336.510960:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780649600, portal 25 00000100:00000001:2.0:1788880336.510963:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364992 : -126822860186624 : ffff8ca7c0781c00) 00000100:00000001:1.0:1788880336.510970:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:3.0:1788880336.510972:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880336.510974:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849369472 : -126822860182144 : ffff8ca7c0782d80) 00000100:00000040:2.0:1788880336.510982:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880336.510983:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780649856:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880336.510987:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880352 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880336.511006:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880336.511016:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880336.511023:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880336.511026:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880336.511028:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880336.511037:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:3.0:1788880336.511037:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000783b292f. 00000100:00000001:2.0:1788880336.511039:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880336.511047:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880336.511048:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000061cfcb81. 00000100:00000040:2.0:1788880336.511052:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000200:1.0:1788880336.511059:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780649600, offset 0 00000400:00000010:2.0:1788880336.511059:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c0c07443. 00000400:00000010:3.0:1788880336.511062:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007e36d068. 00000100:00000200:2.0:1788880336.511063:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780649728, offset 0 00000400:00000200:1.0:1788880336.511072:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880336.511084:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880336.511085:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000400:00000200:1.0:1788880336.511088:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880336.511096:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:3.0:1788880336.511102:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780649856, portal 10 00000400:00000200:1.0:1788880336.511103:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880336.511108:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880336.511111:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849373056 : -126822860178560 : ffff8ca7c0783b80) 00000400:00000200:1.0:1788880336.511112:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880336.511120:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:3.0:1788880336.511123:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880336.511131:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [145:145:256:4294967295] 192.168.204.111@tcp LPNI seq info [145:145:8:4294967295] 00000100:00000001:3.0:1788880336.511154:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880336.511159:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880336.511165:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000097c86a00. 00000100:00000200:3.0:1788880336.511171:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780649856, offset 0 00000400:00000200:1.0:1788880336.511172:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:3.0:1788880336.511187:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880336.511194:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880336.511195:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880336.511198:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880336.511201:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880336.511202:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000010:1.0:1788880336.511203:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000400:00000200:2.0:1788880336.511205:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880336.511208:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [146:146:255:4294967295] 192.168.204.111@tcp LPNI seq info [146:146:7:4294967295] 00000800:00000200:1.0:1788880336.511213:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880336.511218:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880336.511223:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880336.511228:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000010:2.0:1788880336.511228:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000781b5970. 00000400:00000200:3.0:1788880336.511268:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:2.0:1788880336.511269:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880336.511276:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880336.511281:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880336.511287:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880336.511292:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880336.511294:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000781b5970 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880336.511295:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880336.511301:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880336.511305:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880336.511310:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880336.511312:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305972688 00000100:00000040:2.0:1788880336.511317:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 653527 usecs (654 jiffies) 00000400:00000200:3.0:1788880336.511324:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [147:147:254:4294967295] 192.168.204.111@tcp LPNI seq info [147:147:6:4294967295] 00000100:00000001:2.0:1788880336.511326:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305968271 00000100:00000010:1.0:1788880336.511327:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000100:00000001:1.0:1788880336.511338:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:3.0:1788880336.511350:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880336.511352:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305972688 00000100:00000001:2.0:1788880336.511353:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305972688 00000100:00000001:2.0:1788880336.511361:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305968271 00000800:00000200:3.0:1788880336.511361:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880336.511368:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000100:00000010:2.0:1788880336.511370:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000017cf871b. 00000800:00000200:0.0:1788880336.511378:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880336.511379:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000800:00000200:3.0:1788880336.511380:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000010:0.0:1788880336.511387:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000100:00000001:2.0:1788880336.511390:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305968271 00000400:00000200:0.0:1788880336.511395:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880336.511396:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305968271 00000800:00000200:3.0:1788880336.511397:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:2.0:1788880336.511401:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005c6a0f3f. 00000800:00000200:3.0:1788880336.511408:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880336.511412:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880336.511413:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305972688 00000400:00000200:0.0:1788880336.511414:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880336.511416:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880336.511428:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 40075 usecs (41 jiffies) 00000400:00000200:0.0:1788880336.511432:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000061cfcb81 00000100:00000001:2.0:1788880336.511432:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305968271 00000100:00000001:1.0:1788880336.511434:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305968271 00000100:00000001:3.0:1788880336.511435:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305967658 00000400:00000010:0.0:1788880336.511442:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000061cfcb81. 00000100:00000001:3.0:1788880336.511445:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305972688 00000100:00000001:1.0:1788880336.511448:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305972688 00000100:00000010:2.0:1788880336.511452:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000077b36062. 00000100:00000001:0.0:1788880336.511454:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880336.511454:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305967658 00000100:00000001:3.0:1788880336.511465:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305972688 00000100:00000001:2.0:1788880336.511469:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000200:0.0:1788880336.511471:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880352 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880336.511471:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305967658 00000100:00000001:2.0:1788880336.511476:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880336.511479:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305968271 00000100:00000001:3.0:1788880336.511491:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305972688 00000100:00000001:2.0:1788880336.511494:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880336.511498:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880336.511501:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305967658 00000100:00000001:1.0:1788880336.511510:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880336.511510:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880336.511511:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305972688 00000100:00000001:0.0:1788880336.511514:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880336.511526:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880336.511526:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880336.511527:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880352 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880336.511528:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880336.511536:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880336.511540:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880336.511542:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880336.511544:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880336.511547:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880336.511556:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880336.511564:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880336.511567:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880336.511568:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880336.511569:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880336.511573:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880336.511574:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880336.511576:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880336.511579:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880336.511583:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880336.511584:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880336.511589:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880336.511595:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880336.511596:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880336.511600:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000781b5970. 00000400:00000200:0.0:1788880336.511605:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880336.511612:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880336.511615:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c0c07443 00000400:00000010:0.0:1788880336.511617:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c0c07443. 00000100:00000001:0.0:1788880336.511620:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880336.511624:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880336.511649:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880336.511652:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880336.511657:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880336.511659:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880336.511702:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880336.511705:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000400:00000200:0.0:1788880336.511709:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880336.511712:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880336.511715:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000097c86a00 00000400:00000010:0.0:1788880336.511716:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000097c86a00. 00000100:00000001:0.0:1788880336.511719:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880336.511722:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880336.511728:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880336.511730:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880336.511736:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880336.511737:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880336.513563:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880336.513591:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880336.513594:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880336.513597:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880336.513605:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880336.513617:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b04480 00000400:00000200:2.0:1788880336.513625:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x571 [1] + 224 00000800:00000001:2.0:1788880336.513632:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880336.513636:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880336.513638:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880336.513642:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880336.513649:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880336.513655:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880336.513665:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880336.513683:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880336.513693:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880336.513696:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880336.513697:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880336.513699:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880336.513711:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880336.513718:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04500 00000400:00000200:2.0:1788880336.513724:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x56d [1] + 224 00000800:00000001:2.0:1788880336.513729:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880336.513731:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880336.513733:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880336.513736:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880336.513737:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880336.513742:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:1.0:1788880336.513744:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880336.513747:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880336.513749:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000400:00000200:1.0:1788880336.513753:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:1.0:1788880336.513755:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:1.0:1788880336.513758:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880336.513759:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880336.513763:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880336.513766:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880336.513786:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880336.513789:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880336.513806:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880336.513809:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880336.513812:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880336.513814:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880336.513817:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880336.513819:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880336.513822:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880336.513823:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:1.0:1788880336.513827:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=14 v=5 (5 5 0 0) 00000800:00000001:2.0:1788880336.513830:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880336.513834:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880336.513838:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:1.0:1788880336.513842:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=14 v=5 (1 1 0 0) 00000800:00000001:2.0:1788880336.513844:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880336.513850:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880336.513853:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880336.513855:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00010000:00000001:1.0:1788880336.513858:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880336.513862:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880336.513866:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880336.513866:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04580 00000100:00000040:1.0:1788880336.513874:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880336.513876:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x57d [1] + 224 00000100:00000001:3.0:1788880336.513879:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880336.513885:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880336.513891:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1788880336.513893:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880336.513896:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880336.513899:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000096e78f73. 00000100:00000001:1.0:1788880336.513904:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880336.513917:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000032b7316b 00000100:00000001:1.0:1788880336.513920:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000800:00000001:2.0:1788880336.513920:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880336.513922:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000032b7316b. 00000100:00000001:1.0:1788880336.513924:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880336.513929:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880336.513931:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:1.0:1788880336.513936:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880336.513938:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:3.0:1788880336.513945:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880336.513951:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880336.513961:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880336.513991:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780649600:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880336.514011:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880336.514014:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880336.514020:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880336.514022:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880336.514031:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d4e901df x1875776780649600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880336.514069:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880336.514077:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880336.514086:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880336.514093:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880336.514095:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880336.514097:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:1.0:1788880336.514101:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880336.514101:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880336.514108:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009e1ad299. 02000000:00000001:3.0:1788880336.514108:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880336.514112:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880336.514119:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:3.0:1788880336.514121:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=154 v=5 (5 0 0 0) 00000020:00000001:1.0:1788880336.514123:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000800:00000001:2.0:1788880336.514126:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000040:1.0:1788880336.514129:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00001000:3.0:1788880336.514131:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=154 v=5 (1 0 0 0) 00000020:00000001:1.0:1788880336.514134:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000800:00000001:2.0:1788880336.514135:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880336.514137:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000005d090de9. 02000000:00000001:1.0:1788880336.514145:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880336.514147:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000001:1.0:1788880336.514150:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880336.514151:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880336.514155:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d4e901df. 00000100:00000001:0.0:1788880336.514160:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880336.514162:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880336.514165:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880336.514166:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880336.514170:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000001:0.0:1788880336.514172:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880336.514172:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880336.514177:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880336.514178:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880336.514182:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007e36d068. 00000100:00000001:1.0:1788880336.514184:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880336.514185:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000200:0.0:1788880336.514188:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000100:00000040:3.0:1788880336.514189:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000010:0.0:1788880336.514195:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880336.514196:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880336.514197:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880336.514202:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880336.514203:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880336.514205:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880336.514211:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880336.514214:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880336.514215:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880336.514275:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880336.514278:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880336.514280:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880336.514283:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880336.514292:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880336.514294:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880336.514306:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1788880336.514306:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780649728:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880336.514312:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880336.514314:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880336.514316:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c2a263b1 x1875776780649728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880336.514324:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880336.514327:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880336.514328:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:3.0:1788880336.514330:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880336.514331:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000010:3.0:1788880336.514334:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ace1117b. 00000100:00000001:0.0:1788880336.514335:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880336.514342:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:0.0:1788880336.514345:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880336.514345:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880336.514348:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880336.514352:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00001000:0.0:1788880336.514353:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=14 v=5 (5 5 0 0) 02000000:00000010:3.0:1788880336.514354:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000079c96644. 00000100:00001000:0.0:1788880336.514360:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=138 v=5 (1 0 0 0) 02000000:00000001:3.0:1788880336.514361:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880336.514364:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880336.514366:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000010:3.0:1788880336.514367:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c2a263b1. 00000100:00000001:0.0:1788880336.514368:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880336.514372:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880336.514373:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880336.514376:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00010000:00080000:0.0:1788880336.514378:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880336.514386:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880336.514392:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880336.514396:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880336.514396:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880336.514399:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880336.514400:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880336.514402:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880336.514404:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880336.514405:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880336.514409:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880336.514411:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880336.514419:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880336.514421:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880336.514423:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880336.514426:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880336.514434:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780649856:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880336.514439:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880336.514441:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880336.514443:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000b4a4221 x1875776780649856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880352 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880336.514450:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880336.514451:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880336.514458:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000783b292f. 02000000:00000001:0.0:1788880336.514462:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880336.514464:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880336.514465:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880336.514467:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880336.514469:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000022b14507. 02000000:00000001:0.0:1788880336.514472:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880336.514473:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880336.514474:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000b4a4221. 00000100:00000001:0.0:1788880336.514477:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880336.514478:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880336.514480:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880336.514484:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880336.514485:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880336.514487:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880336.514489:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880336.574377:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880336.574383:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880336.574389:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880336.574390:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880336.574450:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880336.574454:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880336.574457:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880337.598389:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880337.598396:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880337.598401:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880337.598402:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880337.598469:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880337.598477:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880337.598482:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880338.622652:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880338.622661:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880338.622669:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880338.622670:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880338.622729:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880338.622734:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880338.622738:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880339.646415:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880339.646423:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880339.646430:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880339.646431:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880339.646488:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880339.646493:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880339.646496:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880340.670365:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880340.670375:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880340.670383:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880340.670384:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880340.670456:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880340.670460:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880340.670462:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880341.630353:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305972688, at 4305972736 00000100:00000040:2.0:1788880341.630353:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305972688, at 4305972736 00000100:00000040:3.0:1788880341.630353:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305972688, at 4305972736 00000100:00000010:2.0:1788880341.630371:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005c6a0f3f. 00000100:00000010:0.0:1788880341.630381:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000010:3.0:1788880341.630383:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000077b36062. 00000100:00000040:1.0:1788880341.630443:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880341.630448:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880341.630456:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 00000020:00000040:1.0:1788880341.630460:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880341.630465:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880341.630468:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880341.630472:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880341.630480:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:1.0:1788880341.630487:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880341.630499:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880341.630508:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880341.630510:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880341.630526:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_00_02+2] req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880341.630546:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880341.630549:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880341.630552:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880341.630555:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c27d4622. 00000020:00000040:1.0:1788880341.630557:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880341.630560:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880341.630561:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880341.630563:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880341.630565:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880341.630567:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:0.0:1788880341.630570:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880341.630570:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880341.630572:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880341.630574:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880341.630575:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880341.630578:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880341.630580:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880341.630583:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880341.630585:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880341.630588:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880341.630589:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c27d4622] to pc [ptlrpcd_00_03+3] req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880341.630590:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880341.630594:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880341.630596:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880341.630599:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880341.630599:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880341.630601:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788880341.630602:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780649984:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880341.630605:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880341.630607:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000010:1.0:1788880341.630607:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b5bd1bf6. 00000100:00000001:3.0:1788880341.630609:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:1.0:1788880341.630610:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880341.630612:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880341.630612:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880341.630614:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880341.630614:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880341.630616:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880341.630616:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880341.630617:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:1.0:1788880341.630619:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880341.630622:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 02000000:00000010:0.0:1788880341.630623:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008f0935d3. 00000100:00000040:3.0:1788880341.630623:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880341.630626:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880341.630627:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880341.630633:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:0.0:1788880341.630638:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008e3802a2. 00000400:00000010:0.0:1788880341.630645:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a09430ba. 00000100:00000001:3.0:1788880341.630650:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880341.630653:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880341.630655:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880341.630657:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880341.630658:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880341.630660:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880341.630664:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b5bd1bf6] to pc [ptlrpcd_00_00+0] req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880341.630664:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880341.630665:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780649984, portal 25 00000100:00000001:0.0:1788880341.630669:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00100000:3.0:1788880341.630670:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780650112:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880341.630671:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 00000100:00000001:3.0:1788880341.630678:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:0.0:1788880341.630679:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880357 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880341.630687:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880341.630688:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880341.630689:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880341.630691:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880341.630691:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880341.630693:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000100:00000040:1.0:1788880341.630693:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:0.0:1788880341.630696:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780649984, offset 0 02000000:00000001:3.0:1788880341.630696:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880341.630697:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:0.0:1788880341.630703:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880341.630703:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11310) 02000000:00000010:3.0:1788880341.630703:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009b8d16de. 02000000:00000001:3.0:1788880341.630708:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880341.630712:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ba5e1826. 00000400:00000010:3.0:1788880341.630716:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000400:00000200:0.0:1788880341.630720:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:1.0:1788880341.630720:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305972688, at 4305972736 00000100:00000200:3.0:1788880341.630720:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780650112, portal 10 00000100:00000010:1.0:1788880341.630725:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000017cf871b. 00000400:00000200:0.0:1788880341.630726:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880341.630730:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880341.630733:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880341.630733:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938049664 : -126823771501952 : ffff8ca78a268a80) 00000400:00000200:0.0:1788880341.630739:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880341.630741:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880341.630743:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880341.630744:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000400:00000200:0.0:1788880341.630745:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880341.630748:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880341.630750:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [148:148:256:4294967295] 192.168.204.111@tcp LPNI seq info [148:148:8:4294967295] 00000100:00000001:1.0:1788880341.630755:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880341.630758:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880341.630761:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880341.630762:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880341.630764:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880341.630766:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880341.630769:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880341.630771:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880341.630773:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:3.0:1788880341.630775:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ad0499ab. 00000100:00000200:3.0:1788880341.630781:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780650112, offset 0 00000800:00000200:0.0:1788880341.630782:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880341.630782:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880341.630784:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880341.630788:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000010:0.0:1788880341.630789:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000022b14507. 02000000:00000001:1.0:1788880341.630789:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880341.630792:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880341.630793:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880341.630797:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b5bd1bf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780650240:192.168.204.111@tcp:400: 00000800:00000200:0.0:1788880341.630798:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880341.630801:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880341.630802:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1788880341.630806:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880341.630807:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000800:00000200:0.0:1788880341.630822:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:1.0:1788880341.630822:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880341.630823:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788880341.630824:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:0.0:1788880341.630828:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000022b14507 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880341.630828:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000010:1.0:1788880341.630833:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008e3e56e7. 00000400:00000200:3.0:1788880341.630834:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:1.0:1788880341.630838:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880341.630840:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [149:149:255:4294967295] 192.168.204.111@tcp LPNI seq info [149:149:7:4294967295] 00000100:00000001:0.0:1788880341.630842:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880341.630844:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000100:00000001:0.0:1788880341.630849:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305977807 00000400:00000010:1.0:1788880341.630849:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000100:00000200:1.0:1788880341.630853:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780650240, portal 10 00000100:00000010:0.0:1788880341.630854:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006bd48548. 00000100:00000001:1.0:1788880341.630857:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880341.630858:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938059520 : -126823771492096 : ffff8ca78a26b100) 00000100:00000001:0.0:1788880341.630860:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880341.630863:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305977807 00000400:00000200:3.0:1788880341.630864:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:0.0:1788880341.630865:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000455e6647. 00000100:00000040:1.0:1788880341.630865:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880341.630869:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:3.0:1788880341.630877:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880341.630878:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305977807 00000100:00000001:1.0:1788880341.630878:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880341.630883:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880341.630884:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e245be24. 00000400:00000010:1.0:1788880341.630885:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000038c5ac58. 00000800:00000010:3.0:1788880341.630885:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000100:00000001:0.0:1788880341.630886:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880341.630889:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305977807 00000100:00000200:1.0:1788880341.630890:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780650240, offset 0 00000100:00000010:0.0:1788880341.630893:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a682425c. 00000800:00000200:3.0:1788880341.630894:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880341.630896:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788880341.630898:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880341.630901:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1788880341.630905:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880341.630906:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880341.630906:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788880341.630910:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880341.630911:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880341.630911:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880341.630916:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880341.630916:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880341.630917:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880341.630919:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880341.630921:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:1.0:1788880341.630921:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880341.630922:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 637297 usecs (638 jiffies) 00000100:00000001:0.0:1788880341.630923:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880341.630924:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880341.630926:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880341.630928:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880341.630928:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305973374 00000400:00000200:1.0:1788880341.630929:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [150:150:254:4294967295] 192.168.204.111@tcp LPNI seq info [150:150:6:4294967295] 00000100:00000001:0.0:1788880341.630931:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880341.630934:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305977807 00000100:00000001:3.0:1788880341.630938:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305973374 00000100:00000001:3.0:1788880341.630943:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305977807 00000400:00000200:1.0:1788880341.630945:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880341.630948:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305973374 00000100:00000001:3.0:1788880341.630952:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305977807 00000800:00000200:1.0:1788880341.630953:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880341.630957:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000800:00000200:1.0:1788880341.630963:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880341.630963:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305973374 00000100:00000001:3.0:1788880341.630968:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305977807 00000800:00000200:1.0:1788880341.630972:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880341.630973:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880341.630977:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880341.630977:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880341.630981:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880341.630983:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880341.630983:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880341.630984:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000022b14507. 00000100:00000001:3.0:1788880341.630987:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880341.630988:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 39082 usecs (40 jiffies) 00000400:00000200:0.0:1788880341.630989:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880341.630992:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305972776 00000100:00000001:3.0:1788880341.630994:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880341.630995:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880341.630995:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305977807 00000100:00000001:1.0:1788880341.630998:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305972776 00000100:00000001:3.0:1788880341.630998:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880341.631001:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000100:00000001:3.0:1788880341.631001:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788880341.631005:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:3.0:1788880341.631005:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880341.631007:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305977807 00000100:00000001:1.0:1788880341.631009:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305972776 00000100:00000001:3.0:1788880341.631009:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880341.631011:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880341.631013:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305977807 00000100:00000001:3.0:1788880341.631014:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880341.631016:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305972776 00000100:00000200:0.0:1788880341.631017:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880357 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880341.631019:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305977807 00000100:00000001:1.0:1788880341.631022:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880341.631024:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880341.631025:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880341.631026:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880341.631028:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880341.631029:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880357 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880341.631029:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880341.631033:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880341.631039:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880341.631041:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880341.631053:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880341.631055:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000400:00000200:0.0:1788880341.631058:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880341.631061:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880341.631069:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ad0499ab 00000400:00000010:0.0:1788880341.631071:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ad0499ab. 00000100:00000001:0.0:1788880341.631073:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880341.631076:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880341.631083:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880341.631085:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880341.631091:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880341.631093:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880341.631147:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880341.631149:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:0.0:1788880341.631152:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880341.631156:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880341.631158:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000038c5ac58 00000400:00000010:0.0:1788880341.631159:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000038c5ac58. 00000100:00000001:0.0:1788880341.631162:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880341.631164:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880341.631171:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880341.631173:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880341.631179:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880341.631180:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880341.632599:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880341.632609:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880341.632613:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880341.632614:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880341.632621:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880341.632632:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b04600 00000400:00000200:0.0:1788880341.632640:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x585 [1] + 224 00000800:00000001:0.0:1788880341.632647:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880341.632662:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880341.632665:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880341.632669:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880341.632674:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880341.632679:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880341.632701:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880341.632723:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880341.632731:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880341.632734:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880341.633100:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880341.633104:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880341.633110:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880341.633112:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880341.633114:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008e3802a2. 00000400:00000200:1.0:1788880341.633118:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a09430ba 00000400:00000010:1.0:1788880341.633120:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a09430ba. 00000100:00000001:1.0:1788880341.633123:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880341.633128:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880341.633137:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880341.633146:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880341.633148:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880341.633150:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880341.633153:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880341.633155:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880341.633158:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880341.633161:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880341.633166:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=19 v=5 (5 5 0 0) 00000100:00001000:1.0:1788880341.633170:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=19 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880341.633173:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880341.633174:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880341.633176:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880341.633184:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880341.633187:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880341.633191:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880341.633198:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880341.633201:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880341.633202:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880341.633205:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880341.633212:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780649984:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880341.633217:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880341.633219:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880341.633221:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780649984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880341.633227:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880341.633229:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880341.633255:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008f0935d3. 02000000:00000001:1.0:1788880341.633260:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880341.633262:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880341.633263:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880341.633265:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880341.633267:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:1.0:1788880341.633271:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880341.633272:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880341.633273:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000001:1.0:1788880341.633281:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880341.633283:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880341.633285:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880341.633290:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880341.633292:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880341.633293:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880341.633295:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880341.634577:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880341.634588:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880341.634593:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880341.634596:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880341.634603:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880341.634614:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04700 00000400:00000200:2.0:1788880341.634622:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x595 [1] + 224 00000800:00000001:2.0:1788880341.634633:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880341.634638:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880341.634641:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880341.634645:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880341.634649:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880341.634654:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880341.634665:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880341.634739:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880341.634742:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880341.634743:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880341.634751:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880341.634757:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880341.634758:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880341.634761:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:2.0:1788880341.634762:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000010:1.0:1788880341.634764:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000400:00000200:1.0:1788880341.634768:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:1.0:1788880341.634771:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000800:00000001:2.0:1788880341.634774:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880341.634776:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880341.634782:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880341.634785:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880341.634797:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04680 00000100:00000200:1.0:1788880341.634798:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880341.634819:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x58d [1] + 224 00000100:00000001:1.0:1788880341.634824:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880341.634827:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880341.634827:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880341.634831:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880341.634836:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880341.634838:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880341.634838:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880341.634841:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880341.634842:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1788880341.634845:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880341.634849:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880341.634853:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:1.0:1788880341.634856:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=19 v=5 (5 5 0 0) 00000100:00000200:2.0:1788880341.634859:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880341.634862:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=143 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880341.634868:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880341.634870:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880341.634873:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:2.0:1788880341.634878:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00080000:1.0:1788880341.634881:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880341.634903:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880341.634906:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880341.634908:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880341.634909:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880341.634911:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880341.634914:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880341.634916:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880341.634922:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880341.634923:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880341.634927:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000800:00000001:2.0:1788880341.634927:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880341.634935:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880341.634940:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880341.634948:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b5bd1bf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780650240:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880341.634954:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880341.634955:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880341.634957:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880341.634959:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880341.634962:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b5bd1bf6 x1875776780650240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:3.0:1788880341.634965:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880341.634968:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880341.634971:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ba5e1826. 00000100:00000001:1.0:1788880341.634975:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000200:3.0:1788880341.634976:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 02000000:00000001:1.0:1788880341.634979:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000010:3.0:1788880341.634980:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 02000000:00000010:1.0:1788880341.634983:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008e3e56e7. 00000100:00000001:3.0:1788880341.634985:0:201288:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:1.0:1788880341.634989:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:3.0:1788880341.634992:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788880341.634994:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880341.634996:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880341.635001:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880341.635004:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:1.0:1788880341.635009:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000200:3.0:1788880341.635010:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880341.635012:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880341.635015:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b5bd1bf6. 00000100:00000001:1.0:1788880341.635021:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880341.635024:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880341.635024:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880341.635027:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880341.635028:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880341.635030:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880341.635033:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880341.635034:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880341.635035:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880341.635038:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880341.635040:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880341.635042:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880341.635043:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880341.635049:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880341.635054:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=9 v=5 (5 5 0 0) 00000100:00001000:3.0:1788880341.635057:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=9 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880341.635065:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880341.635066:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880341.635068:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880341.635070:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880341.635073:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880341.635074:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880341.635076:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880341.635077:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880341.635080:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880341.635087:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880341.635089:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880341.635090:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880341.635092:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880341.635099:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780650112:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880341.635103:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880341.635104:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880341.635106:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c27d4622 x1875776780650112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880357 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880341.635112:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880341.635114:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880341.635115:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009b8d16de. 02000000:00000001:3.0:1788880341.635118:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880341.635119:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880341.635120:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880341.635122:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880341.635123:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 02000000:00000001:3.0:1788880341.635126:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880341.635127:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880341.635128:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c27d4622. 00000100:00000001:3.0:1788880341.635130:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880341.635131:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880341.635136:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880341.635140:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880341.635141:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880341.635143:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880341.635145:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880341.694347:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880341.694355:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880341.694362:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880341.694364:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880341.694423:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880341.694427:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880341.694430:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880342.718414:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880342.718425:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880342.718434:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880342.718435:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880342.718538:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880342.718541:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880342.718543:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880343.742369:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880343.742405:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880343.742408:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880343.742411:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880343.742415:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880343.742417:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880343.742419:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880343.742421:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880343.742427:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880343.742428:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880343.742488:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880343.742491:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880343.742494:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880344.766357:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880344.766363:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880344.766369:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880344.766371:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880344.766427:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880344.766431:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880344.766434:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880345.790344:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880345.790352:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880345.790358:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880345.790361:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880345.790370:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880345.790373:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880345.790376:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880345.790378:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880345.790380:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880345.790382:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880345.790435:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880345.790438:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880345.790440:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880346.750467:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305977807, at 4305977856 00000100:00000040:3.0:1788880346.750476:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305977807, at 4305977856 00000100:00000040:2.0:1788880346.750478:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305977807, at 4305977856 00000100:00000010:0.0:1788880346.750490:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006bd48548. 00000100:00000010:2.0:1788880346.750493:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e245be24. 00000100:00000010:3.0:1788880346.750497:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a682425c. 00000100:00000040:0.0:1788880346.750515:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880346.750526:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880346.750536:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000b4a4221. 00000020:00000040:0.0:1788880346.750541:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880346.750547:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880346.750549:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880346.750555:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880346.750562:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000022b14507. 00000100:00000001:0.0:1788880346.750569:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880346.750580:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880346.750591:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880346.750593:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880346.750593:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305977807, at 4305977856 00000100:00000040:0.0:1788880346.750602:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000b4a4221] to pc [ptlrpcd_00_01+1] req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880346.750607:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000455e6647. 00000100:00000001:0.0:1788880346.750637:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880346.750641:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880346.750646:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880346.750658:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880346.750666:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880346.750670:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880346.750675:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880346.750681:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880346.750682:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880346.750694:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880346.750697:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880346.750702:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880346.750706:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880346.750709:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880346.750726:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780650368:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880346.750747:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880346.750755:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880346.750759:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880346.750762:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880346.750768:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000c21a3ff8. 02000000:00000001:1.0:1788880346.750770:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880346.750778:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000400:00000010:1.0:1788880346.750790:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000100:00000010:0.0:1788880346.750791:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000363de9c9. 00000020:00000040:0.0:1788880346.750800:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880346.750803:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880346.750805:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000200:1.0:1788880346.750806:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780650368, portal 25 02000000:00000001:0.0:1788880346.750807:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880346.750812:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880346.750815:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849373056 : -126822860178560 : ffff8ca7c0783b80) 02000000:00000010:0.0:1788880346.750816:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000df3afd78. 00000100:00000001:0.0:1788880346.750825:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880346.750828:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880362 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880346.750837:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880346.750851:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880346.750861:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:1.0:1788880346.750861:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880346.750867:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880346.750868:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000974f52f4. 00000100:00000200:1.0:1788880346.750877:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780650368, offset 0 00000100:00000040:0.0:1788880346.750882:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000363de9c9] to pc [ptlrpcd_00_02+2] req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880346.750894:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880346.750923:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880346.750931:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880346.750937:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880346.750941:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880346.750945:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:0.0:1788880346.750948:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:1.0:1788880346.750953:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880346.750955:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880346.750963:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000fd20c80. 00000400:00000200:1.0:1788880346.750963:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [151:151:256:4294967295] 192.168.204.111@tcp LPNI seq info [151:151:8:4294967295] 00000020:00000040:0.0:1788880346.750973:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880346.750976:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880346.750978:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880346.750982:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00100000:3.0:1788880346.750984:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 02000000:00000001:0.0:1788880346.750988:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.750990:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880346.750997:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000010:0.0:1788880346.750998:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000079c96644. 00000100:00000001:3.0:1788880346.751005:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880346.751009:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.751010:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880346.751016:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880346.751019:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880346.751020:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880346.751024:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:1.0:1788880346.751028:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880346.751035:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:1.0:1788880346.751040:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000800:00000200:1.0:1788880346.751065:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880346.751085:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880346.751086:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880346.751087:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880346.751093:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880346.751095:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880346.751097:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 02000000:00000001:3.0:1788880346.751100:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880346.751105:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880346.751109:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000fd20c80] to pc [ptlrpcd_00_03+3] req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880346.751111:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880346.751117:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880346.751123:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780650496:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880346.751134:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305982927 00000100:00000001:3.0:1788880346.751138:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000010:1.0:1788880346.751146:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 02000000:00000001:3.0:1788880346.751153:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880346.751154:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880346.751156:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 02000000:00000001:3.0:1788880346.751156:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880346.751161:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880346.751163:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880346.751166:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305982927 02000000:00000010:3.0:1788880346.751168:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000030240e3c. 02000000:00000001:3.0:1788880346.751175:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880346.751176:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000400:00000010:3.0:1788880346.751187:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ba5e1826. 00000100:00000040:0.0:1788880346.751189:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880346.751189:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000010:3.0:1788880346.751196:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000100:00000001:1.0:1788880346.751198:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305982927 00000100:00000040:0.0:1788880346.751201:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11315) 00000100:00000200:3.0:1788880346.751203:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780650496, portal 10 00000100:00000010:1.0:1788880346.751204:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000100:00000001:3.0:1788880346.751209:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880346.751211:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880346.751212:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849365888 : -126822860185728 : ffff8ca7c0781f80) 00000100:00000040:3.0:1788880346.751223:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880346.751225:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305982927 00000100:00000010:1.0:1788880346.751231:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000100:00000001:1.0:1788880346.751258:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880346.751265:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880346.751265:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880346.751271:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880346.751272:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880346.751277:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880346.751277:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ad0499ab. 00000100:00000001:1.0:1788880346.751281:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880346.751287:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880346.751290:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000200:3.0:1788880346.751290:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780650496, offset 0 00000100:00000001:1.0:1788880346.751292:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880346.751295:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880346.751296:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880346.751299:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880346.751300:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880346.751309:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880346.751313:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880346.751317:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880346.751320:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880346.751323:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880346.751327:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [152:152:255:4294967295] 192.168.204.111@tcp LPNI seq info [152:152:7:4294967295] 00000400:00000200:3.0:1788880346.751336:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880346.751341:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880346.751345:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000800:00000200:3.0:1788880346.751350:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880346.751357:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880346.751358:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:3.0:1788880346.751363:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000800:00000010:0.0:1788880346.751366:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000100:00000001:3.0:1788880346.751371:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880346.751376:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:3.0:1788880346.751378:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 621888 usecs (622 jiffies) 00000100:00000001:3.0:1788880346.751384:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305978479 00000400:00000200:0.0:1788880346.751388:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880346.751397:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305982927 00000400:00000200:0.0:1788880346.751398:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000974f52f4 00000400:00000010:0.0:1788880346.751408:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000974f52f4. 00000100:00000001:3.0:1788880346.751408:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305978479 00000100:00000001:3.0:1788880346.751417:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305982927 00000100:00000001:0.0:1788880346.751421:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880346.751421:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305978479 00000100:00000001:3.0:1788880346.751427:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305982927 00000100:00000200:0.0:1788880346.751432:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880362 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880346.751433:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305978479 00000100:00000001:3.0:1788880346.751439:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305982927 00000100:00000001:3.0:1788880346.751446:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880346.751451:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.751459:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.751465:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880346.751472:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880346.751475:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880346.751476:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880346.751479:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880346.751484:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880346.751485:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880362 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880346.751489:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880346.751498:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880346.751505:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880346.751510:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880346.751521:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880346.751524:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880346.751528:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880346.751530:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880346.751532:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880346.751535:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780650624:192.168.204.111@tcp:400: 00000800:00000200:0.0:1788880346.751537:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880346.751545:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000010:0.0:1788880346.751547:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 02000000:00000001:3.0:1788880346.751551:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880346.751555:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:3.0:1788880346.751555:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880346.751563:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880346.751569:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000010:3.0:1788880346.751571:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000aea170a1. 00000400:00000200:0.0:1788880346.751575:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ad0499ab 02000000:00000001:3.0:1788880346.751577:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880346.751581:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ad0499ab. 00000400:00000010:3.0:1788880346.751585:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c948c598. 00000100:00000001:0.0:1788880346.751587:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880346.751593:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880346.751599:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000200:3.0:1788880346.751605:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780650624, portal 10 00000100:00000001:0.0:1788880346.751608:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880346.751610:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880346.751613:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849357824 : -126822860193792 : ffff8ca7c0780000) 00000100:00000040:0.0:1788880346.751615:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880346.751625:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880346.751630:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880346.751640:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880346.751649:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880346.751654:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880346.751656:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000097c86a00. 00000100:00000200:3.0:1788880346.751659:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780650624, offset 0 00000400:00000200:3.0:1788880346.751665:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880346.751670:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880346.751674:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880346.751676:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880346.751678:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880346.751680:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880346.751683:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [153:153:256:4294967295] 192.168.204.111@tcp LPNI seq info [153:153:8:4294967295] 00000400:00000200:3.0:1788880346.751690:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880346.751695:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880346.751699:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000800:00000200:3.0:1788880346.751703:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880346.751708:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880346.751726:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880346.751752:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880346.751756:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 38166 usecs (39 jiffies) 00000100:00000001:3.0:1788880346.751759:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305977896 00000100:00000001:3.0:1788880346.751761:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305982927 00000100:00000001:3.0:1788880346.751763:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305977896 00000100:00000001:3.0:1788880346.751765:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305982927 00000100:00000001:2.0:1788880346.751771:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880346.751778:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305977896 00000100:00000001:2.0:1788880346.751780:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.751785:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305982927 00000100:00000001:3.0:1788880346.751789:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305977896 00000100:00000001:3.0:1788880346.751791:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305982927 00000100:00000001:3.0:1788880346.751793:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880346.751796:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.751798:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.751800:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880346.751804:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880346.751805:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880346.751807:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880346.751808:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880346.751809:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.751811:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880346.751933:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880346.751938:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000400:00000200:1.0:1788880346.751943:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880346.751948:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880346.751953:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000097c86a00 00000400:00000010:1.0:1788880346.751955:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000097c86a00. 00000100:00000001:1.0:1788880346.751959:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880346.751963:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880346.751973:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880346.751975:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880346.751988:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880346.751990:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880346.754280:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880346.754297:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880346.754301:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880346.754304:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880346.754313:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880346.754329:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b04780 00000400:00000200:2.0:1788880346.754337:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x59d [1] + 224 00000800:00000001:2.0:1788880346.754353:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880346.754356:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880346.754357:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880346.754360:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880346.754365:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880346.754370:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880346.754378:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880346.754399:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880346.754405:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880346.754407:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880346.754408:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880346.754410:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880346.754412:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880346.754416:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04800 00000400:00000200:2.0:1788880346.754421:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x5a5 [1] + 224 00000800:00000001:2.0:1788880346.754423:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880346.754428:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880346.754429:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880346.754431:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880346.754433:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880346.754435:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880346.754440:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880346.754450:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880346.754453:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880346.754454:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880346.754481:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880346.754486:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880346.754493:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880346.754495:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880346.754497:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ba5e1826. 00000400:00000200:3.0:1788880346.754502:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000400:00000010:3.0:1788880346.754504:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000100:00000001:3.0:1788880346.754508:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880346.754513:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:3.0:1788880346.754514:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880346.754517:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880346.754522:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880346.754524:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880346.754527:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000100:00000200:3.0:1788880346.754529:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880346.754531:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:1.0:1788880346.754534:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:1.0:1788880346.754539:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880346.754542:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:1.0:1788880346.754545:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880346.754545:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880346.754551:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000200:1.0:1788880346.754555:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880346.754555:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880346.754559:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880346.754563:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880346.754563:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880346.754566:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880346.754568:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880346.754569:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880346.754571:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880346.754572:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880346.754573:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880346.754575:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880346.754578:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=24 v=5 (5 5 0 0) 00000100:00001000:3.0:1788880346.754579:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=14 v=5 (5 5 0 0) 00000100:00001000:1.0:1788880346.754582:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=24 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880346.754584:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880346.754585:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880346.754590:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880346.754593:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880346.754594:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880346.754597:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880346.754601:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880346.754603:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880346.754604:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880346.754606:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880346.754617:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780650368:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880346.754621:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880346.754623:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880346.754625:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000b4a4221 x1875776780650368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880346.754629:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880346.754631:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880346.754633:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000c21a3ff8. 02000000:00000001:1.0:1788880346.754636:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880346.754637:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880346.754639:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880346.754641:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880346.754642:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000022b14507. 02000000:00000001:1.0:1788880346.754645:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880346.754647:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880346.754648:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000b4a4221. 00000100:00000001:1.0:1788880346.754650:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880346.754652:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880346.754653:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880346.754668:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880346.754670:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880346.754672:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880346.754673:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880346.754703:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=14 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880346.754707:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880346.754708:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880346.754720:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880346.754723:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.754726:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880346.754728:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880346.754730:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880346.754731:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880346.754736:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880346.754743:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880346.754746:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880346.754747:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880346.754750:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880346.754753:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:3.0:1788880346.754760:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780650496:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880346.754764:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.754766:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880346.754767:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000363de9c9 x1875776780650496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880346.754773:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880346.754774:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880346.754776:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000030240e3c. 02000000:00000001:3.0:1788880346.754778:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880346.754779:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880346.754780:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880346.754782:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880346.754783:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000df3afd78. 02000000:00000001:3.0:1788880346.754786:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880346.754787:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880346.754788:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000363de9c9. 00000100:00000001:3.0:1788880346.754791:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880346.754797:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880346.754829:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880346.754839:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880346.754843:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880346.754843:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.754848:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880346.754851:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880346.754854:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880346.754855:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880346.754858:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880346.754867:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04880 00000400:00000200:0.0:1788880346.754872:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x5ad [1] + 224 00000800:00000001:0.0:1788880346.754877:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880346.754882:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880346.754884:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880346.754887:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880346.754890:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880346.754894:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880346.754901:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880346.754918:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880346.754924:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880346.754926:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880346.755023:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880346.755025:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880346.755028:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880346.755029:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880346.755030:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c948c598. 00000400:00000200:3.0:1788880346.755033:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880346.755035:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880346.755038:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880346.755041:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880346.755047:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880346.755053:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880346.755054:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880346.755056:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880346.755057:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880346.755058:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880346.755060:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880346.755061:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880346.755065:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=24 v=5 (5 5 0 0) 00000100:00001000:3.0:1788880346.755067:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=148 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880346.755069:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880346.755070:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880346.755072:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880346.755074:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880346.755080:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.755082:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880346.755083:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880346.755084:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880346.755091:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880346.755094:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880346.755099:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880346.755101:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880346.755102:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880346.755104:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880346.755110:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780650624:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880346.755113:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880346.755115:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880346.755117:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000fd20c80 x1875776780650624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880362 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880346.755122:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880346.755124:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880346.755125:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000aea170a1. 02000000:00000001:3.0:1788880346.755127:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880346.755129:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880346.755130:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880346.755131:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880346.755132:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000079c96644. 02000000:00000001:3.0:1788880346.755135:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880346.755136:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880346.755136:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000fd20c80. 00000100:00000001:3.0:1788880346.755139:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880346.755140:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880346.755142:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880346.755145:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880346.755146:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880346.755148:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880346.755149:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880346.814359:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880346.814365:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880346.814369:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880346.814370:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880346.814535:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880346.814538:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880346.814558:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880347.838350:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880347.838356:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880347.838364:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880347.838365:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880347.838427:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880347.838432:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880347.838461:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880348.862498:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880348.862507:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880348.862514:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880348.862538:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880348.862595:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880348.862600:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880348.862603:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880349.886343:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880349.886350:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880349.886355:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880349.886356:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880349.886442:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880349.886452:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880349.886457:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880350.910453:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880350.910460:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880350.910466:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880350.910467:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880350.910511:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880350.910516:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880350.910519:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788880351.870386:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305982927, at 4305982976 00000100:00000010:3.0:1788880351.870408:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000040:1.0:1788880351.870547:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305982927, at 4305982976 00000100:00000010:1.0:1788880351.870563:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000040:0.0:1788880351.870603:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305982927, at 4305982976 00000100:00000010:0.0:1788880351.870612:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000040:0.0:1788880351.870638:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880351.870644:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880351.870650:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cdd8dec9. 00000020:00000040:0.0:1788880351.870655:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880351.870659:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880351.870662:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880351.870667:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880351.870674:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000005d090de9. 00000100:00000001:0.0:1788880351.870682:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880351.870691:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880351.870703:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880351.870721:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880351.870729:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cdd8dec9] to pc [ptlrpcd_00_00+0] req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880351.870761:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880351.870764:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880351.870768:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880351.870773:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000fd20c80. 00000020:00000040:0.0:1788880351.870776:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880351.870780:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880351.870782:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880351.870784:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880351.870788:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000079c96644. 00000100:00000001:1.0:1788880351.870788:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880351.870794:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880351.870794:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880351.870799:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880351.870801:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880351.870810:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880351.870812:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880351.870819:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000fd20c80] to pc [ptlrpcd_00_01+1] req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880351.870819:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880351.870840:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880351.870840:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880351.870841:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880351.870844:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880351.870845:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880351.870846:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880351.870851:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1788880351.870851:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788880351.870853:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880351.870857:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:0.0:1788880351.870859:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000363de9c9. 00000100:00000040:2.0:1788880351.870859:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880351.870862:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880351.870864:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880351.870869:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880351.870872:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00100000:1.0:1788880351.870872:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cdd8dec9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780650752:192.168.204.111@tcp:400: 02000000:00000001:0.0:1788880351.870875:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880351.870878:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000010:0.0:1788880351.870880:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000df3afd78. 00000100:00000001:2.0:1788880351.870881:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880351.870882:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880351.870884:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880351.870887:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880351.870890:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880351.870891:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880351.870891:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880351.870895:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880351.870897:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880351.870901:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00100000:2.0:1788880351.870905:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780650880:192.168.204.111@tcp:400: 02000000:00000010:1.0:1788880351.870908:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000014e021ca. 00000100:00000001:2.0:1788880351.870912:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880351.870914:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880351.870918:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880351.870922:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880351.870924:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000100:00000001:0.0:1788880351.870925:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788880351.870926:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880351.870930:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880351.870933:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000036ec89fa. 00000100:00000040:0.0:1788880351.870938:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000363de9c9] to pc [ptlrpcd_00_02+2] req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880351.870938:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880351.870944:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000003e3a0df8. 00000400:00000010:1.0:1788880351.870945:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000400:00000010:2.0:1788880351.870952:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009c31f261. 00000100:00000200:1.0:1788880351.870955:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780650752, portal 25 00000100:00000200:2.0:1788880351.870957:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780650880, portal 10 00000100:00000001:1.0:1788880351.870959:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880351.870961:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880351.870963:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849372160 : -126822860179456 : ffff8ca7c0783800) 00000100:00000001:2.0:1788880351.870964:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849357824 : -126822860193792 : ffff8ca7c0780000) 00000100:00000040:1.0:1788880351.870972:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880367 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880351.870976:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880351.870983:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880351.870988:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880351.870998:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880351.870999:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880351.871004:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880351.871005:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880351.871008:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11320) 00000400:00000010:2.0:1788880351.871010:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cc94f399. 00000100:00000040:1.0:1788880351.871011:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880351.871017:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000641230df. 00000100:00000001:0.0:1788880351.871021:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880351.871022:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780650880, offset 0 00000100:00000200:1.0:1788880351.871024:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780650752, offset 0 00000100:00100000:0.0:1788880351.871031:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:0.0:1788880351.871035:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880351.871035:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880351.871050:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880351.871056:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880351.871060:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880351.871063:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880351.871066:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880351.871070:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [154:154:256:4294967295] 192.168.204.111@tcp LPNI seq info [154:154:8:4294967295] 00000400:00000200:2.0:1788880351.871081:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880351.871081:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880351.871086:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880351.871091:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880351.871097:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000800:00000200:2.0:1788880351.871104:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880351.871112:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880351.871116:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880351.871127:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880351.871130:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880351.871133:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880351.871134:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880351.871135:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880351.871141:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880351.871141:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 607190 usecs (608 jiffies) 00000100:00000001:2.0:1788880351.871147:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305983584 00000100:00000040:0.0:1788880351.871149:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788880351.871153:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005c6a0f3f. 00000100:00000001:0.0:1788880351.871161:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880351.871161:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880351.871164:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880351.871168:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880351.871168:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305983584 02000000:00000001:0.0:1788880351.871171:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:2.0:1788880351.871173:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000668fd8a. 02000000:00000001:0.0:1788880351.871175:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880351.871179:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00100000:0.0:1788880351.871181:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780651008:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880351.871184:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305983584 00000100:00000001:0.0:1788880351.871187:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880351.871190:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880351.871192:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880351.871192:0:201286:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, simply extend its deadline from 4305982927 02000000:00000001:0.0:1788880351.871198:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880351.871199:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305983584 02000000:00000010:0.0:1788880351.871202:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003a17641d. 00000100:00000010:2.0:1788880351.871204:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c99cc67. 02000000:00000001:0.0:1788880351.871207:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880351.871210:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000400:00000010:0.0:1788880351.871212:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000844d9b13. 00000100:00000001:2.0:1788880351.871214:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880351.871226:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880351.871229:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880351.871230:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880351.871255:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880351.871257:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880351.871259:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880351.871260:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880351.871262:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880351.871264:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880351.871278:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880351.871283:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000400:00000200:3.0:1788880351.871287:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880351.871293:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880351.871309:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880351.871317:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000400:00000200:3.0:1788880351.871317:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cc94f399 00000400:00000200:1.0:1788880351.871318:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:3.0:1788880351.871321:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cc94f399. 00000400:00000200:1.0:1788880351.871328:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:0.0:1788880351.871329:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780651008, portal 10 00000100:00000001:3.0:1788880351.871330:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:1.0:1788880351.871332:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880351.871335:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880351.871338:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:3.0:1788880351.871338:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880351.871339:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849365888 : -126822860185728 : ffff8ca7c0781f80) 00000400:00000200:1.0:1788880351.871345:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:0.0:1788880351.871350:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880351.871352:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [155:155:255:4294967295] 192.168.204.111@tcp LPNI seq info [155:155:7:4294967295] 00000100:00000001:0.0:1788880351.871366:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880351.871372:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880351.871373:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880351.871377:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d6cef011. 00000100:00000200:0.0:1788880351.871380:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780651008, offset 0 00000800:00000200:1.0:1788880351.871381:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880351.871387:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000400:00000200:0.0:1788880351.871389:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880351.871395:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880351.871398:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880351.871404:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:1.0:1788880351.871404:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880351.871409:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880351.871410:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880351.871414:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880351.871415:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880351.871419:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880351.871421:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305988048 00000400:00000200:0.0:1788880351.871426:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [156:156:254:4294967295] 192.168.204.111@tcp LPNI seq info [156:156:6:4294967295] 00000100:00000001:1.0:1788880351.871428:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305983584 00000100:00000001:1.0:1788880351.871433:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305988048 00000100:00000001:1.0:1788880351.871440:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305983584 00000400:00000200:0.0:1788880351.871442:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880351.871447:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305988048 00000100:00000001:1.0:1788880351.871454:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305983584 00000800:00000200:0.0:1788880351.871458:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880351.871458:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880351.871460:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305988048 00000800:00000010:0.0:1788880351.871464:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000022b14507. 00000100:00000001:1.0:1788880351.871466:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305983584 00000100:00000040:3.0:1788880351.871466:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880351.871473:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880351.871474:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880351.871480:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880351.871484:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880351.871485:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880351.871489:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880351.871490:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000022b14507 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880351.871492:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880351.871497:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880351.871497:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880351.871502:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 37290 usecs (38 jiffies) 00000100:00000001:1.0:1788880351.871502:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880351.871506:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880351.871507:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305983015 00000800:00000200:3.0:1788880351.871509:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880351.871512:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880351.871514:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305988048 00000800:00000010:3.0:1788880351.871514:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000001:1.0:1788880351.871515:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880351.871518:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880351.871519:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305983015 00000400:00000200:3.0:1788880351.871520:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880351.871523:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880351.871526:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305988048 00000400:00000200:3.0:1788880351.871529:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880351.871533:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000641230df 00000400:00000010:3.0:1788880351.871536:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000641230df. 00000100:00000001:0.0:1788880351.871539:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305983015 00000100:00000001:3.0:1788880351.871541:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880351.871544:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305988048 00000100:00000200:3.0:1788880351.871547:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880367 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880351.871549:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305983015 00000100:00000001:0.0:1788880351.871554:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305988048 00000100:00000001:0.0:1788880351.871559:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880351.871563:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880351.871564:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880351.871568:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880351.871568:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880367 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880351.871571:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880351.871582:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880351.871584:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880351.871633:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880351.871636:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000022b14507. 00000400:00000200:3.0:1788880351.871640:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880351.871644:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880351.871647:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d6cef011 00000400:00000010:3.0:1788880351.871648:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d6cef011. 00000100:00000001:3.0:1788880351.871651:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880351.871654:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880351.871661:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880351.871663:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880351.871669:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880351.871671:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880351.873158:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880351.873171:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880351.873175:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880351.873177:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880351.873186:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880351.873198:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b04900 00000400:00000200:2.0:1788880351.873206:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x5b5 [1] + 224 00000800:00000001:2.0:1788880351.873211:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880351.873226:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880351.873229:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880351.873268:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880351.873273:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880351.873278:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880351.873288:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880351.873304:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880351.873310:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880351.873314:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880351.873322:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880351.873327:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880351.873333:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880351.873334:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880351.873336:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000400:00000200:1.0:1788880351.873340:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:1.0:1788880351.873342:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:1.0:1788880351.873345:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880351.873349:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880351.873356:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880351.873362:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880351.873364:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880351.873366:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880351.873368:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880351.873370:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880351.873373:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880351.873376:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880351.873381:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=29 v=5 (5 5 0 0) 00000100:00001000:1.0:1788880351.873385:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=29 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880351.873387:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880351.873388:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880351.873390:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880351.873392:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880351.873394:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880351.873397:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880351.873419:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880351.873422:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880351.873424:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880351.873428:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880351.873436:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cdd8dec9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780650752:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880351.873442:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880351.873444:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880351.873447:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cdd8dec9 x1875776780650752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880351.873453:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880351.873455:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880351.873458:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000014e021ca. 02000000:00000001:1.0:1788880351.873462:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880351.873464:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880351.873466:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880351.873468:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880351.873470:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000005d090de9. 02000000:00000001:1.0:1788880351.873473:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880351.873474:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880351.873476:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cdd8dec9. 00000100:00000001:1.0:1788880351.873479:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880351.873480:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880351.873482:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880351.873488:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880351.873489:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880351.873491:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880351.873492:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880351.873860:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880351.873873:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880351.873878:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880351.873879:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880351.873890:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880351.873905:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04980 00000400:00000200:0.0:1788880351.873914:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x5b9 [1] + 224 00000800:00000001:0.0:1788880351.873920:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880351.873926:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880351.873929:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880351.873933:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880351.873939:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880351.873945:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880351.873956:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880351.873978:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880351.873986:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880351.873989:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880351.874054:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880351.874063:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880351.874069:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880351.874071:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880351.874074:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000003e3a0df8. 00000400:00000200:2.0:1788880351.874078:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009c31f261 00000400:00000010:2.0:1788880351.874080:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009c31f261. 00000100:00000001:2.0:1788880351.874097:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880351.874101:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880351.874109:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880351.874116:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880351.874117:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880351.874120:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880351.874123:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880351.874125:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880351.874127:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880351.874130:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880351.874138:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=19 v=5 (5 5 0 0) 00000100:00001000:2.0:1788880351.874142:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=19 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880351.874145:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880351.874146:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880351.874148:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880351.874152:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880351.874155:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880351.874157:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880351.874159:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880351.874161:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880351.874165:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880351.874171:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880351.874174:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880351.874176:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880351.874178:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880351.874186:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780650880:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880351.874191:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880351.874193:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880351.874202:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000fd20c80 x1875776780650880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880351.874209:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880351.874211:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880351.874214:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000036ec89fa. 02000000:00000001:2.0:1788880351.874217:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880351.874219:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880351.874221:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880351.874224:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880351.874226:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000079c96644. 02000000:00000001:2.0:1788880351.874229:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880351.874230:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880351.874316:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000fd20c80. 00000100:00000001:2.0:1788880351.874321:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880351.874322:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880351.874324:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880351.874332:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880351.874334:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880351.874336:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880351.874338:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880351.874954:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880351.874961:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880351.874964:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880351.874967:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880351.874974:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880351.874984:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04a00 00000400:00000200:3.0:1788880351.874990:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x5c5 [1] + 224 00000800:00000001:3.0:1788880351.874994:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880351.874999:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880351.875001:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880351.875004:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880351.875008:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880351.875014:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880351.875023:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880351.875052:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880351.875058:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880351.875061:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880351.875086:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880351.875089:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880351.875094:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880351.875095:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880351.875104:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000844d9b13. 00000400:00000200:0.0:1788880351.875107:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000400:00000010:0.0:1788880351.875109:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:0.0:1788880351.875113:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880351.875117:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880351.875125:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880351.875130:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880351.875131:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880351.875133:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880351.875136:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880351.875137:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880351.875139:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880351.875141:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880351.875146:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=29 v=5 (5 5 0 0) 00000100:00001000:0.0:1788880351.875148:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=153 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880351.875151:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880351.875152:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880351.875154:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880351.875158:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880351.875165:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880351.875167:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880351.875169:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880351.875170:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880351.875171:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880351.875174:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880351.875180:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880351.875182:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880351.875183:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880351.875186:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880351.875200:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780651008:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880351.875205:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880351.875206:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880351.875209:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000363de9c9 x1875776780651008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880367 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880351.875215:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880351.875217:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880351.875219:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003a17641d. 02000000:00000001:0.0:1788880351.875222:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880351.875223:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880351.875224:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880351.875226:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880351.875228:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000df3afd78. 02000000:00000001:0.0:1788880351.875230:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880351.875250:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880351.875252:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000363de9c9. 00000100:00000001:0.0:1788880351.875256:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880351.875257:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880351.875259:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880351.875264:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880351.875265:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880351.875266:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880351.875268:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880351.934372:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880351.934379:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880351.934384:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880351.934385:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880351.934495:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880351.934499:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880351.934502:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880352.958378:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880352.958401:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880352.958410:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880352.958411:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880352.958481:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880352.958486:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880352.958489:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880353.982354:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880353.982362:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880353.982365:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880353.982368:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880353.982372:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880353.982375:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880353.982377:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880353.982379:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880353.982383:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880353.982384:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880353.982442:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880353.982446:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880353.982448:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880355.006437:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880355.006445:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880355.006451:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880355.006452:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880355.006519:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880355.006523:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880355.006526:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880356.030318:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880356.030325:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880356.030331:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880356.030333:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880356.030336:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880356.030339:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880356.030342:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880356.030344:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880356.030346:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880356.030348:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880356.030406:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880356.030413:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880356.030417:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880356.990420:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305988048, at 4305988096 00000100:00000040:2.0:1788880356.990420:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305988048, at 4305988096 00000100:00000040:1.0:1788880356.990422:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305988048, at 4305988096 00000100:00000040:3.0:1788880356.990444:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305988048, at 4305988096 00000100:00000010:1.0:1788880356.990447:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000668fd8a. 00000100:00000010:2.0:1788880356.990451:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:0.0:1788880356.990453:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005c6a0f3f. 00000100:00000010:3.0:1788880356.990459:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c99cc67. 00000100:00000040:0.0:1788880356.990496:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880356.990500:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880356.990505:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000363de9c9. 00000020:00000040:0.0:1788880356.990509:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880356.990514:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880356.990527:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880356.990532:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880356.990550:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000df3afd78. 00000100:00000001:0.0:1788880356.990556:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880356.990569:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880356.990576:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880356.990578:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880356.990582:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000363de9c9] to pc [ptlrpcd_00_03+3] req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880356.990608:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880356.990611:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880356.990615:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880356.990617:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000b4a4221. 00000020:00000040:0.0:1788880356.990620:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880356.990622:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880356.990624:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880356.990626:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880356.990631:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000079c96644. 00000100:00000001:0.0:1788880356.990635:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880356.990639:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880356.990647:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880356.990647:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880356.990651:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880356.990653:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880356.990654:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880356.990657:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880356.990658:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880356.990663:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880356.990670:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000b4a4221] to pc [ptlrpcd_00_00+0] req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880356.990675:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880356.990679:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880356.990684:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880356.990687:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880356.990690:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880356.990695:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880356.990695:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780651136:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880356.990702:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880356.990702:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880356.990710:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:3.0:1788880356.990711:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:0.0:1788880356.990714:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c2a263b1. 02000000:00000001:3.0:1788880356.990714:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880356.990716:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880356.990718:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000020:00000040:0.0:1788880356.990720:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880356.990720:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880356.990723:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:3.0:1788880356.990724:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cd0104b. 00000100:00000001:0.0:1788880356.990725:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:2.0:1788880356.990728:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880356.990735:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880356.990736:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880356.990740:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880356.990744:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:3.0:1788880356.990745:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c948c598. 02000000:00000010:0.0:1788880356.990747:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000005d090de9. 00000100:00000001:2.0:1788880356.990747:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880356.990753:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880356.990754:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880356.990755:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 02000000:00000001:2.0:1788880356.990756:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880356.990762:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000200:3.0:1788880356.990763:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780651136, portal 25 02000000:00000001:2.0:1788880356.990768:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880356.990776:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780651264:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880356.990777:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880356.990779:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880356.990781:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880356.990783:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880356.990785:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880356.990786:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880356.990788:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c2a263b1] to pc [ptlrpcd_00_01+1] req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880356.990789:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000010c9710f. 02000000:00000001:2.0:1788880356.990792:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880356.990797:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000003e3a0df8. 00000400:00000010:2.0:1788880356.990802:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009c31f261. 00000100:00000001:0.0:1788880356.990803:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880356.990805:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780651264, portal 10 00000100:00000040:0.0:1788880356.990806:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880356.990810:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880356.990811:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880356.990812:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849373056 : -126822860178560 : ffff8ca7c0783b80) 00000100:00000040:0.0:1788880356.990817:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11325) 00000100:00000040:2.0:1788880356.990821:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880356.990827:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880356.990829:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880356.990831:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000084794ee4. 00000100:00000200:2.0:1788880356.990833:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780651264, offset 0 00000100:00000001:3.0:1788880356.990837:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880356.990839:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880356.990841:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849365888 : -126822860185728 : ffff8ca7c0781f80) 00000400:00000200:2.0:1788880356.990842:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880356.990844:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880356.990847:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880356.990852:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880372 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880356.990854:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880356.990854:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880356.990863:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880356.990867:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880356.990873:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880356.990879:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880356.990882:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880356.990885:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880356.990889:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880356.990889:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [157:157:256:4294967295] 192.168.204.111@tcp LPNI seq info [157:157:8:4294967295] 00000100:00000040:3.0:1788880356.990890:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880356.990893:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:3.0:1788880356.990895:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 02000000:00000001:0.0:1788880356.990896:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880356.990900:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880356.990900:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780651136, offset 0 00000100:00100000:0.0:1788880356.990905:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780651392:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880356.990908:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880356.990914:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880356.990914:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880356.990919:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880356.990922:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880356.990926:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788880356.990926:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:2.0:1788880356.990927:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000010:0.0:1788880356.990933:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007d30b075. 00000400:00000200:3.0:1788880356.990934:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:2.0:1788880356.990935:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000011e1ab42. 00000400:00000200:3.0:1788880356.990938:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880356.990939:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880356.990941:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1788880356.990945:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880356.990946:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:0.0:1788880356.990952:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000844d9b13. 00000400:00000200:3.0:1788880356.990952:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [158:158:255:4294967295] 192.168.204.111@tcp LPNI seq info [158:158:7:4294967295] 00000400:00000010:0.0:1788880356.990959:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000800:00000200:2.0:1788880356.990959:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:0.0:1788880356.990966:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780651392, portal 10 00000400:00000200:3.0:1788880356.990968:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880356.990971:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000011e1ab42 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880356.990972:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880356.990975:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364992 : -126822860186624 : ffff8ca7c0781c00) 00000800:00000200:3.0:1788880356.990977:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880356.990986:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:3.0:1788880356.990989:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000100:00000001:2.0:1788880356.990991:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880356.990996:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 593165 usecs (594 jiffies) 00000800:00000200:3.0:1788880356.990998:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880356.991002:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305988690 00000100:00000001:0.0:1788880356.991005:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:3.0:1788880356.991007:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:2.0:1788880356.991008:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000247083fe. 00000100:00000040:0.0:1788880356.991012:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880356.991014:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880356.991015:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000400:00000010:0.0:1788880356.991017:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 00000100:00000001:3.0:1788880356.991020:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880356.991022:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780651392, offset 0 00000100:00000001:2.0:1788880356.991022:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305988690 00000100:00000001:3.0:1788880356.991025:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305993167 00000100:00000010:2.0:1788880356.991030:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000076127301. 00000400:00000200:0.0:1788880356.991033:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880356.991033:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4305988690 00000100:00000001:2.0:1788880356.991036:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880356.991042:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305993167 00000100:00000001:2.0:1788880356.991043:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305988690 00000400:00000200:0.0:1788880356.991044:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880356.991048:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4305988690 00000100:00000010:2.0:1788880356.991050:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e91f0731. 00000400:00000200:0.0:1788880356.991051:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880356.991056:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000400:00000200:0.0:1788880356.991058:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880356.991064:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880356.991064:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305988690 00000100:00000001:3.0:1788880356.991064:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305993167 00000800:00000200:1.0:1788880356.991065:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880356.991070:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4305988690 00000400:00000200:0.0:1788880356.991072:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000010:1.0:1788880356.991072:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000011e1ab42. 00000100:00000010:2.0:1788880356.991073:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045db2575. 00000100:00000001:3.0:1788880356.991078:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305993167 00000100:00000001:2.0:1788880356.991080:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000400:00000200:0.0:1788880356.991082:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [159:159:254:4294967295] 192.168.204.111@tcp LPNI seq info [159:159:6:4294967295] 00000400:00000200:1.0:1788880356.991082:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880356.991088:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880356.991093:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4305988690 00000100:00000001:2.0:1788880356.991094:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880356.991095:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880356.991102:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880356.991103:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880356.991103:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880356.991109:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880356.991110:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880356.991112:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880356.991112:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880356.991113:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000084794ee4 00000400:00000010:1.0:1788880356.991118:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000084794ee4. 00000100:00000001:3.0:1788880356.991118:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:0.0:1788880356.991119:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000022b14507. 00000100:00000001:2.0:1788880356.991120:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880356.991125:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880356.991126:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880356.991126:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:0.0:1788880356.991129:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880356.991130:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880356.991131:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880356.991134:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:1.0:1788880356.991135:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880356.991136:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:0.0:1788880356.991137:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880356.991137:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880356.991141:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:0.0:1788880356.991142:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000022b14507 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880356.991143:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880356.991144:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880356.991150:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880356.991151:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880356.991156:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 36468 usecs (37 jiffies) 00000100:00000001:0.0:1788880356.991162:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305988133 00000100:00000001:1.0:1788880356.991164:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880356.991165:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305993167 00000100:00000001:0.0:1788880356.991169:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305988133 00000100:00000040:1.0:1788880356.991171:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880356.991173:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305993167 00000100:00000001:1.0:1788880356.991181:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880356.991183:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305988133 00000100:00000001:1.0:1788880356.991185:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880356.991188:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305993167 00000100:00000001:0.0:1788880356.991191:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305988133 00000100:00000001:0.0:1788880356.991193:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305993167 00000100:00000001:0.0:1788880356.991195:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880356.991197:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880356.991199:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880356.991202:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:1.0:1788880356.991204:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880356.991206:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:1.0:1788880356.991207:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000100:00000001:0.0:1788880356.991210:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:1.0:1788880356.991212:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880356.991214:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880356.991217:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880356.991219:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880356.991220:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880356.991224:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880356.991256:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000400:00000010:1.0:1788880356.991259:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000100:00000001:1.0:1788880356.991262:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880356.991265:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880372 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880356.991271:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880356.991273:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880372 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880356.991279:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880356.991280:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880356.991318:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880356.991321:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000022b14507. 00000400:00000200:1.0:1788880356.991325:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880356.991328:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880356.991330:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000400:00000010:1.0:1788880356.991332:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000001:1.0:1788880356.991334:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880356.991336:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880356.991343:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880356.991345:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880356.991361:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880356.991362:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880356.993344:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880356.993357:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880356.993360:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880356.993361:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880356.993368:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880356.993379:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b04a80 00000400:00000200:0.0:1788880356.993386:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x5cd [1] + 224 00000800:00000001:0.0:1788880356.993390:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880356.993392:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880356.993394:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880356.993397:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880356.993401:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880356.993405:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880356.993412:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880356.993483:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880356.993487:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880356.993510:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880356.993540:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880356.993544:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880356.993546:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880356.993548:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880356.993552:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880356.993558:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04b80 00000400:00000200:0.0:1788880356.993564:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x5dd [1] + 224 00000800:00000001:0.0:1788880356.993567:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880356.993570:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880356.993572:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880356.993574:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880356.993577:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880356.993580:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880356.993588:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880356.993605:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880356.993610:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880356.993613:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880356.993615:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880356.993616:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880356.993620:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880356.993625:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04b00 00000400:00000200:0.0:1788880356.993630:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x5d1 [1] + 224 00000800:00000001:0.0:1788880356.993633:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880356.993640:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880356.993642:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880356.993645:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880356.993647:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880356.993648:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880356.993652:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:0.0:1788880356.993653:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788880356.993660:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:0.0:1788880356.993664:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788880356.993664:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880356.993668:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000844d9b13. 00000400:00000200:2.0:1788880356.993672:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000400:00000010:2.0:1788880356.993676:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:2.0:1788880356.993689:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880356.993694:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:2.0:1788880356.993695:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880356.993701:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880356.993707:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880356.993707:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880356.993716:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880356.993718:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880356.993721:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880356.993725:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880356.993727:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880356.993730:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880356.993733:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880356.993736:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880356.993740:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:2.0:1788880356.993743:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=34 v=5 (5 5 0 0) 00000400:00000001:1.0:1788880356.993744:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880356.993747:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:2.0:1788880356.993750:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=8 v=5 (1 1 0 0) 00000400:00000010:1.0:1788880356.993751:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000003e3a0df8. 00000400:00000200:1.0:1788880356.993757:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009c31f261 00000100:00000001:2.0:1788880356.993757:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000010:1.0:1788880356.993760:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009c31f261. 00000100:00000001:2.0:1788880356.993761:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880356.993766:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:2.0:1788880356.993766:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000001:3.0:1788880356.993767:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880356.993771:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000200:1.0:1788880356.993772:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880356.993775:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c948c598. 00010000:00080000:2.0:1788880356.993776:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880356.993781:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880356.993785:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880356.993791:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880356.993798:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880356.993804:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880356.993804:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880356.993811:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880356.993814:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880356.993819:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000200:3.0:1788880356.993821:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880356.993823:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880356.993825:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880356.993828:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880356.993830:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880356.993831:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880356.993833:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880356.993834:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880356.993837:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880356.993838:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880356.993841:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880356.993841:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880356.993844:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:1.0:1788880356.993845:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880356.993846:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880356.993853:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=24 v=5 (5 5 0 0) 00000100:00000040:2.0:1788880356.993856:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880356.993856:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880356.993860:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=24 v=5 (1 1 0 0) 00000100:00001000:3.0:1788880356.993864:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=34 v=5 (5 5 0 0) 00000100:00000001:1.0:1788880356.993868:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880356.993871:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880356.993873:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788880356.993876:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880356.993879:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00010000:00000001:1.0:1788880356.993880:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880356.993880:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=34 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880356.993884:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880356.993887:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880356.993890:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880356.993891:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880356.993893:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880356.993894:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880356.993897:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00010000:00000001:3.0:1788880356.993898:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880356.993903:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880356.993904:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880356.993906:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880356.993911:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880356.993914:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780651392:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880356.993915:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880356.993933:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880356.993933:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880356.993937:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880356.993944:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880356.993945:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c2a263b1 x1875776780651392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880356.993946:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880356.993948:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880356.993953:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880356.993957:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880356.993958:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880356.993963:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:3.0:1788880356.993966:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880356.993968:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880356.993973:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007d30b075. 02000000:00000001:2.0:1788880356.993979:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880356.993982:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880356.993986:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00100000:3.0:1788880356.993988:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780651136:192.168.204.111@tcp:400: 00000100:00100000:1.0:1788880356.993992:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780651264:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880356.993995:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788880356.993997:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880356.994001:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880356.994002:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:2.0:1788880356.994003:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000005d090de9. 00000100:00000001:1.0:1788880356.994007:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880356.994008:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000363de9c9 x1875776780651136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880356.994011:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:1.0:1788880356.994013:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000b4a4221 x1875776780651264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880372 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880356.994015:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880356.994018:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c2a263b1. 00000100:00000001:3.0:1788880356.994024:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880356.994025:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:3.0:1788880356.994028:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880356.994029:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1788880356.994032:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cd0104b. 00000100:00000001:2.0:1788880356.994035:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880356.994039:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880356.994040:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:1.0:1788880356.994042:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880356.994044:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000020:00000001:3.0:1788880356.994044:0:201288:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000010:1.0:1788880356.994045:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000010c9710f. 00000100:00000001:2.0:1788880356.994048:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880356.994048:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 02000000:00000001:1.0:1788880356.994051:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880356.994053:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:1.0:1788880356.994055:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000001:3.0:1788880356.994055:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880356.994057:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880356.994059:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000010:3.0:1788880356.994059:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000df3afd78. 00000020:00000001:1.0:1788880356.994064:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:3.0:1788880356.994065:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000010:1.0:1788880356.994067:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000079c96644. 02000000:00000001:3.0:1788880356.994068:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880356.994070:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000363de9c9. 02000000:00000001:1.0:1788880356.994073:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880356.994074:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:1.0:1788880356.994076:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880356.994077:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:1.0:1788880356.994078:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000b4a4221. 00000100:00000001:3.0:1788880356.994082:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880356.994084:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880356.994086:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880356.994088:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880356.994089:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880356.994093:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880356.994094:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880356.994097:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880356.994098:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880356.994102:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880356.994103:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880356.994106:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880357.054346:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880357.054352:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880357.054357:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880357.054358:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880357.054422:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880357.054426:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880357.054429:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880358.078355:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880358.078362:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880358.078368:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880358.078385:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880358.078448:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880358.078451:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880358.078453:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880359.102333:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880359.102340:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880359.102348:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880359.102350:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880359.102424:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880359.102429:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880359.102432:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880360.126306:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880360.126312:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880360.126318:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880360.126319:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880360.126345:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880360.126348:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880360.126351:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880361.150343:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880361.150350:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880361.150357:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880361.150358:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880361.150416:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880361.150420:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880361.150423:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788880362.110399:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305993167, at 4305993216 00000100:00000040:1.0:1788880362.110433:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305993167, at 4305993216 00000100:00000040:2.0:1788880362.110433:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305993167, at 4305993216 00000100:00000040:0.0:1788880362.110434:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:3.0:1788880362.110436:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045db2575. 00000100:00000001:0.0:1788880362.110455:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880362.110459:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e91f0731. 00000100:00000010:1.0:1788880362.110464:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000076127301. 00000100:00000010:0.0:1788880362.110469:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d4e901df. 00000020:00000040:0.0:1788880362.110478:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880362.110485:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880362.110489:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880362.110497:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880362.110509:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000079c96644. 00000100:00000001:0.0:1788880362.110518:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880362.110532:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880362.110544:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880362.110547:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880362.110555:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d4e901df] to pc [ptlrpcd_00_02+2] req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880362.110581:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880362.110584:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880362.110588:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880362.110591:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006f591845. 00000020:00000040:0.0:1788880362.110594:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880362.110597:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880362.110599:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880362.110602:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880362.110604:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880362.110605:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000df3afd78. 00000100:00100000:3.0:1788880362.110609:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:0.0:1788880362.110611:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880362.110612:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880362.110614:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880362.110618:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880362.110620:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880362.110622:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880362.110625:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880362.110630:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880362.110632:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880362.110635:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880362.110640:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880362.110646:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880362.110649:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880362.110654:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880362.110657:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880362.110661:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880362.110662:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:3.0:1788880362.110664:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780651520:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880362.110666:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880362.110668:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:0.0:1788880362.110675:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006f591845] to pc [ptlrpcd_00_03+3] req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880362.110675:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880362.110678:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880362.110683:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880362.110690:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e7b82f46. 00000100:00000001:0.0:1788880362.110696:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880362.110699:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880362.110704:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880362.110706:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880362.110710:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000802a821. 00000020:00000040:0.0:1788880362.110714:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000010:3.0:1788880362.110714:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c948c598. 00000100:00000001:0.0:1788880362.110719:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880362.110721:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880362.110723:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000010:3.0:1788880362.110725:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 02000000:00000001:0.0:1788880362.110727:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880362.110728:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:2.0:1788880362.110733:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788880362.110735:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000005d090de9. 00000100:00000200:3.0:1788880362.110735:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780651520, portal 25 00000100:00000001:3.0:1788880362.110741:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880362.110742:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880362.110743:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849369472 : -126822860182144 : ffff8ca7c0782d80) 00000100:00000040:0.0:1788880362.110752:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880362.110755:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788880362.110755:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880378 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880362.110760:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880362.110766:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880362.110770:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880362.110774:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880362.110780:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880362.110781:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880362.110784:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880362.110784:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880362.110789:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880362.110794:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000802a821] to pc [ptlrpcd_00_00+0] req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880362.110795:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eab8d071. 00000100:00000200:3.0:1788880362.110801:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780651520, offset 0 00000100:00000001:2.0:1788880362.110808:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880362.110810:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880362.110815:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880362.110816:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880362.110819:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880362.110822:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880362.110825:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006f591845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780651648:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880362.110829:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880362.110831:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880362.110833:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:3.0:1788880362.110836:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:0.0:1788880362.110837:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880362.110837:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880362.110842:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880362.110845:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:3.0:1788880362.110846:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:2.0:1788880362.110848:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008e3e56e7. 02000000:00000001:2.0:1788880362.110853:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880362.110854:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880362.110854:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:0.0:1788880362.110855:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11330) 00000100:00000001:1.0:1788880362.110859:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880362.110863:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:2.0:1788880362.110864:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006bdd9925. 00000400:00000200:3.0:1788880362.110866:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:2.0:1788880362.110878:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000793778c8. 00000100:00000040:1.0:1788880362.110879:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880362.110884:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305993167, at 4305993216 00000400:00000200:3.0:1788880362.110884:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:2.0:1788880362.110890:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780651648, portal 10 00000100:00000010:0.0:1788880362.110900:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000247083fe. 00000100:00000001:2.0:1788880362.110905:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788880362.110907:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [160:160:256:4294967295] 192.168.204.111@tcp LPNI seq info [160:160:8:4294967295] 00000100:00000001:2.0:1788880362.110915:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849363200 : -126822860188416 : ffff8ca7c0781500) 00000100:00000001:1.0:1788880362.110931:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880362.110934:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880362.110934:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880362.110942:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880362.110944:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880362.110945:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880362.110950:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880362.110966:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:1.0:1788880362.110967:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880362.110978:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00100000:1.0:1788880362.110982:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780651776:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880362.110987:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:3.0:1788880362.110991:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880362.110994:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880362.110996:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880362.111000:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000010:3.0:1788880362.111000:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000100:00000040:2.0:1788880362.111005:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:1.0:1788880362.111011:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000015664212. 00000400:00000010:2.0:1788880362.111012:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d208f26. 02000000:00000001:1.0:1788880362.111019:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880362.111020:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880362.111023:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780651648, offset 0 00000400:00000010:1.0:1788880362.111027:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000800:00000200:3.0:1788880362.111033:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:1.0:1788880362.111036:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000800:00000200:3.0:1788880362.111038:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880362.111043:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:1.0:1788880362.111053:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780651776, portal 10 00000400:00000200:2.0:1788880362.111056:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880362.111057:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880362.111063:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4305998287 00000400:00000200:2.0:1788880362.111067:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:3.0:1788880362.111074:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004800b45d. 00000100:00000001:1.0:1788880362.111075:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880362.111085:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849370368 : -126822860181248 : ffff8ca7c0783100) 00000100:00000001:3.0:1788880362.111086:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:2.0:1788880362.111090:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880362.111093:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4305998287 00000100:00000010:3.0:1788880362.111098:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001632a9ba. 00000400:00000200:2.0:1788880362.111100:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:1.0:1788880362.111105:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880362.111109:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:2.0:1788880362.111111:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880362.111116:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4305998287 00000100:00000010:3.0:1788880362.111126:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d6fe56. 00000400:00000200:2.0:1788880362.111127:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [161:161:255:4294967295] 192.168.204.111@tcp LPNI seq info [161:161:7:4294967295] 00000100:00000001:3.0:1788880362.111135:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880362.111144:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4305998287 00000100:00000001:1.0:1788880362.111150:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:3.0:1788880362.111160:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f093018f. 00000100:00000040:1.0:1788880362.111164:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880362.111166:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880362.111169:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000010:1.0:1788880362.111177:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 00000100:00000001:3.0:1788880362.111178:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788880362.111185:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780651776, offset 0 00000800:00000200:2.0:1788880362.111186:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880362.111186:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880362.111197:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880362.111199:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880362.111202:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 00000800:00000010:0.0:1788880362.111204:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000100:00000001:3.0:1788880362.111206:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880362.111209:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880362.111215:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880362.111217:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880362.111219:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:2.0:1788880362.111220:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880362.111223:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880362.111228:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880362.111254:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880362.111271:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:2.0:1788880362.111272:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880362.111277:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880362.111283:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880362.111285:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880362.111291:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:2.0:1788880362.111295:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 579773 usecs (580 jiffies) 00000400:00000200:1.0:1788880362.111298:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880362.111306:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880362.111306:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305993797 00000100:00000001:2.0:1788880362.111315:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305998287 00000400:00000200:1.0:1788880362.111318:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [162:162:254:4294967295] 192.168.204.111@tcp LPNI seq info [162:162:6:4294967295] 00000100:00000001:2.0:1788880362.111322:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305993797 00000100:00000001:2.0:1788880362.111328:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305998287 00000100:00000001:2.0:1788880362.111332:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305993797 00000100:00000001:2.0:1788880362.111337:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305998287 00000100:00000001:2.0:1788880362.111344:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305993797 00000400:00000200:1.0:1788880362.111346:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880362.111349:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305998287 00000100:00000001:2.0:1788880362.111355:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880362.111358:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880362.111360:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eab8d071 00000800:00000200:1.0:1788880362.111363:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880362.111366:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880362.111367:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eab8d071. 00000800:00000010:1.0:1788880362.111371:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000001:2.0:1788880362.111375:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880362.111376:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:1.0:1788880362.111385:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:0.0:1788880362.111387:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880378 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880362.111401:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880362.111408:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880362.111410:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880362.111411:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880362.111428:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 35647 usecs (36 jiffies) 00000100:00000040:0.0:1788880362.111431:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880378 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880362.111433:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305993253 00000100:00000001:1.0:1788880362.111444:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4305998287 00000100:00000001:1.0:1788880362.111452:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305993253 00000100:00000001:1.0:1788880362.111459:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4305998287 00000100:00000001:1.0:1788880362.111463:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305993253 00000100:00000001:1.0:1788880362.111469:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4305998287 00000100:00000001:0.0:1788880362.111472:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880362.111478:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880362.111479:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305993253 00000100:00000001:1.0:1788880362.111482:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4305998287 00000100:00000001:1.0:1788880362.111483:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880362.111485:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880362.111487:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880362.111490:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880362.111497:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880362.111498:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880362.111500:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880362.111501:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:0.0:1788880362.111505:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880362.111505:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880362.111516:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000100:00000001:1.0:1788880362.111516:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880362.111528:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880362.111536:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880362.111539:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d208f26 00000400:00000010:0.0:1788880362.111542:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d208f26. 00000100:00000001:0.0:1788880362.111546:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880362.111549:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880362.111556:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880362.111558:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880362.111564:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880362.111565:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880362.111594:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880362.111597:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:0.0:1788880362.111600:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880362.111604:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880362.111606:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:0.0:1788880362.111615:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:0.0:1788880362.111619:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880362.111622:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880362.111629:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880362.111632:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880362.111637:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880362.111639:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880362.112477:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880362.112489:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880362.112491:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880362.112492:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880362.112499:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880362.112508:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b04c00 00000400:00000200:1.0:1788880362.112513:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x5e5 [1] + 224 00000800:00000001:1.0:1788880362.112517:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880362.112525:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880362.112526:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880362.112529:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880362.112533:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880362.112537:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880362.112546:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880362.112560:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880362.112567:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880362.112582:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880362.112584:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880362.112592:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880362.112601:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880362.112603:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880362.112608:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880362.112615:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04c80 00000400:00000001:3.0:1788880362.112618:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880362.112622:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880362.112712:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c948c598. 00000400:00000200:3.0:1788880362.112716:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880362.112717:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880362.112720:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788880362.112726:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x5ed [1] + 224 00000800:00000001:1.0:1788880362.112732:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1788880362.112735:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880362.112736:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880362.112740:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880362.112746:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880362.112748:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880362.112768:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880362.112786:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880362.112801:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880362.112824:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880362.112825:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880362.112829:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880362.112830:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:3.0:1788880362.112834:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880362.112838:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:1.0:1788880362.112841:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880362.112841:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880362.112847:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880362.112849:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788880362.112853:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=40 v=5 (5 5 0 0) 00000800:00000001:1.0:1788880362.112854:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880362.112859:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788880362.112869:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=40 v=5 (1 1 0 0) 00000400:00000200:1.0:1788880362.112874:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880362.112877:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880362.112877:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880362.112881:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880362.112882:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:3.0:1788880362.112887:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:1.0:1788880362.112888:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04d00 00000400:00000001:2.0:1788880362.112891:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00010000:00000001:3.0:1788880362.112891:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788880362.112897:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880362.112899:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880362.112907:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006bdd9925. 00000100:00000040:3.0:1788880362.112907:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880362.112915:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880362.112919:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:2.0:1788880362.112923:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000793778c8 00000100:00000001:3.0:1788880362.112924:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880362.112929:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000793778c8. 00000100:00000040:3.0:1788880362.112932:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880362.112939:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788880362.112947:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x5f5 [1] + 224 00000100:00100000:3.0:1788880362.112948:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780651520:192.168.204.111@tcp:400: 00000100:00000200:2.0:1788880362.112952:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880362.112967:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880362.112968:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880362.112974:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:1.0:1788880362.112979:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880362.112982:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d4e901df x1875776780651520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880362.112985:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880362.112987:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880362.113011:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880362.113019:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880362.113029:0:201278:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1788880362.113029:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880362.113030:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:3.0:1788880362.113033:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e7b82f46. 00000100:00000001:2.0:1788880362.113036:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880362.113039:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880362.113040:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880362.113043:0:201287:0:(client.c:1580:after_reply()) Process entered 00000020:00000001:3.0:1788880362.113044:0:201288:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000001:2.0:1788880362.113048:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000040:3.0:1788880362.113048:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880362.113055:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000001:3.0:1788880362.113058:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880362.113065:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880362.113067:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000079c96644. 02000000:00000001:2.0:1788880362.113076:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880362.113081:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:1.0:1788880362.113082:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:2.0:1788880362.113086:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=30 v=5 (5 5 0 0) 02000000:00000001:3.0:1788880362.113087:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880362.113092:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d4e901df. 00000100:00001000:2.0:1788880362.113105:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=30 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880362.113107:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880362.113113:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880362.113115:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880362.113117:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880362.113123:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880362.113130:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:2.0:1788880362.113136:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880362.113144:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880362.113145:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880362.113150:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880362.113152:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880362.113156:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880362.113162:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880362.113162:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880362.113167:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:1.0:1788880362.113169:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880362.113171:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880362.113175:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880362.113183:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880362.113192:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880362.113194:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880362.113195:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880362.113202:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880362.113203:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880362.113208:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880362.113210:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:2.0:1788880362.113214:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880362.113215:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000400:00000200:0.0:1788880362.113222:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:0.0:1788880362.113226:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:0.0:1788880362.113252:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880362.113256:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880362.113264:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880362.113276:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880362.113281:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880362.113287:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00100000:2.0:1788880362.113292:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006f591845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780651648:192.168.204.111@tcp:400: 02000000:00000001:0.0:1788880362.113294:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880362.113299:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880362.113302:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880362.113305:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880362.113319:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=40 v=5 (5 5 0 0) 00000100:00000001:2.0:1788880362.113319:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880362.113327:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00001000:0.0:1788880362.113329:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=14 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880362.113334:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880362.113339:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880362.113339:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006f591845 x1875776780651648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880362.113350:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880362.113359:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00010000:00080000:0.0:1788880362.113360:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880362.113364:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880362.113369:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008e3e56e7. 02000000:00000001:2.0:1788880362.113375:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880362.113377:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880362.113381:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880362.113393:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880362.113397:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000df3afd78. 00010000:00000001:0.0:1788880362.113398:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880362.113408:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:2.0:1788880362.113410:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:0.0:1788880362.113412:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:2.0:1788880362.113414:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880362.113426:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880362.113430:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880362.113430:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006f591845. 00000100:00000001:2.0:1788880362.113441:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:0.0:1788880362.113444:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880362.113446:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880362.113454:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880362.113464:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880362.113465:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880362.113469:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880362.113470:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880362.113475:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880362.113478:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880362.113482:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880362.113485:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880362.113498:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780651776:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880362.113502:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880362.113504:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880362.113505:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000802a821 x1875776780651776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880378 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880362.113510:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880362.113512:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880362.113515:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000015664212. 02000000:00000001:0.0:1788880362.113518:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880362.113520:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880362.113521:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880362.113523:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880362.113524:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000005d090de9. 02000000:00000001:0.0:1788880362.113526:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880362.113527:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880362.113529:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000802a821. 00000100:00000001:0.0:1788880362.113531:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880362.113532:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880362.113538:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880362.113542:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880362.113543:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880362.113544:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880362.113545:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880362.174337:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880362.174343:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880362.174349:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880362.174350:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880362.174402:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880362.174411:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880362.174413:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880363.198364:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880363.198373:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880363.198375:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880363.198377:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880363.198381:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880363.198384:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880363.198386:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880363.198388:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880363.198393:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880363.198394:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880363.198467:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880363.198475:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880363.198479:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880364.222364:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880364.222371:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880364.222388:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880364.222389:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880364.222443:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880364.222447:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880364.222449:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880365.246380:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880365.246389:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880365.246396:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880365.246400:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880365.246403:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880365.246406:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880365.246410:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880365.246413:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880365.246415:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880365.246418:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880365.246469:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880365.246473:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880365.246476:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880366.270364:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880366.270370:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880366.270377:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880366.270378:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880366.270436:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880366.270440:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880366.270443:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880367.230379:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4305998287, at 4305998336 00000100:00000040:1.0:1788880367.230380:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880367.230380:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4305998287, at 4305998336 00000100:00000040:0.0:1788880367.230382:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4305998287, at 4305998336 00000100:00000010:2.0:1788880367.230401:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d6fe56. 00000100:00000010:0.0:1788880367.230406:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004800b45d. 00000100:00000010:3.0:1788880367.230410:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f093018f. 00000100:00000001:1.0:1788880367.230415:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880367.230424:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b5bd1bf6. 00000020:00000040:1.0:1788880367.230430:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880367.230436:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880367.230441:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880367.230450:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880367.230462:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:1.0:1788880367.230471:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880367.230486:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880367.230497:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880367.230500:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880367.230507:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b5bd1bf6] to pc [ptlrpcd_00_01+1] req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880367.230542:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880367.230545:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880367.230549:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880367.230552:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 00000020:00000040:1.0:1788880367.230554:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880367.230557:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880367.230558:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880367.230560:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880367.230563:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880367.230564:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:1.0:1788880367.230569:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880367.230570:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880367.230574:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880367.230576:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880367.230583:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880367.230583:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880367.230584:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880367.230586:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880367.230593:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_00_02+2] req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880367.230593:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880367.230602:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880367.230605:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880367.230612:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880367.230616:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880367.230618:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880367.230621:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b5bd1bf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780651904:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880367.230625:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880367.230631:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880367.230632:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880367.230635:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880367.230638:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880367.230642:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880367.230643:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880367.230644:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b31f2b6a. 02000000:00000001:0.0:1788880367.230648:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880367.230648:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:1.0:1788880367.230649:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880367.230652:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880367.230653:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:1.0:1788880367.230654:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002b387587. 00000100:00000001:3.0:1788880367.230655:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880367.230657:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ed06891d. 00000020:00000040:1.0:1788880367.230659:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880367.230662:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880367.230664:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:0.0:1788880367.230665:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000033f4d237. 00000100:00000001:2.0:1788880367.230666:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880367.230667:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880367.230672:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880367.230672:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:0.0:1788880367.230675:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780651904, portal 25 00000100:00000001:2.0:1788880367.230675:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880367.230677:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880367.230678:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880367.230681:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938059520 : -126823771492096 : ffff8ca78a26b100) 02000000:00000010:1.0:1788880367.230685:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000040:2.0:1788880367.230686:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880367.230692:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880367.230695:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880383 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880367.230701:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880367.230703:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880367.230706:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880367.230711:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880367.230714:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880367.230719:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880367.230725:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780652032:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880367.230727:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880367.230730:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880367.230731:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880367.230734:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880367.230735:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880367.230736:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880367.230740:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880367.230741:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 00000100:00000040:1.0:1788880367.230743:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002b387587] to pc [ptlrpcd_00_03+3] req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880367.230743:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880367.230749:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a6d90e67. 00000100:00000200:0.0:1788880367.230754:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780651904, offset 0 02000000:00000001:2.0:1788880367.230754:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880367.230763:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006bdd9925. 00000400:00000010:2.0:1788880367.230770:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000793778c8. 00000400:00000200:0.0:1788880367.230775:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880367.230775:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880367.230795:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880367.230801:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880367.230805:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880367.230809:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880367.230811:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880367.230815:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [163:163:256:4294967295] 192.168.204.111@tcp LPNI seq info [163:163:8:4294967295] 00000400:00000200:0.0:1788880367.230833:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880367.230839:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880367.230843:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000040:1.0:1788880367.230845:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000200:0.0:1788880367.230846:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880367.230848:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880367.230857:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780652032, portal 10 00000100:00000001:3.0:1788880367.230859:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:0.0:1788880367.230860:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000005d090de9. 00000100:00000040:1.0:1788880367.230861:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880367.230861:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880367.230863:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880367.230867:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 00000100:00000001:3.0:1788880367.230869:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880367.230877:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:0.0:1788880367.230881:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880367.230881:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880367.230888:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880367.230889:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880367.230895:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:1.0:1788880367.230899:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11336) 00000800:00000200:0.0:1788880367.230903:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000005d090de9 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880367.230916:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880367.230921:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880367.230922:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880367.230926:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880367.230926:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880367.230931:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306003407 00000400:00000010:2.0:1788880367.230931:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000170fdf47. 02000000:00000001:3.0:1788880367.230933:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:1.0:1788880367.230934:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4305998287, at 4305998336 02000000:00000001:3.0:1788880367.230937:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788880367.230938:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780652032, offset 0 00000100:00000010:0.0:1788880367.230939:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014321469. 02000000:00000001:3.0:1788880367.230946:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880367.230947:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00100000:3.0:1788880367.230955:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780652160:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880367.230956:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306003407 00000100:00000010:1.0:1788880367.230961:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001632a9ba. 00000400:00000200:2.0:1788880367.230963:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880367.230964:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e602733. 00000100:00000001:3.0:1788880367.230965:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880367.230973:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 02000000:00000001:3.0:1788880367.230977:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880367.230981:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880367.230983:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306003407 00000400:00000200:2.0:1788880367.230983:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880367.230985:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:0.0:1788880367.230992:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000751d0eed. 00000400:00000200:2.0:1788880367.230992:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:3.0:1788880367.230994:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f8a9d8a8. 00000400:00000200:2.0:1788880367.230998:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880367.231000:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 02000000:00000001:3.0:1788880367.231000:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880367.231004:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880367.231011:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880367.231015:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [164:164:255:4294967295] 192.168.204.111@tcp LPNI seq info [164:164:7:4294967295] 00000400:00000010:3.0:1788880367.231015:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c948c598. 00000100:00000001:0.0:1788880367.231017:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306003407 00000400:00000010:3.0:1788880367.231024:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000010:0.0:1788880367.231026:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006bd48548. 00000100:00000200:3.0:1788880367.231031:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780652160, portal 10 00000100:00000001:0.0:1788880367.231033:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:2.0:1788880367.231033:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880367.231036:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880367.231039:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938057728 : -126823771493888 : ffff8ca78a26aa00) 00000100:00000001:0.0:1788880367.231041:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880367.231045:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880367.231048:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880367.231052:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000100:00000040:3.0:1788880367.231053:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880367.231060:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880367.231062:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880367.231066:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880367.231074:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:2.0:1788880367.231074:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880367.231075:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880367.231078:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:2.0:1788880367.231080:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880367.231084:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880367.231084:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880367.231088:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880367.231088:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880367.231090:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007839347f. 00000100:00000001:0.0:1788880367.231092:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880367.231095:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780652160, offset 0 00000100:00000001:0.0:1788880367.231096:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880367.231096:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 566955 usecs (567 jiffies) 00000100:00000001:2.0:1788880367.231103:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305998903 00000400:00000200:3.0:1788880367.231106:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880367.231108:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306003407 00000100:00000001:2.0:1788880367.231113:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305998903 00000400:00000200:3.0:1788880367.231115:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880367.231116:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306003407 00000100:00000001:2.0:1788880367.231121:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305998903 00000400:00000200:3.0:1788880367.231124:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880367.231125:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306003407 00000100:00000001:2.0:1788880367.231129:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305998903 00000400:00000200:3.0:1788880367.231130:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880367.231131:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306003407 00000100:00000001:2.0:1788880367.231134:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880367.231136:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:3.0:1788880367.231137:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880367.231138:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880367.231145:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880367.231146:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000005d090de9. 00000400:00000200:3.0:1788880367.231146:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880367.231147:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880367.231153:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880367.231157:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880367.231158:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [165:165:254:4294967295] 192.168.204.111@tcp LPNI seq info [165:165:6:4294967295] 00000100:00000001:2.0:1788880367.231159:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880367.231161:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880367.231169:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880367.231186:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880367.231186:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880367.231206:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880367.231207:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000400:00000010:1.0:1788880367.231214:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000800:00000010:3.0:1788880367.231215:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000055d7d92. 00000800:00000200:3.0:1788880367.231227:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880367.231263:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880367.231271:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880383 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880367.231309:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880367.231314:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880383 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880367.231334:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880367.231337:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880367.231367:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880367.231372:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000055d7d92 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880367.231377:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880367.231380:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 34853 usecs (35 jiffies) 00000100:00000001:3.0:1788880367.231382:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4305998372 00000100:00000001:3.0:1788880367.231385:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306003407 00000100:00000001:3.0:1788880367.231387:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4305998372 00000100:00000001:3.0:1788880367.231390:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306003407 00000800:00000200:1.0:1788880367.231392:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880367.231396:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000100:00000001:3.0:1788880367.231403:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4305998372 00000100:00000001:3.0:1788880367.231407:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306003407 00000400:00000200:1.0:1788880367.231408:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880367.231413:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4305998372 00000100:00000001:3.0:1788880367.231417:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306003407 00000100:00000001:3.0:1788880367.231421:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880367.231426:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880367.231428:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880367.231434:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000170fdf47 00000100:00000001:3.0:1788880367.231434:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880367.231441:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000170fdf47. 00000100:00000001:1.0:1788880367.231444:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880367.231448:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880367.231455:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880367.231457:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880367.231468:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880367.231470:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880367.231480:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880367.231513:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880367.231518:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000055d7d92. 00000400:00000200:1.0:1788880367.231522:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880367.231525:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880367.231528:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007839347f 00000400:00000010:1.0:1788880367.231530:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007839347f. 00000100:00000001:1.0:1788880367.231532:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880367.231535:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880367.231543:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880367.231544:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880367.231562:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880367.231564:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880367.233027:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880367.233047:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880367.233051:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880367.233054:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880367.233066:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880367.233081:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b04d80 00000400:00000200:0.0:1788880367.233089:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x5fd [1] + 224 00000800:00000001:0.0:1788880367.233098:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880367.233102:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880367.233105:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880367.233109:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880367.233116:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880367.233122:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880367.233132:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880367.233152:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880367.233162:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880367.233165:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880367.233168:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880367.233170:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880367.233175:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880367.233182:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04e00 00000400:00000200:0.0:1788880367.233188:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x605 [1] + 224 00000100:00000001:2.0:1788880367.233189:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:0.0:1788880367.233194:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880367.233194:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880367.233202:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880367.233206:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880367.233209:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ed06891d. 00000800:00000001:0.0:1788880367.233215:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880367.233215:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000033f4d237 00000400:00000010:2.0:1788880367.233217:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000033f4d237. 00000400:00000200:0.0:1788880367.233220:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880367.233222:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:0.0:1788880367.233226:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:2.0:1788880367.233230:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880367.233259:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880367.233266:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880367.233278:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880367.233284:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880367.233293:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880367.233296:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880367.233300:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880367.233303:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880367.233305:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880367.233308:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880367.233308:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880367.233313:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880367.233331:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=45 v=5 (5 5 0 0) 00000800:00000001:0.0:1788880367.233336:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:2.0:1788880367.233337:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=45 v=5 (1 1 0 0) 00000800:00000001:0.0:1788880367.233343:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880367.233343:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880367.233346:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880367.233350:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880367.233353:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880367.233356:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880367.233361:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880367.233369:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880367.233372:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880367.233374:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880367.233378:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880367.233386:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b5bd1bf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780651904:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880367.233399:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880367.233402:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880367.233406:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b5bd1bf6 x1875776780651904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880367.233415:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880367.233419:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880367.233421:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880367.233423:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b31f2b6a. 00000100:00000001:3.0:1788880367.233428:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880367.233430:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880367.233434:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000001:3.0:1788880367.233436:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000040:2.0:1788880367.233437:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000001:3.0:1788880367.233441:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000020:00000001:2.0:1788880367.233443:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880367.233447:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 02000000:00000001:2.0:1788880367.233451:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880367.233453:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880367.233455:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b5bd1bf6. 00000100:00000001:2.0:1788880367.233458:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880367.233460:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880367.233462:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880367.233468:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880367.233470:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880367.233474:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880367.233478:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880367.233481:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006bdd9925. 00000400:00000200:3.0:1788880367.233488:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000793778c8 00000400:00000010:3.0:1788880367.233489:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000793778c8. 00000100:00000001:3.0:1788880367.233492:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880367.233501:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880367.233513:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880367.233520:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880367.233522:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880367.233525:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880367.233528:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880367.233530:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880367.233532:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880367.233535:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880367.233544:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=35 v=5 (5 5 0 0) 00000100:00001000:3.0:1788880367.233547:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=35 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880367.233550:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880367.233551:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880367.233553:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880367.233557:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880367.233571:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880367.233573:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880367.233576:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880367.233577:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880367.233581:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880367.233589:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880367.233592:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880367.233593:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880367.233596:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880367.233603:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780652032:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880367.233608:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880367.233610:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880367.233612:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780652032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880367.233619:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880367.233621:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880367.233624:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a6d90e67. 02000000:00000001:3.0:1788880367.233627:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880367.233628:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880367.233630:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880367.233632:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880367.233634:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:3.0:1788880367.233638:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880367.233639:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880367.233641:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000001:3.0:1788880367.233644:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880367.233645:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880367.233646:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880367.233653:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880367.233655:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880367.233657:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880367.233662:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880367.233725:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880367.233736:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880367.233739:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880367.233741:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880367.233748:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880367.233762:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04e80 00000400:00000200:1.0:1788880367.233770:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x60d [1] + 224 00000800:00000001:1.0:1788880367.233776:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880367.233787:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880367.233790:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880367.233794:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880367.233798:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880367.233802:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880367.233810:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880367.233825:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880367.233832:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880367.233838:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880367.233882:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880367.233885:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880367.233889:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880367.233891:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880367.233892:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c948c598. 00000400:00000200:3.0:1788880367.233896:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880367.233897:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880367.233900:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880367.233903:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880367.233911:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880367.233916:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880367.233918:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880367.233919:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880367.233921:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880367.233922:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880367.233923:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880367.233925:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880367.233928:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=45 v=5 (5 5 0 0) 00000100:00001000:3.0:1788880367.233931:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=19 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880367.233933:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880367.233934:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880367.233935:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880367.233938:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880367.233944:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880367.233946:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880367.233947:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880367.233948:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880367.233949:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880367.233952:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880367.233967:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880367.233969:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880367.233970:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880367.233973:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880367.233980:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780652160:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880367.233984:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880367.233985:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880367.233986:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002b387587 x1875776780652160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880383 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880367.233990:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880367.233991:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880367.233992:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f8a9d8a8. 02000000:00000001:3.0:1788880367.233994:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880367.233994:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880367.233995:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880367.233996:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880367.233997:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:3.0:1788880367.233999:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880367.233999:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880367.234000:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002b387587. 00000100:00000001:3.0:1788880367.234001:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880367.234002:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880367.234003:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880367.234006:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880367.234007:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880367.234008:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880367.234009:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880367.294393:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880367.294401:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880367.294408:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880367.294409:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880367.294476:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880367.294480:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880367.294483:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880368.318336:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880368.318343:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880368.318348:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880368.318355:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880368.318422:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880368.318429:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880368.318432:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880369.342405:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880369.342416:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880369.342424:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880369.342454:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880369.342528:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880369.342552:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880369.342556:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880370.366586:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880370.366595:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880370.366603:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880370.366604:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880370.368451:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880370.368458:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880370.368463:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880371.390375:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880371.390383:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880371.390388:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880371.390388:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880371.390466:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880371.390473:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880371.390475:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788880372.350314:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306003407, at 4306003456 00000100:00000040:2.0:1788880372.350324:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306003407, at 4306003456 00000100:00000040:1.0:1788880372.350336:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306003407, at 4306003456 00000100:00000010:3.0:1788880372.350345:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006bd48548. 00000100:00000010:1.0:1788880372.350347:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e602733. 00000100:00000010:2.0:1788880372.350350:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000751d0eed. 00000100:00000040:0.0:1788880372.350364:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306003407, at 4306003456 00000100:00000010:0.0:1788880372.350382:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014321469. 00000100:00000040:3.0:1788880372.350397:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880372.350405:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880372.350414:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880372.350420:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880372.350426:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880372.350431:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880372.350438:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880372.350449:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:3.0:1788880372.350458:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880372.350473:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880372.350486:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880372.350490:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880372.350508:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_00+0] req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880372.350536:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880372.350538:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880372.350541:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880372.350544:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880372.350547:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880372.350549:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880372.350551:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880372.350552:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880372.350556:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000055d7d92. 00000100:00000001:1.0:1788880372.350557:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880372.350560:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880372.350566:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880372.350571:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880372.350580:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880372.350581:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880372.350598:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880372.350604:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880372.350612:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880372.350613:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880372.350616:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880372.350617:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880372.350625:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880372.350629:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880372.350633:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880372.350638:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_01+1] req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880372.350639:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780652288:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880372.350649:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880372.350657:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880372.350661:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880372.350664:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880372.350669:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000020450263. 02000000:00000001:1.0:1788880372.350671:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880372.350679:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c21d159e. 00000400:00000010:1.0:1788880372.350688:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000007206624. 00000100:00000200:1.0:1788880372.350697:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780652288, portal 25 00000100:00000001:1.0:1788880372.350699:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880372.350701:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000040:1.0:1788880372.350706:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880388 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880372.350723:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880372.350727:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880372.350730:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000100:00000001:3.0:1788880372.350730:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880372.350734:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880372.350735:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:1.0:1788880372.350736:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780652288, offset 0 00000100:00000001:3.0:1788880372.350740:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1788880372.350741:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:3.0:1788880372.350744:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000100:00000001:2.0:1788880372.350745:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788880372.350749:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000020:00000040:3.0:1788880372.350750:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:2.0:1788880372.350754:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880372.350757:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880372.350759:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880372.350761:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880372.350767:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000400:00000200:1.0:1788880372.350772:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880372.350774:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880372.350780:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880372.350783:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880372.350784:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880372.350785:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880372.350789:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880372.350793:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880372.350794:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:2.0:1788880372.350800:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880372.350804:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880372.350804:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:2.0:1788880372.350808:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780652416:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880372.350808:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880372.350813:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880372.350817:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880372.350818:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_02+2] req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880372.350822:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1788880372.350824:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [166:166:256:4294967295] 192.168.204.111@tcp LPNI seq info [166:166:8:4294967295] 02000000:00000001:2.0:1788880372.350826:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880372.350831:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880372.350837:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000afd3911b. 02000000:00000001:2.0:1788880372.350841:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880372.350846:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a55bc54d. 00000400:00000200:1.0:1788880372.350849:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:2.0:1788880372.350853:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000005509e00. 00000800:00000200:1.0:1788880372.350858:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788880372.350859:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780652416, portal 10 00000100:00000001:3.0:1788880372.350861:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880372.350864:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000001:2.0:1788880372.350864:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:3.0:1788880372.350865:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880372.350867:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000800:00000200:1.0:1788880372.350875:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:3.0:1788880372.350875:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:2.0:1788880372.350881:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880372.350886:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11341) 00000800:00000200:1.0:1788880372.350889:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880372.350896:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880372.350901:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880372.350901:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880372.350906:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000040:2.0:1788880372.350908:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880372.350909:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880372.350911:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880372.350913:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880372.350914:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c56cc487. 00000100:00000001:0.0:1788880372.350915:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880372.350927:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780652416, offset 0 00000100:00000001:1.0:1788880372.350929:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306008527 00000100:00000001:3.0:1788880372.350935:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000010:1.0:1788880372.350936:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000400:00000200:2.0:1788880372.350939:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880372.350939:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880372.350944:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880372.350948:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880372.350951:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306008527 00000400:00000200:2.0:1788880372.350952:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880372.350952:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:1.0:1788880372.350957:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000100:00000001:3.0:1788880372.350957:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880372.350960:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880372.350962:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:3.0:1788880372.350966:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880372.350967:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306008527 00000400:00000200:2.0:1788880372.350967:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:1.0:1788880372.350974:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000100:00000001:1.0:1788880372.350981:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:2.0:1788880372.350981:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880372.350981:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880372.350985:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880372.350989:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880372.350990:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306008527 00000400:00000200:2.0:1788880372.350992:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880372.350993:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:1.0:1788880372.350998:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 02000000:00000001:3.0:1788880372.350998:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880372.351001:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [167:167:255:4294967295] 192.168.204.111@tcp LPNI seq info [167:167:7:4294967295] 00000100:00100000:3.0:1788880372.351005:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780652544:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880372.351012:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880372.351013:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880372.351018:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880372.351020:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880372.351022:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880372.351024:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880372.351026:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880372.351027:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:0.0:1788880372.351030:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880372.351033:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880372.351033:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000006ab02ff. 00000800:00000010:0.0:1788880372.351037:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:1.0:1788880372.351041:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880372.351041:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880372.351049:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880372.351051:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880372.351051:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:3.0:1788880372.351052:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c948c598. 00000100:00000001:1.0:1788880372.351056:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:2.0:1788880372.351056:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c52725cf. 00000100:00000001:1.0:1788880372.351061:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880372.351061:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000001:1.0:1788880372.351064:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:3.0:1788880372.351066:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780652544, portal 10 00000100:00000001:1.0:1788880372.351067:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880372.351074:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880372.351074:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880372.351076:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880372.351076:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880372.351077:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000800:00000200:2.0:1788880372.351091:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:3.0:1788880372.351094:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880372.351097:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000800:00000200:2.0:1788880372.351097:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c52725cf type 1, nob 320 niov 1 nkiov 1 00000400:00000010:0.0:1788880372.351107:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:2.0:1788880372.351110:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880372.351116:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000040:2.0:1788880372.351119:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 554708 usecs (555 jiffies) 00000100:00000001:3.0:1788880372.351122:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880372.351125:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306004011 00000100:00000200:0.0:1788880372.351131:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880388 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880372.351133:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880372.351134:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306008527 00000400:00000010:3.0:1788880372.351139:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cd783db0. 00000100:00000001:2.0:1788880372.351141:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306004011 00000100:00000001:2.0:1788880372.351149:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306008527 00000100:00000200:3.0:1788880372.351150:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780652544, offset 0 00000100:00000001:2.0:1788880372.351153:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306004011 00000100:00000001:2.0:1788880372.351158:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306008527 00000100:00000001:2.0:1788880372.351166:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306004011 00000100:00000001:2.0:1788880372.351172:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306008527 00000100:00000001:2.0:1788880372.351177:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880372.351180:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880372.351182:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880372.351187:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880388 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880372.351189:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880372.351199:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880372.351211:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880372.351214:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880372.351215:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880372.351275:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880372.351276:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880372.351279:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880372.351282:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880372.351286:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880372.351299:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880372.351304:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c52725cf. 00000400:00000200:0.0:1788880372.351308:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880372.351312:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880372.351412:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880372.351418:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c56cc487 00000400:00000200:3.0:1788880372.351422:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:0.0:1788880372.351423:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c56cc487. 00000400:00000200:3.0:1788880372.351429:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880372.351434:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880372.351441:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880372.351449:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880372.351451:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880372.351458:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880372.351460:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:3.0:1788880372.351548:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880372.351551:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880372.351553:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880372.351556:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [168:168:255:4294967295] 192.168.204.111@tcp LPNI seq info [168:168:7:4294967295] 00000400:00000200:3.0:1788880372.351564:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880372.351569:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880372.351573:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000800:00000200:3.0:1788880372.351578:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880372.351583:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880372.351586:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880372.351598:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880372.351601:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 34118 usecs (35 jiffies) 00000100:00000001:3.0:1788880372.351605:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306003492 00000100:00000001:3.0:1788880372.351608:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306008527 00000100:00000001:3.0:1788880372.351610:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306003492 00000100:00000001:3.0:1788880372.351613:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306008527 00000100:00000001:3.0:1788880372.351615:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306003492 00000100:00000001:3.0:1788880372.351617:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306008527 00000100:00000001:3.0:1788880372.351620:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306003492 00000100:00000001:3.0:1788880372.351621:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306008527 00000100:00000001:3.0:1788880372.351623:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880372.351627:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880372.351629:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880372.351632:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880372.351661:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880372.351665:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000400:00000200:0.0:1788880372.351669:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880372.351674:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880372.351677:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cd783db0 00000400:00000010:0.0:1788880372.351679:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cd783db0. 00000100:00000001:0.0:1788880372.351681:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880372.351685:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880372.351692:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880372.351694:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880372.351700:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880372.351702:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880372.352461:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880372.352469:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880372.352472:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880372.352475:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880372.352482:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880372.352494:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b04f00 00000400:00000200:1.0:1788880372.352502:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x615 [1] + 224 00000800:00000001:1.0:1788880372.352510:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880372.352516:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880372.352518:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880372.352521:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880372.352526:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880372.352530:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880372.352537:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880372.352550:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880372.352556:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880372.352558:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880372.352589:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880372.352593:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880372.352599:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880372.352602:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880372.352605:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c21d159e. 00000400:00000200:2.0:1788880372.352610:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000007206624 00000400:00000010:2.0:1788880372.352620:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000007206624. 00000100:00000001:2.0:1788880372.352624:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:0.0:1788880372.352629:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:2.0:1788880372.352630:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880372.352645:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880372.352645:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880372.352653:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880372.352656:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880372.352672:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:0.0:1788880372.352676:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880372.352677:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880372.352685:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880372.352689:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880372.352691:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880372.352694:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880372.352701:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b04f80 02000000:00000001:2.0:1788880372.352702:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880372.352715:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=50 v=5 (5 5 0 0) 00000400:00000200:0.0:1788880372.352722:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x61d [1] + 224 00000100:00001000:2.0:1788880372.352725:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=50 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880372.352732:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:0.0:1788880372.352733:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880372.352735:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880372.352743:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880372.352748:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880372.352752:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880372.352757:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880372.352762:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880372.352766:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880372.352773:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880372.352775:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:0.0:1788880372.352779:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880372.352779:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880372.352794:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:2.0:1788880372.352799:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880372.352804:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880372.352838:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780652288:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880372.352847:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880372.352850:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880372.352861:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880372.352868:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780652288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880372.352877:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880372.352887:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880372.352892:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880372.352903:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880372.352907:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880372.352912:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000020450263. 00000100:00000001:3.0:1788880372.352914:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880372.352917:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880372.352919:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000001:3.0:1788880372.352922:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000001:2.0:1788880372.352923:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000001:3.0:1788880372.352926:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000020:00000040:2.0:1788880372.352927:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000010:3.0:1788880372.352930:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a55bc54d. 00000020:00000001:2.0:1788880372.352931:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880372.352934:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 02000000:00000001:2.0:1788880372.352940:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000200:3.0:1788880372.352942:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000005509e00 02000000:00000001:2.0:1788880372.352944:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000010:3.0:1788880372.352945:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000005509e00. 00000100:00000010:2.0:1788880372.352947:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:3.0:1788880372.352950:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880372.352954:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880372.352957:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880372.352960:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880372.352966:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880372.352967:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880372.352970:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880372.352971:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880372.353041:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880372.353054:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880372.353062:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880372.353065:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880372.353066:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880372.353072:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880372.353074:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880372.353076:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880372.353080:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880372.353085:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05000 00000400:00000200:0.0:1788880372.353089:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x625 [1] + 224 00000800:00000001:0.0:1788880372.353092:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880372.353102:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880372.353103:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880372.353109:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880372.353109:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880372.353119:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880372.353120:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880372.353126:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880372.353129:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880372.353131:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880372.353134:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880372.353146:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=40 v=5 (5 5 0 0) 00000100:00000040:0.0:1788880372.353151:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880372.353163:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=40 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880372.353169:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880372.353170:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880372.353173:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880372.353176:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880372.353177:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880372.353181:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880372.353184:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:0.0:1788880372.353187:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880372.353187:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880372.353192:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880372.353197:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880372.353201:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880372.353203:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880372.353207:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880372.353213:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880372.353214:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000001:1.0:1788880372.353216:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880372.353218:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880372.353220:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880372.353259:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c948c598. 00000100:00000040:3.0:1788880372.353260:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880372.353263:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:1.0:1788880372.353266:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:1.0:1788880372.353270:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880372.353276:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880372.353278:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780652416:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880372.353293:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880372.353298:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880372.353302:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880372.353307:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880372.353308:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780652416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880372.353310:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880372.353318:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880372.353320:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880372.353323:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000010:3.0:1788880372.353323:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000afd3911b. 02000000:00000001:3.0:1788880372.353328:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:1.0:1788880372.353330:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:3.0:1788880372.353331:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880372.353333:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000040:3.0:1788880372.353334:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880372.353337:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880372.353339:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:1.0:1788880372.353342:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880372.353343:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000055d7d92. 02000000:00000001:3.0:1788880372.353349:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00001000:1.0:1788880372.353350:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=50 v=5 (5 5 0 0) 02000000:00000001:3.0:1788880372.353351:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880372.353354:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00001000:1.0:1788880372.353357:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=24 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880372.353358:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880372.353361:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880372.353364:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880372.353365:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880372.353367:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880372.353371:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880372.353372:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880372.353375:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:1.0:1788880372.353377:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880372.353379:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880372.353382:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880372.353389:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880372.353392:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880372.353394:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880372.353396:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880372.353397:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880372.353400:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880372.353407:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880372.353409:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880372.353410:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880372.353413:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880372.353427:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780652544:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880372.353432:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880372.353434:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880372.353436:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780652544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880388 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880372.353442:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880372.353443:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880372.353445:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000006ab02ff. 02000000:00000001:1.0:1788880372.353448:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880372.353449:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880372.353450:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880372.353452:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880372.353454:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 02000000:00000001:1.0:1788880372.353456:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880372.353457:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880372.353459:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:1.0:1788880372.353461:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880372.353462:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880372.353464:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880372.353468:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880372.353469:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880372.353471:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880372.353472:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880372.414506:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880372.414514:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880372.414520:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880372.414521:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880372.414621:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880372.414628:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880372.414632:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880373.438341:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880373.438348:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880373.438351:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880373.438353:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880373.438357:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880373.438360:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880373.438362:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880373.438379:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880373.438383:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880373.438384:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880373.438450:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880373.438456:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880373.438461:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880374.462339:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880374.462346:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880374.462353:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880374.462354:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880374.462414:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880374.462419:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880374.462422:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880375.486404:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880375.486413:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880375.486420:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880375.486426:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880375.486428:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880375.486432:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880375.486434:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880375.486437:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880375.486440:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880375.486442:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880375.486550:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880375.486554:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880375.486556:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880376.510356:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880376.510364:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880376.510370:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880376.510371:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880376.510429:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880376.510433:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880376.510436:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880377.470313:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306008527, at 4306008576 00000100:00000040:0.0:1788880377.470318:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306008527, at 4306008576 00000100:00000040:3.0:1788880377.470319:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306008527, at 4306008576 00000100:00000010:3.0:1788880377.470329:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000010:1.0:1788880377.470334:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000010:0.0:1788880377.470342:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000040:3.0:1788880377.470353:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880377.470360:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880377.470365:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880377.470370:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880377.470379:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880377.470380:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880377.470384:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880377.470389:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000055d7d92. 00000100:00000001:3.0:1788880377.470395:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880377.470403:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880377.470411:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880377.470413:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880377.470417:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_03+3] req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880377.470425:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306008527, at 4306008576 00000100:00000010:2.0:1788880377.470432:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000001:3.0:1788880377.470439:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880377.470442:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880377.470446:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880377.470448:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880377.470450:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880377.470453:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880377.470454:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880377.470455:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880377.470456:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880377.470457:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:1.0:1788880377.470460:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880377.470460:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880377.470462:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880377.470463:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:1.0:1788880377.470466:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880377.470470:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880377.470472:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880377.470474:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788880377.470475:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_00+0] req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880377.470476:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880377.470480:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880377.470481:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880377.470482:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880377.470487:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880377.470487:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880377.470494:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780652672:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880377.470497:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880377.470498:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880377.470499:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880377.470503:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880377.470509:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 02000000:00000001:1.0:1788880377.470510:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000020:00000040:3.0:1788880377.470511:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880377.470512:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880377.470513:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880377.470514:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880377.470515:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:3.0:1788880377.470516:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880377.470519:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000046cece22. 02000000:00000010:3.0:1788880377.470519:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:2.0:1788880377.470520:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880377.470521:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880377.470522:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880377.470524:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880377.470526:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880377.470527:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:1.0:1788880377.470530:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dc9bac80. 00000100:00000040:2.0:1788880377.470532:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880377.470537:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a131edae. 00000100:00000200:1.0:1788880377.470545:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780652672, portal 25 00000100:00000001:3.0:1788880377.470547:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880377.470549:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880377.470549:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880377.470552:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000100:00000001:2.0:1788880377.470552:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788880377.470555:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_01+1] req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880377.470557:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880393 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880377.470560:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880377.470565:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880377.470567:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880377.470569:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:2.0:1788880377.470570:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880377.470570:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880377.470573:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880377.470575:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00100000:2.0:1788880377.470577:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780652800:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880377.470579:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:1.0:1788880377.470580:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000994ad42e. 00000100:00000001:2.0:1788880377.470583:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000200:1.0:1788880377.470584:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780652672, offset 0 02000000:00000001:2.0:1788880377.470586:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880377.470586:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11346) 02000000:00000001:2.0:1788880377.470588:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880377.470590:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880377.470592:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000784e3b9. 00000400:00000200:1.0:1788880377.470593:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880377.470595:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880377.470597:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:2.0:1788880377.470600:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007593939c. 00000100:00000001:3.0:1788880377.470600:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880377.470603:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:2.0:1788880377.470606:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008f790e4a. 00000400:00000200:1.0:1788880377.470610:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:3.0:1788880377.470610:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880377.470611:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780652800, portal 10 00000100:00000001:2.0:1788880377.470615:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880377.470616:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880377.470618:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000400:00000200:1.0:1788880377.470621:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880377.470623:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880377.470625:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880377.470626:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1788880377.470628:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880377.470629:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880377.470631:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880377.470631:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880377.470636:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880377.470638:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [169:169:256:4294967295] 192.168.204.111@tcp LPNI seq info [169:169:8:4294967295] 00000100:00100000:3.0:1788880377.470642:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780652928:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880377.470650:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880377.470651:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880377.470653:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1788880377.470655:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:3.0:1788880377.470655:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880377.470659:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880377.470664:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:1.0:1788880377.470665:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880377.470665:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000841b943f. 02000000:00000010:3.0:1788880377.470667:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000006ab02ff. 02000000:00000001:3.0:1788880377.470670:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880377.470671:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780652800, offset 0 00000800:00000010:1.0:1788880377.470672:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000400:00000010:3.0:1788880377.470674:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ba5e1826. 00000400:00000010:3.0:1788880377.470679:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000400:00000200:2.0:1788880377.470683:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880377.470684:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880377.470692:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:3.0:1788880377.470692:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780652928, portal 10 00000100:00000001:3.0:1788880377.470696:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:1.0:1788880377.470697:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880377.470699:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000800:00000200:1.0:1788880377.470701:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880377.470701:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880377.470707:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880377.470709:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880377.470713:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880377.470718:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880377.470719:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880377.470724:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306013647 00000400:00000200:2.0:1788880377.470724:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [170:170:255:4294967295] 192.168.204.111@tcp LPNI seq info [170:170:7:4294967295] 00000100:00000001:3.0:1788880377.470724:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880377.470728:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880377.470730:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000400:00000010:3.0:1788880377.470731:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8336eb2. 00000100:00000200:3.0:1788880377.470733:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780652928, offset 0 00000100:00000001:1.0:1788880377.470739:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:3.0:1788880377.470739:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880377.470743:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306013647 00000100:00000010:1.0:1788880377.470748:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000400:00000200:2.0:1788880377.470750:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880377.470753:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880377.470759:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306013647 00000800:00000200:2.0:1788880377.470759:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880377.470760:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:1.0:1788880377.470766:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000400:00000200:3.0:1788880377.470766:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:2.0:1788880377.470767:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000007cb951ea. 00000100:00000001:1.0:1788880377.470769:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:3.0:1788880377.470769:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880377.470772:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306013647 00000400:00000200:3.0:1788880377.470772:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880377.470776:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:1.0:1788880377.470778:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000100:00000001:1.0:1788880377.470780:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788880377.470783:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [171:171:254:4294967295] 192.168.204.111@tcp LPNI seq info [171:171:6:4294967295] 00000100:00000001:1.0:1788880377.470785:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880377.470787:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880377.470790:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880377.470790:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880377.470793:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880377.470793:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000010:0.0:1788880377.470795:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000100:00000001:1.0:1788880377.470795:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880377.470801:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880377.470803:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:3.0:1788880377.470805:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880377.470807:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880377.470808:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:3.0:1788880377.470811:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000400:00000200:0.0:1788880377.470813:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880377.470814:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:2.0:1788880377.470814:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000007cb951ea type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880377.470817:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:3.0:1788880377.470820:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880377.470821:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880377.470821:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880377.470824:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000994ad42e 00000100:00000001:1.0:1788880377.470827:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880377.470827:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 542972 usecs (543 jiffies) 00000800:00000200:3.0:1788880377.470827:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:0.0:1788880377.470830:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000994ad42e. 00000100:00000001:2.0:1788880377.470833:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306009119 00000100:00000001:2.0:1788880377.470836:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306013647 00000100:00000001:0.0:1788880377.470837:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880377.470838:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306009119 00000100:00000001:2.0:1788880377.470841:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306013647 00000100:00000001:2.0:1788880377.470843:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306009119 00000100:00000200:0.0:1788880377.470845:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880393 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880377.470846:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306013647 00000100:00000001:2.0:1788880377.470849:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306009119 00000100:00000001:2.0:1788880377.470852:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306013647 00000100:00000001:2.0:1788880377.470855:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880377.470859:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880377.470862:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880377.470865:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880377.470869:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880377.470869:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880377.470870:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880377.470873:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880377.470875:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880377.470876:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880393 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880377.470876:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880377.470879:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880377.470886:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880377.470887:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880377.470891:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880377.470892:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880377.470895:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 33397 usecs (34 jiffies) 00000100:00000001:3.0:1788880377.470897:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306008610 00000100:00000001:3.0:1788880377.470899:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306013647 00000100:00000001:3.0:1788880377.470904:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306008610 00000100:00000001:3.0:1788880377.470905:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306013647 00000100:00000001:3.0:1788880377.470907:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306008610 00000100:00000001:3.0:1788880377.470908:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306013647 00000800:00000200:0.0:1788880377.470909:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880377.470910:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306008610 00000100:00000001:3.0:1788880377.470913:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306013647 00000800:00000010:0.0:1788880377.470914:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000007cb951ea. 00000100:00000001:3.0:1788880377.470915:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880377.470917:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880377.470920:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880377.470921:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880377.470921:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880377.470927:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880377.470929:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880377.470929:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880377.470933:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880377.470935:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000841b943f 00000100:00000001:3.0:1788880377.470935:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880377.470936:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880377.470937:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000841b943f. 00000100:00000001:3.0:1788880377.470938:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880377.470942:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880377.470945:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880377.470951:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880377.470952:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880377.470956:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880377.470957:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880377.470981:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880377.470983:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000400:00000200:0.0:1788880377.470990:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880377.470992:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880377.470994:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8336eb2 00000400:00000010:0.0:1788880377.470995:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8336eb2. 00000100:00000001:0.0:1788880377.470996:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880377.470998:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880377.471002:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880377.471004:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880377.471007:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880377.471008:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880377.472307:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880377.472317:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880377.472319:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880377.472320:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880377.472326:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880377.472333:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b05080 00000400:00000200:0.0:1788880377.472338:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x62d [1] + 224 00000800:00000001:0.0:1788880377.472343:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880377.472345:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880377.472346:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880377.472348:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880377.472351:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880377.472354:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880377.472361:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880377.472373:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880377.472380:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880377.472399:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880377.472400:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880377.472408:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880377.472408:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880377.472412:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880377.472413:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880377.472419:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05100 00000400:00000001:1.0:1788880377.472419:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880377.472423:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880377.472426:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dc9bac80. 00000400:00000200:1.0:1788880377.472429:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a131edae 00000400:00000010:1.0:1788880377.472430:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a131edae. 00000100:00000001:1.0:1788880377.472432:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:0.0:1788880377.472436:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x635 [1] + 224 00000100:00000200:1.0:1788880377.472437:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880377.472448:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880377.472449:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880377.472454:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880377.472458:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880377.472462:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:0.0:1788880377.472464:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880377.472466:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880377.472469:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880377.472472:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880377.472478:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:0.0:1788880377.472480:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880377.472483:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880377.472487:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880377.472491:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880377.472494:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880377.472494:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880377.472500:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:1.0:1788880377.472500:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=55 v=5 (5 5 0 0) 00000800:00000001:0.0:1788880377.472502:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880377.472506:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880377.472508:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880377.472508:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=55 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880377.472514:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:0.0:1788880377.472515:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880377.472517:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880377.472521:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:0.0:1788880377.472522:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05180 00010000:00000001:1.0:1788880377.472525:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880377.472525:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880377.472528:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880377.472529:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x63d [1] + 224 00000100:00000001:1.0:1788880377.472531:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880377.472535:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:2.0:1788880377.472535:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:1.0:1788880377.472538:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788880377.472539:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880377.472541:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007593939c. 00000400:00000200:2.0:1788880377.472546:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008f790e4a 00000100:00000001:1.0:1788880377.472549:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880377.472549:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008f790e4a. 00000100:00000001:1.0:1788880377.472554:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880377.472555:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880377.472556:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880377.472558:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880377.472560:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880377.472562:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880377.472564:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880377.472576:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00100000:1.0:1788880377.472584:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780652672:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880377.472585:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880377.472590:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880377.472592:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880377.472596:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880377.472600:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:0.0:1788880377.472601:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880377.472603:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780652672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880377.472603:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880377.472608:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880377.472610:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880377.472612:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880377.472614:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880377.472619:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880377.472627:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00001000:2.0:1788880377.472627:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=45 v=5 (5 5 0 0) 02000000:00000001:1.0:1788880377.472631:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00001000:2.0:1788880377.472634:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=45 v=5 (1 1 0 0) 02000000:00000010:1.0:1788880377.472636:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000046cece22. 00000100:00000040:0.0:1788880377.472641:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880377.472641:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000001:1.0:1788880377.472643:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880377.472645:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880377.472647:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00010000:00000001:2.0:1788880377.472651:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000040:1.0:1788880377.472652:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00010000:00000001:2.0:1788880377.472656:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880377.472657:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880377.472662:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000055d7d92. 00000100:00000001:2.0:1788880377.472662:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880377.472666:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:1.0:1788880377.472668:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880377.472670:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880377.472673:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880377.472679:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:2.0:1788880377.472679:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880377.472683:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:0.0:1788880377.472688:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880377.472689:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880377.472692:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880377.472697:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880377.472697:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880377.472697:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880377.472702:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880377.472703:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880377.472705:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880377.472705:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:2.0:1788880377.472708:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880377.472714:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880377.472714:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:2.0:1788880377.472718:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780652800:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880377.472718:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880377.472720:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880377.472723:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880377.472724:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880377.472727:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880377.472727:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880377.472728:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ba5e1826. 00000100:00000001:2.0:1788880377.472731:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:3.0:1788880377.472732:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000100:00000040:2.0:1788880377.472736:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780652800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880377.472736:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:3.0:1788880377.472739:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880377.472745:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880377.472753:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880377.472769:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880377.472771:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880377.472774:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880377.472775:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880377.472775:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:2.0:1788880377.472779:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880377.472781:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000784e3b9. 00000100:00000001:3.0:1788880377.472782:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880377.472785:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880377.472787:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:3.0:1788880377.472787:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880377.472792:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00001000:3.0:1788880377.472793:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=55 v=5 (5 5 0 0) 00000020:00000040:2.0:1788880377.472794:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00001000:3.0:1788880377.472797:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=29 v=5 (1 1 0 0) 00000020:00000001:2.0:1788880377.472799:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880377.472800:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880377.472802:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880377.472804:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880377.472806:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880377.472811:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 00010000:00000001:3.0:1788880377.472811:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880377.472814:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880377.472814:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880377.472815:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880377.472816:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880377.472820:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880377.472821:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880377.472824:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880377.472826:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:3.0:1788880377.472827:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880377.472831:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880377.472832:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880377.472832:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880377.472834:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880377.472836:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880377.472841:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1788880377.472844:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780652928:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880377.472847:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880377.472848:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880377.472849:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780652928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880393 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880377.472853:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880377.472854:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:3.0:1788880377.472854:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880377.472858:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880377.472858:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000006ab02ff. 02000000:00000001:3.0:1788880377.472861:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880377.472863:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880377.472866:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880377.472866:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880377.472868:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880377.472871:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880377.472872:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 02000000:00000001:3.0:1788880377.472874:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880377.472875:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880377.472876:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:3.0:1788880377.472878:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880377.472879:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880377.472880:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880377.472883:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880377.472884:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880377.472885:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880377.472886:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880377.534325:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880377.534333:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880377.534338:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880377.534339:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880377.534395:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880377.534400:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880377.534403:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880378.558360:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880378.558369:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880378.558377:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880378.558377:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880378.558432:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880378.558437:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880378.558441:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880379.582333:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880379.582340:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880379.582345:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880379.582346:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880379.582398:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880379.582400:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880379.582402:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880380.606360:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880380.606369:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880380.606375:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880380.606376:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880380.606444:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880380.606449:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880380.606452:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880381.630334:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880381.630342:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880381.630348:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880381.630349:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880381.630394:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880381.630397:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880381.630399:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880382.590344:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306013647, at 4306013696 00000100:00000040:1.0:1788880382.590344:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306013647, at 4306013696 00000100:00000040:2.0:1788880382.590344:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306013647, at 4306013696 00000100:00000040:3.0:1788880382.590357:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306013647, at 4306013696 00000100:00000010:0.0:1788880382.590360:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000010:2.0:1788880382.590362:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000010:1.0:1788880382.590364:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000010:3.0:1788880382.590379:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000040:2.0:1788880382.590400:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880382.590406:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880382.590424:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000020:00000040:2.0:1788880382.590433:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880382.590437:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880382.590439:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880382.590443:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880382.590450:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006aca3ff7. 00000100:00000001:2.0:1788880382.590457:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880382.590467:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880382.590477:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880382.590479:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880382.590484:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d838cab4] to pc [ptlrpcd_00_02+2] req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880382.590512:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880382.590514:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880382.590517:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880382.590520:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000078d5a15d. 00000020:00000040:2.0:1788880382.590523:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880382.590525:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880382.590527:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880382.590528:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880382.590532:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003139f6ba. 00000100:00000001:2.0:1788880382.590534:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880382.590538:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880382.590540:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880382.590546:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880382.590547:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880382.590548:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880382.590549:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880382.590550:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880382.590553:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880382.590558:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000078d5a15d] to pc [ptlrpcd_00_03+3] req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880382.590559:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880382.590582:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880382.590584:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880382.590587:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880382.590587:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880382.590590:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880382.590591:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880382.590594:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880382.590598:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780653056:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880382.590604:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880382.590606:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000010:2.0:1788880382.590608:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880382.590611:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880382.590612:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880382.590613:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880382.590616:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880382.590618:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880382.590618:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880382.590623:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f9812869. 02000000:00000001:0.0:1788880382.590624:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880382.590626:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880382.590630:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000978ba7c2. 00000100:00000040:2.0:1788880382.590631:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880382.590635:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880382.590641:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880382.590642:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880382.590644:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008a8928ee. 00000100:00000040:2.0:1788880382.590647:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_00_00+0] req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880382.590652:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a08ac001. 00000100:00000200:0.0:1788880382.590666:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780653056, portal 25 00000100:00000001:0.0:1788880382.590669:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880382.590670:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000100:00000001:2.0:1788880382.590675:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880382.590676:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880398 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880382.590679:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880382.590687:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880382.590692:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880382.590694:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11351) 00000100:00000001:1.0:1788880382.590696:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880382.590697:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880382.590700:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880382.590703:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880382.590708:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000100:00000040:1.0:1788880382.590710:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880382.590713:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780653056, offset 0 00000400:00000200:0.0:1788880382.590725:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880382.590734:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880382.590736:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880382.590740:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880382.590741:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880382.590745:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880382.590746:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788880382.590747:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880382.590753:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880382.590756:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880382.590757:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:1.0:1788880382.590762:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780653184:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880382.590763:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788880382.590769:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880382.590771:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880382.590774:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:1.0:1788880382.590776:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880382.590782:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880382.590786:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:1.0:1788880382.590791:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880382.590799:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [172:172:256:4294967295] 192.168.204.111@tcp LPNI seq info [172:172:8:4294967295] 02000000:00000010:1.0:1788880382.590799:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000cd5d51f4. 02000000:00000001:1.0:1788880382.590809:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880382.590813:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:1.0:1788880382.590819:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dc9bac80. 00000100:00000001:3.0:1788880382.590819:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880382.590829:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788880382.590832:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:1.0:1788880382.590832:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a131edae. 02000000:00000001:3.0:1788880382.590834:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788880382.590838:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780653184, portal 10 02000000:00000001:3.0:1788880382.590846:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880382.590847:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880382.590847:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880382.590852:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938426496 : -126823771125120 : ffff8ca78a2c4a80) 00000100:00100000:3.0:1788880382.590860:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780653312:192.168.204.111@tcp:400: 00000800:00000010:0.0:1788880382.590861:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000100:00000040:1.0:1788880382.590867:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880382.590873:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:0.0:1788880382.590879:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:3.0:1788880382.590880:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880382.590883:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880382.590890:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880382.590892:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:3.0:1788880382.590898:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a6d90e67. 00000800:00000200:0.0:1788880382.590901:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:1.0:1788880382.590901:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880382.590908:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 00000400:00000010:1.0:1788880382.590908:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005377d7ad. 00000100:00000200:1.0:1788880382.590913:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780653184, offset 0 02000000:00000001:3.0:1788880382.590915:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880382.590924:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ba5e1826. 00000400:00000200:1.0:1788880382.590931:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880382.590934:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880382.590939:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306018767 00000400:00000200:1.0:1788880382.590940:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880382.590945:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:3.0:1788880382.590947:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000100:00000010:0.0:1788880382.590948:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014321469. 00000400:00000200:1.0:1788880382.590952:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:3.0:1788880382.590955:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780653312, portal 10 00000100:00000001:0.0:1788880382.590958:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:1.0:1788880382.590962:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880382.590962:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880382.590967:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000100:00000001:0.0:1788880382.590970:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306018767 00000400:00000200:1.0:1788880382.590972:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:0.0:1788880382.590977:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d2bf5f. 00000100:00000040:3.0:1788880382.590981:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880382.590982:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [173:173:255:4294967295] 192.168.204.111@tcp LPNI seq info [173:173:7:4294967295] 00000100:00000001:0.0:1788880382.590990:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880382.591003:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306018767 00000100:00000010:0.0:1788880382.591014:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000506ae1c0. 00000100:00000001:3.0:1788880382.591026:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880382.591038:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880382.591046:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306018767 00000100:00000040:3.0:1788880382.591046:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880382.591052:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007fe20b6f. 00000100:00000010:0.0:1788880382.591054:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001d590ced. 00000100:00000200:3.0:1788880382.591059:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780653312, offset 0 00000100:00000001:0.0:1788880382.591061:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880382.591067:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880382.591074:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880382.591074:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880382.591078:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880382.591084:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000100:00000001:0.0:1788880382.591085:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880382.591088:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880382.591091:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880382.591094:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880382.591097:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880382.591100:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880382.591101:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880382.591102:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880382.591104:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880382.591107:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880382.591122:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880382.591123:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880382.591171:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880382.591185:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880382.591188:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000800:00000200:1.0:1788880382.591193:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880382.591197:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880382.591199:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880382.591203:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880382.591206:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 531716 usecs (532 jiffies) 00000100:00000001:1.0:1788880382.591209:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306014228 00000100:00000001:1.0:1788880382.591211:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306018767 00000100:00000001:1.0:1788880382.591214:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306014228 00000100:00000001:1.0:1788880382.591216:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306018767 00000100:00000001:1.0:1788880382.591218:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306014228 00000100:00000001:1.0:1788880382.591221:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306018767 00000100:00000001:1.0:1788880382.591222:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306014228 00000100:00000001:1.0:1788880382.591224:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306018767 00000100:00000001:1.0:1788880382.591225:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880382.591228:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880382.591230:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880382.591232:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880382.591272:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880382.591274:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880382.591276:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880382.591277:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880382.591279:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880382.591281:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880382.591295:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880382.591303:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880382.591306:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880382.591310:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880382.591312:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880382.591315:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [174:174:254:4294967295] 192.168.204.111@tcp LPNI seq info [174:174:6:4294967295] 00000400:00000200:3.0:1788880382.591325:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880382.591331:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880382.591334:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000800:00000010:3.0:1788880382.591337:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000400:00000010:2.0:1788880382.591339:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00000100:00000001:2.0:1788880382.591348:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:3.0:1788880382.591349:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880382.591355:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880398 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880382.591362:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880382.591373:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880382.591388:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880382.591388:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880382.591394:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880398 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880382.591395:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 32701 usecs (33 jiffies) 00000100:00000001:3.0:1788880382.591405:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306013730 00000100:00000001:3.0:1788880382.591414:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306018767 00000100:00000001:3.0:1788880382.591417:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306013730 00000100:00000001:3.0:1788880382.591432:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306018767 00000100:00000001:2.0:1788880382.591435:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880382.591438:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306013730 00000100:00000001:2.0:1788880382.591440:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880382.591444:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306018767 00000100:00000001:3.0:1788880382.591448:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306013730 00000100:00000001:3.0:1788880382.591450:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306018767 00000100:00000001:3.0:1788880382.591453:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880382.591456:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880382.591459:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880382.591461:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788880382.591465:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880382.591468:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:2.0:1788880382.591469:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000100:00000001:3.0:1788880382.591472:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880382.591477:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880382.591478:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880382.591482:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880382.591486:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880382.591488:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880382.591491:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880382.591493:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005377d7ad 00000400:00000010:2.0:1788880382.591497:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005377d7ad. 00000100:00000001:2.0:1788880382.591500:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880382.591504:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880382.591511:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880382.591516:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880382.591523:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880382.591524:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880382.591554:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880382.591557:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:2.0:1788880382.591561:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880382.591564:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880382.591566:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007fe20b6f 00000400:00000010:2.0:1788880382.591568:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007fe20b6f. 00000100:00000001:2.0:1788880382.591570:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880382.591573:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880382.591579:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880382.591582:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880382.591588:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880382.591589:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880382.592421:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880382.592433:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880382.592436:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880382.592438:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880382.592445:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880382.592456:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b05200 00000400:00000200:0.0:1788880382.592463:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x645 [1] + 224 00000800:00000001:0.0:1788880382.592470:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880382.592478:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880382.592480:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880382.592483:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880382.592489:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880382.592492:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880382.592502:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880382.592519:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880382.592526:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880382.592530:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880382.592553:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880382.592557:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880382.592565:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880382.592567:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880382.592569:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008a8928ee. 00000400:00000200:3.0:1788880382.592574:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a08ac001 00000400:00000010:3.0:1788880382.592576:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a08ac001. 00000100:00000001:3.0:1788880382.592580:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880382.592586:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880382.592603:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880382.592610:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880382.592612:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880382.592615:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880382.592618:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880382.592620:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880382.592698:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880382.592709:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880382.592717:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=60 v=5 (5 5 0 0) 00000100:00001000:3.0:1788880382.592721:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=60 v=5 (1 1 0 0) 00000800:00000001:0.0:1788880382.592724:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880382.592724:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880382.592729:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880382.592732:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880382.592734:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880382.592735:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880382.592740:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880382.592743:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880382.592748:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880382.592755:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05280 00000400:00000200:0.0:1788880382.592766:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x64d [1] + 224 00000800:00000001:0.0:1788880382.592771:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880382.592775:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880382.592776:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880382.592778:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880382.592780:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880382.592783:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880382.592789:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880382.592801:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880382.592807:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880382.592822:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880382.592822:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880382.592828:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:3.0:1788880382.592831:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880382.592833:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:0.0:1788880382.592834:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880382.592838:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880382.592847:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880382.592851:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:0.0:1788880382.592854:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000010:1.0:1788880382.592856:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dc9bac80. 00000100:00000001:3.0:1788880382.592859:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880382.592864:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a131edae 00000400:00000200:0.0:1788880382.592870:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05300 00000400:00000010:1.0:1788880382.592871:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a131edae. 00000100:00000001:1.0:1788880382.592874:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:0.0:1788880382.592875:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x655 [1] + 224 00000100:00000200:1.0:1788880382.592877:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880382.592878:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880382.592883:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880382.592885:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:1.0:1788880382.592885:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880382.592888:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880382.592893:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880382.592894:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880382.592895:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880382.592897:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880382.592900:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880382.592900:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880382.592901:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880382.592909:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:3.0:1788880382.592914:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880382.592915:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880382.592921:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880382.592927:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880382.592930:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880382.592941:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=50 v=5 (5 5 0 0) 00000100:00100000:3.0:1788880382.592956:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780653056:192.168.204.111@tcp:400: 00000100:00001000:1.0:1788880382.592958:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=50 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880382.592965:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880382.592969:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880382.592971:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880382.592971:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880382.592972:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880382.592976:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880382.592985:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880382.592985:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00010000:00000001:1.0:1788880382.592995:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880382.592996:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d838cab4 x1875776780653056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880382.593001:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880382.593006:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880382.593008:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880382.593011:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880382.593016:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880382.593020:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880382.593020:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000001:2.0:1788880382.593023:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:3.0:1788880382.593039:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000001:2.0:1788880382.593044:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000010:3.0:1788880382.593044:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000978ba7c2. 00000100:00000040:1.0:1788880382.593045:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880382.593052:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ba5e1826. 02000000:00000001:3.0:1788880382.593056:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000200:2.0:1788880382.593059:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000020:00000001:3.0:1788880382.593061:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000010:2.0:1788880382.593064:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000020:00000040:3.0:1788880382.593065:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880382.593067:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880382.593072:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000001:3.0:1788880382.593074:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880382.593076:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000010:3.0:1788880382.593078:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006aca3ff7. 00000100:00000001:1.0:1788880382.593081:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880382.593082:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880382.593087:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:1.0:1788880382.593092:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880382.593092:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880382.593097:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d838cab4. 00000100:00000001:3.0:1788880382.593110:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000200:2.0:1788880382.593111:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880382.593113:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780653184:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880382.593115:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880382.593122:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880382.593125:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880382.593129:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880382.593131:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880382.593131:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880382.593136:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880382.593140:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880382.593142:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880382.593142:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880382.593145:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788880382.593146:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880382.593151:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000078d5a15d x1875776780653184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880382.593152:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880382.593155:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=60 v=5 (5 5 0 0) 00000100:00000001:3.0:1788880382.593163:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:2.0:1788880382.593166:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=34 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880382.593168:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880382.593172:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880382.593176:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880382.593181:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880382.593182:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880382.593186:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00010000:00080000:2.0:1788880382.593188:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880382.593190:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000cd5d51f4. 02000000:00000001:1.0:1788880382.593195:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880382.593198:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880382.593201:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00010000:00000001:2.0:1788880382.593201:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880382.593205:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880382.593207:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000010:1.0:1788880382.593208:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003139f6ba. 00000100:00000040:2.0:1788880382.593209:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:1.0:1788880382.593211:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880382.593211:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000001:1.0:1788880382.593214:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788880382.593214:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880382.593217:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000078d5a15d. 00000100:00000040:2.0:1788880382.593220:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880382.593222:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880382.593224:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880382.593228:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880382.593230:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880382.593253:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880382.593256:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880382.593259:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880382.593260:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880382.593262:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880382.593266:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880382.593270:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880382.593275:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780653312:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880382.593279:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880382.593280:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880382.593282:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780653312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880398 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880382.593286:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880382.593287:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880382.593292:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a6d90e67. 02000000:00000001:2.0:1788880382.593295:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880382.593295:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880382.593296:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880382.593298:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880382.593299:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f9812869. 02000000:00000001:2.0:1788880382.593300:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880382.593301:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880382.593302:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:2.0:1788880382.593303:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880382.593304:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880382.593305:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880382.593307:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880382.593308:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880382.593309:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880382.593311:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880382.654298:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880382.654303:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880382.654308:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880382.654308:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880382.654363:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880382.654368:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880382.654371:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880383.678357:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880383.678366:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880383.678369:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880383.678373:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880383.678378:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880383.678381:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880383.678383:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880383.678385:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880383.678389:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880383.678391:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880383.678463:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880383.678468:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880383.678471:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880384.702339:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880384.702346:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880384.702375:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880384.702377:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880384.702437:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880384.702441:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880384.702444:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880385.726349:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880385.726356:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880385.726362:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880385.726367:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880385.726370:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880385.726372:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880385.726376:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880385.726378:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880385.726380:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880385.726382:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880385.726451:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880385.726456:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880385.726458:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880386.750349:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880386.750354:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880386.750360:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880386.750362:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880386.750446:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880386.750453:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880386.750457:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880387.710362:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306018767, at 4306018816 00000100:00000040:0.0:1788880387.710365:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306018767, at 4306018816 00000100:00000040:3.0:1788880387.710365:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306018767, at 4306018816 00000100:00000040:2.0:1788880387.710373:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880387.710388:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880387.710391:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d2bf5f. 00000100:00000010:0.0:1788880387.710395:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014321469. 00000100:00000010:3.0:1788880387.710397:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001d590ced. 00000100:00000010:2.0:1788880387.710401:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880387.710408:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880387.710420:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880387.710423:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880387.710429:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880387.710438:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f9812869. 00000100:00000001:2.0:1788880387.710445:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880387.710456:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880387.710467:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880387.710470:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880387.710475:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_00_01+1] req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880387.710497:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880387.710498:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880387.710502:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880387.710505:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000020:00000040:2.0:1788880387.710508:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880387.710511:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880387.710513:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880387.710516:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880387.710520:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008eccda10. 00000100:00000001:2.0:1788880387.710523:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880387.710525:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880387.710525:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880387.710530:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880387.710532:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880387.710533:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880387.710537:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880387.710548:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880387.710581:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880387.710583:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880387.710584:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880387.710587:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880387.710590:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fd6b7ee1] to pc [ptlrpcd_00_02+2] req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880387.710592:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880387.710598:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880387.710606:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880387.710612:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780653440:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880387.710616:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880387.710617:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880387.710623:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880387.710626:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:0.0:1788880387.710628:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:2.0:1788880387.710629:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 00000020:00000040:2.0:1788880387.710633:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880387.710634:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880387.710636:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880387.710641:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880387.710642:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:2.0:1788880387.710644:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880387.710645:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880387.710646:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880387.710653:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b31f2b6a. 00000100:00000001:1.0:1788880387.710655:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880387.710657:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880387.710658:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880387.710658:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5d1e8ed. 00000100:00000001:3.0:1788880387.710664:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880387.710672:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008ff937f0. 00000100:00000001:2.0:1788880387.710672:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880387.710676:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880387.710685:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:0.0:1788880387.710686:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e0ace8ca. 00000100:00000001:3.0:1788880387.710696:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000200:0.0:1788880387.710698:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780653440, portal 25 00000100:00000001:3.0:1788880387.710703:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880387.710708:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880387.710714:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 02000000:00000001:3.0:1788880387.710714:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880387.710716:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880387.710717:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880387.710719:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880387.710728:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880387.710732:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880403 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880387.710733:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_03+3] req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880387.710739:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780653568:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880387.710760:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880387.710768:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880387.710775:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880387.710784:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880387.710788:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880387.710789:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880387.710790:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d653e4f6. 00000100:00000040:2.0:1788880387.710793:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880387.710801:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880387.710802:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880387.710805:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:0.0:1788880387.710808:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009fe01c6d. 00000400:00000010:3.0:1788880387.710813:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009877e35e. 00000100:00000040:2.0:1788880387.710815:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11356) 00000100:00000200:0.0:1788880387.710816:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780653440, offset 0 00000100:00000001:1.0:1788880387.710817:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880387.710820:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f73021f1. 00000100:00000001:1.0:1788880387.710825:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:3.0:1788880387.710826:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780653568, portal 10 00000100:00000001:1.0:1788880387.710831:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880387.710831:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880387.710832:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880387.710836:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306018767, at 4306018816 00000100:00000001:3.0:1788880387.710841:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000100:00000010:2.0:1788880387.710845:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000506ae1c0. 00000100:00000040:1.0:1788880387.710846:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880387.710852:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880387.710859:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880387.710879:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880387.710900:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880387.710904:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880387.710905:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788880387.710908:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880387.710910:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880387.710913:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:3.0:1788880387.710918:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880387.710919:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:3.0:1788880387.710926:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005f2b805f. 00000400:00000200:0.0:1788880387.710927:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [175:175:256:4294967295] 192.168.204.111@tcp LPNI seq info [175:175:8:4294967295] 02000000:00000001:1.0:1788880387.710927:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:3.0:1788880387.710938:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780653568, offset 0 02000000:00000001:1.0:1788880387.710948:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880387.710954:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788880387.710960:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780653696:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880387.710974:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880387.710975:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880387.710985:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880387.710989:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880387.710990:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880387.710991:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788880387.710996:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788880387.711000:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:0.0:1788880387.711001:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000400:00000200:3.0:1788880387.711006:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000010:1.0:1788880387.711011:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b2a347c7. 00000400:00000200:3.0:1788880387.711015:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:1.0:1788880387.711021:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880387.711022:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880387.711026:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:1.0:1788880387.711034:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dc9bac80. 00000400:00000200:3.0:1788880387.711035:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [176:176:255:4294967295] 192.168.204.111@tcp LPNI seq info [176:176:7:4294967295] 00000800:00000200:0.0:1788880387.711045:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:1.0:1788880387.711047:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a131edae. 00000800:00000200:0.0:1788880387.711054:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000100:00000200:1.0:1788880387.711055:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780653696, portal 10 00000100:00000001:1.0:1788880387.711062:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788880387.711063:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880387.711070:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938435456 : -126823771116160 : ffff8ca78a2c6d80) 00000800:00000200:3.0:1788880387.711078:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880387.711085:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880387.711085:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000100:00000040:1.0:1788880387.711086:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880387.711093:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306023887 00000800:00000200:3.0:1788880387.711099:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:0.0:1788880387.711104:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014321469. 00000100:00000001:0.0:1788880387.711114:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:3.0:1788880387.711117:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880387.711127:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880387.711129:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306023887 00000100:00000001:1.0:1788880387.711134:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880387.711136:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880387.711137:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009f22a9a0. 00000100:00000040:1.0:1788880387.711146:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880387.711147:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 520923 usecs (521 jiffies) 00000100:00000001:0.0:1788880387.711149:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000010:1.0:1788880387.711151:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009822ffbc. 00000100:00000001:3.0:1788880387.711154:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306019337 00000100:00000200:1.0:1788880387.711159:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780653696, offset 0 00000100:00000001:0.0:1788880387.711160:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306023887 00000100:00000001:3.0:1788880387.711160:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306023887 00000800:00000200:2.0:1788880387.711161:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880387.711170:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306019337 00000100:00000010:0.0:1788880387.711171:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000bc13d1e4. 00000800:00000010:2.0:1788880387.711175:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000100:00000001:3.0:1788880387.711176:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306023887 00000100:00000001:0.0:1788880387.711179:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:2.0:1788880387.711186:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880387.711188:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880387.711188:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306019337 00000100:00000001:0.0:1788880387.711190:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306023887 00000100:00000001:3.0:1788880387.711200:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306023887 00000400:00000200:2.0:1788880387.711201:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000010:0.0:1788880387.711203:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009a4dee8c. 00000100:00000001:3.0:1788880387.711205:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306019337 00000400:00000200:1.0:1788880387.711214:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880387.711215:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880387.711225:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880387.711225:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880387.711225:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306023887 00000400:00000200:1.0:1788880387.711276:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880387.711279:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880387.711279:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880387.711284:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880387.711287:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880387.711288:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880387.711293:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880387.711293:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880387.711302:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [177:177:254:4294967295] 192.168.204.111@tcp LPNI seq info [177:177:6:4294967295] 00000100:00000001:3.0:1788880387.711302:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880387.711305:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880387.711313:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880387.711314:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880387.711322:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880387.711323:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880387.711329:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880387.711333:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880387.711339:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880387.711340:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880387.711340:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880387.711348:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880387.711350:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880387.711353:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880387.711358:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880387.711359:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009fe01c6d 00000800:00000200:1.0:1788880387.711364:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880387.711369:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009fe01c6d. 00000800:00000010:1.0:1788880387.711372:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000001:2.0:1788880387.711376:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:1.0:1788880387.711383:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880387.711385:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880403 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880387.711394:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880387.711400:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880387.711404:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880387.711406:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880387.711410:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880403 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880387.711412:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 32042 usecs (33 jiffies) 00000100:00000001:1.0:1788880387.711418:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306018850 00000100:00000001:2.0:1788880387.711422:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880387.711423:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306023887 00000100:00000001:2.0:1788880387.711427:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880387.711428:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306018850 00000100:00000001:1.0:1788880387.711438:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306023887 00000100:00000001:1.0:1788880387.711440:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306018850 00000100:00000001:1.0:1788880387.711443:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306023887 00000100:00000001:1.0:1788880387.711445:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306018850 00000100:00000001:1.0:1788880387.711447:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306023887 00000100:00000001:1.0:1788880387.711450:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880387.711450:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880387.711454:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880387.711455:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000100:00000001:1.0:1788880387.711460:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880387.711462:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880387.711464:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880387.711469:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880387.711473:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005f2b805f 00000400:00000010:2.0:1788880387.711474:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005f2b805f. 00000100:00000001:1.0:1788880387.711475:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880387.711478:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880387.711478:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880387.711482:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880387.711484:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:2.0:1788880387.711485:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880387.711487:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880387.711492:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880387.711500:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880387.711503:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880387.711510:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880387.711511:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880387.711584:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880387.711586:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:2.0:1788880387.711589:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880387.711592:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880387.711594:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009822ffbc 00000400:00000010:2.0:1788880387.711595:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009822ffbc. 00000100:00000001:2.0:1788880387.711603:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880387.711606:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880387.711613:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880387.711615:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880387.711620:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880387.711621:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880387.713132:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880387.713183:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880387.713186:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880387.713189:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880387.713197:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880387.713208:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b05380 00000400:00000200:0.0:1788880387.713215:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x65d [1] + 224 00000800:00000001:0.0:1788880387.713222:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880387.713225:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880387.713228:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880387.713250:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880387.713256:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880387.713261:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880387.713270:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880387.713286:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880387.713295:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880387.713299:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880387.713302:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880387.713303:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880387.713307:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880387.713312:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05400 00000100:00000001:3.0:1788880387.713314:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880387.713317:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x665 [1] + 224 00000100:00000001:3.0:1788880387.713319:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880387.713321:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880387.713325:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880387.713326:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:3.0:1788880387.713326:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880387.713330:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:0.0:1788880387.713332:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:3.0:1788880387.713334:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008ff937f0. 00000400:00000200:3.0:1788880387.713339:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e0ace8ca 00000400:00000010:3.0:1788880387.713341:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e0ace8ca. 00000100:00000001:3.0:1788880387.713344:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880387.713345:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880387.713350:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880387.713353:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880387.713363:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880387.713386:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:0.0:1788880387.713387:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880387.713389:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880387.713394:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880387.713398:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880387.713401:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880387.713402:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880387.713405:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880387.713410:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=65 v=5 (5 5 0 0) 00000100:00000001:0.0:1788880387.713411:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880387.713417:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:3.0:1788880387.713417:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=65 v=5 (1 1 0 0) 00000800:00000001:0.0:1788880387.713423:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880387.713423:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:0.0:1788880387.713426:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880387.713426:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880387.713429:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880387.713431:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:0.0:1788880387.713435:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00010000:00000001:3.0:1788880387.713435:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880387.713440:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880387.713443:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05480 00000100:00000040:3.0:1788880387.713447:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880387.713450:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880387.713452:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x66d [1] + 224 00000100:00000001:1.0:1788880387.713455:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880387.713461:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880387.713462:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880387.713465:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:0.0:1788880387.713466:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880387.713467:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009877e35e. 00000400:00000200:0.0:1788880387.713470:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880387.713474:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f73021f1 00000400:00000200:0.0:1788880387.713478:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:1.0:1788880387.713478:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f73021f1. 00000100:00000001:1.0:1788880387.713485:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880387.713486:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880387.713487:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880387.713490:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:1.0:1788880387.713493:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880387.713493:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880387.713498:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880387.713501:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880387.713531:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880387.713535:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880387.713536:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780653440:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880387.713547:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880387.713553:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880387.713558:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880387.713559:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780653440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880387.713561:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880387.713565:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880387.713565:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880387.713570:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:0.0:1788880387.713573:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880387.713573:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880387.713574:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880387.713577:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880387.713579:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880387.713579:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880387.713584:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b31f2b6a. 02000000:00000001:1.0:1788880387.713587:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880387.713592:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=55 v=5 (5 5 0 0) 02000000:00000001:3.0:1788880387.713592:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880387.713597:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00001000:1.0:1788880387.713598:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=55 v=5 (1 1 0 0) 00000020:00000040:3.0:1788880387.713600:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880387.713601:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880387.713605:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880387.713606:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:3.0:1788880387.713607:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:1.0:1788880387.713608:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880387.713612:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:1.0:1788880387.713613:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880387.713614:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f9812869. 00000400:00000001:2.0:1788880387.713621:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:3.0:1788880387.713621:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880387.713622:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000001:2.0:1788880387.713625:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:3.0:1788880387.713626:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:1.0:1788880387.713627:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000010:2.0:1788880387.713632:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dc9bac80. 00000100:00000010:3.0:1788880387.713632:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:1.0:1788880387.713633:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880387.713638:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880387.713646:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a131edae 00000100:00000001:3.0:1788880387.713646:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:1.0:1788880387.713647:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880387.713652:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a131edae. 00000100:00000001:3.0:1788880387.713653:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880387.713661:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880387.713663:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880387.713669:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880387.713669:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:2.0:1788880387.713673:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880387.713675:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880387.713678:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880387.713684:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880387.713685:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880387.713688:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880387.713695:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880387.713716:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880387.713717:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780653568:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880387.713727:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880387.713733:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880387.713739:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000fd6b7ee1 x1875776780653568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880387.713755:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880387.713757:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880387.713763:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880387.713763:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880387.713768:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d653e4f6. 00000100:00000001:2.0:1788880387.713769:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880387.713773:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:2.0:1788880387.713774:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:1.0:1788880387.713775:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880387.713779:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880387.713780:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880387.713788:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880387.713789:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880387.713793:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008eccda10. 02000000:00000001:2.0:1788880387.713797:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880387.713798:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880387.713805:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00001000:2.0:1788880387.713806:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=65 v=5 (5 5 0 0) 00000100:00000010:1.0:1788880387.713807:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000fd6b7ee1. 00000100:00000001:1.0:1788880387.713814:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880387.713816:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880387.713817:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=39 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880387.713818:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880387.713829:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880387.713832:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880387.713833:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880387.713836:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880387.713836:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880387.713843:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00080000:2.0:1788880387.713845:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880387.713853:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880387.713873:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880387.713876:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880387.713877:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880387.713879:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880387.713880:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880387.713883:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880387.713887:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880387.713889:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880387.713890:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880387.713892:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880387.713896:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780653696:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880387.713899:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880387.713901:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880387.713902:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000175072aa x1875776780653696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880403 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880387.713905:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880387.713907:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880387.713909:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b2a347c7. 02000000:00000001:2.0:1788880387.713911:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880387.713913:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880387.713914:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880387.713916:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880387.713917:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5d1e8ed. 02000000:00000001:2.0:1788880387.713920:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880387.713921:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880387.713922:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000175072aa. 00000100:00000001:2.0:1788880387.713924:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880387.713924:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880387.713928:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880387.713947:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880387.713948:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880387.713949:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880387.713951:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880387.774322:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880387.774328:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880387.774334:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880387.774335:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880387.774389:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880387.774392:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880387.774394:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880388.798351:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880388.798358:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880388.798365:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880388.798366:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880388.798439:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880388.798445:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880388.798449:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880389.822388:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880389.822398:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880389.822405:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880389.822406:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880389.822467:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880389.822471:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880389.822474:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880390.847436:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880390.847444:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880390.847451:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880390.847452:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880390.847498:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880390.847503:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880390.847507:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880391.870314:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880391.870319:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880391.870322:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880391.870323:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880391.870359:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880391.870363:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880391.870365:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880392.830360:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306023887, at 4306023936 00000100:00000040:3.0:1788880392.830360:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306023887, at 4306023936 00000100:00000010:3.0:1788880392.830379:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009a4dee8c. 00000100:00000010:1.0:1788880392.830382:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009f22a9a0. 00000100:00000040:2.0:1788880392.830394:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306023887, at 4306023936 00000100:00000010:2.0:1788880392.830411:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000bc13d1e4. 00000100:00000040:0.0:1788880392.830489:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306023887, at 4306023936 00000100:00000010:0.0:1788880392.830496:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014321469. 00000100:00000040:0.0:1788880392.830506:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880392.830510:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880392.830519:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000802a821. 00000020:00000040:0.0:1788880392.830523:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880392.830528:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880392.830545:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880392.830549:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880392.830557:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000131c1a55. 00000100:00000001:0.0:1788880392.830564:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880392.830575:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880392.830587:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880392.830590:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880392.830597:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000802a821] to pc [ptlrpcd_00_00+0] req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880392.830641:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880392.830646:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880392.830655:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880392.830659:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880392.830661:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880392.830666:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880392.830668:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880392.830671:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880392.830674:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880392.830676:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880392.830679:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880392.830682:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780653824:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880392.830693:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880392.830700:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880392.830701:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880392.830705:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880392.830705:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880392.830709:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:3.0:1788880392.830709:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:0.0:1788880392.830713:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b8846eac. 02000000:00000010:3.0:1788880392.830714:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000063883087. 00000020:00000040:0.0:1788880392.830715:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:3.0:1788880392.830718:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880392.830719:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880392.830721:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880392.830723:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880392.830725:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000cbf1a475. 02000000:00000010:0.0:1788880392.830728:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000a371c7dc. 00000400:00000010:3.0:1788880392.830732:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005b101ac6. 00000100:00000001:0.0:1788880392.830734:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880392.830739:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780653824, portal 25 00000100:00000040:0.0:1788880392.830742:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880392.830743:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880392.830745:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849370368 : -126822860181248 : ffff8ca7c0783100) 00000100:00000040:3.0:1788880392.830752:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880408 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880392.830758:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880392.830761:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880392.830762:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880392.830766:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880392.830767:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000082116f14. 00000100:00000040:0.0:1788880392.830770:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b8846eac] to pc [ptlrpcd_00_01+1] req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880392.830771:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780653824, offset 0 00000400:00000200:3.0:1788880392.830780:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880392.830795:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880392.830800:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880392.830804:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880392.830808:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880392.830810:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880392.830812:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880392.830814:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880392.830817:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:3.0:1788880392.830820:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [178:178:256:4294967295] 192.168.204.111@tcp LPNI seq info [178:178:8:4294967295] 00000100:00000040:1.0:1788880392.830823:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880392.830851:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880392.830860:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880392.830870:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000010:3.0:1788880392.830870:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000100:00000001:1.0:1788880392.830872:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880392.830880:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:3.0:1788880392.830880:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880392.830883:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880392.830884:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880392.830886:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880392.830887:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880392.830890:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00100000:1.0:1788880392.830890:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780653952:192.168.204.111@tcp:400: 00000100:00000010:0.0:1788880392.830895:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000a2475c3c. 00000100:00000001:1.0:1788880392.830896:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:3.0:1788880392.830896:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000020:00000040:0.0:1788880392.830898:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880392.830899:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880392.830902:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880392.830903:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:1.0:1788880392.830906:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880392.830907:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880392.830909:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880392.830911:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880392.830911:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000001be0e14c. 02000000:00000001:1.0:1788880392.830912:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880392.830914:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000030024c2c. 00000400:00000010:1.0:1788880392.830916:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005972c901. 00000100:00000001:0.0:1788880392.830918:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880392.830920:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 00000100:00000001:3.0:1788880392.830922:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880392.830923:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000200:1.0:1788880392.830923:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780653952, portal 10 00000100:00000001:1.0:1788880392.830928:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880392.830929:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364096 : -126822860187520 : ffff8ca7c0781880) 00000100:00000001:3.0:1788880392.830930:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306029007 00000100:00000001:0.0:1788880392.830934:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000010:3.0:1788880392.830936:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009636b993. 00000100:00000001:0.0:1788880392.830937:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880392.830939:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880392.830942:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000040:0.0:1788880392.830943:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a2475c3c] to pc [ptlrpcd_00_02+2] req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880392.830949:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880392.830949:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306029007 00000100:00000040:1.0:1788880392.830954:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788880392.830954:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000eeb663f. 00000400:00000010:1.0:1788880392.830958:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a131edae. 00000100:00000001:3.0:1788880392.830959:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000200:1.0:1788880392.830963:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780653952, offset 0 00000100:00000001:3.0:1788880392.830965:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306029007 00000100:00000010:3.0:1788880392.830969:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f093018f. 00000100:00000001:0.0:1788880392.830970:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880392.830972:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880392.830973:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000040:0.0:1788880392.830974:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:1.0:1788880392.830978:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880392.830978:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306029007 00000100:00000040:0.0:1788880392.830979:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:1.0:1788880392.830983:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:3.0:1788880392.830984:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b0b8cfdf. 00000100:00000040:0.0:1788880392.830987:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11361) 00000400:00000200:1.0:1788880392.830987:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880392.830988:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788880392.830991:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880392.830995:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880392.831004:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880392.831008:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880392.831011:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [179:179:255:4294967295] 192.168.204.111@tcp LPNI seq info [179:179:7:4294967295] 00000100:00000001:3.0:1788880392.831011:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880392.831017:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880392.831017:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880392.831020:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880392.831023:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788880392.831024:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880392.831026:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880392.831029:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880392.831033:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880392.831038:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000200:1.0:1788880392.831039:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880392.831039:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880392.831040:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880392.831043:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880392.831044:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000010:1.0:1788880392.831045:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:3.0:1788880392.831046:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880392.831047:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880392.831051:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880392.831051:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880392.831053:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00100000:0.0:1788880392.831059:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780654080:192.168.204.111@tcp:400: 00000800:00000200:2.0:1788880392.831059:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:1.0:1788880392.831061:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880392.831067:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880392.831068:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000010:2.0:1788880392.831070:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 02000000:00000001:0.0:1788880392.831071:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880392.831072:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880392.831074:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880392.831078:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880392.831079:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 510580 usecs (511 jiffies) 02000000:00000001:0.0:1788880392.831080:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880392.831082:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880392.831083:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306024447 02000000:00000010:0.0:1788880392.831084:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005dd6b9fd. 00000400:00000200:2.0:1788880392.831085:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880392.831088:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306029007 02000000:00000001:0.0:1788880392.831096:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880392.831096:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306024447 00000100:00000001:1.0:1788880392.831099:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306029007 00000400:00000010:0.0:1788880392.831101:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033488580. 00000100:00000001:1.0:1788880392.831107:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306024447 00000400:00000010:0.0:1788880392.831109:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000400:00000200:2.0:1788880392.831110:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880392.831111:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306029007 00000100:00000200:0.0:1788880392.831113:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780654080, portal 10 00000100:00000001:0.0:1788880392.831118:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880392.831118:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306024447 00000100:00000001:0.0:1788880392.831120:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849361408 : -126822860190208 : ffff8ca7c0780e00) 00000100:00000001:1.0:1788880392.831122:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306029007 00000100:00000001:1.0:1788880392.831124:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880392.831126:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880392.831127:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000082116f14 00000100:00000040:0.0:1788880392.831129:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880392.831129:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880392.831134:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880392.831135:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000082116f14. 00000100:00000001:1.0:1788880392.831139:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880392.831143:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880392.831147:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880392.831147:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880392.831148:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880392.831149:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880392.831153:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880392.831155:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880392.831159:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005c796512. 00000100:00000001:1.0:1788880392.831159:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880392.831163:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780654080, offset 0 00000100:00000200:2.0:1788880392.831168:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880408 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880392.831176:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880392.831191:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880392.831198:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880392.831199:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1788880392.831206:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:2.0:1788880392.831210:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880408 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880392.831211:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880392.831216:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880392.831225:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [180:180:254:4294967295] 192.168.204.111@tcp LPNI seq info [180:180:6:4294967295] 00000400:00000200:0.0:1788880392.831265:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880392.831279:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880392.831279:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880392.831284:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000800:00000200:0.0:1788880392.831289:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880392.831295:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880392.831297:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880392.831299:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880392.831304:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880392.831307:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 31427 usecs (32 jiffies) 00000100:00000001:0.0:1788880392.831312:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306023969 00000100:00000001:0.0:1788880392.831314:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306029007 00000100:00000001:0.0:1788880392.831315:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306023969 00000100:00000001:0.0:1788880392.831322:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306029007 00000100:00000001:0.0:1788880392.831324:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306023969 00000100:00000001:0.0:1788880392.831325:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306029007 00000800:00000200:2.0:1788880392.831326:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880392.831327:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306023969 00000100:00000001:0.0:1788880392.831332:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306029007 00000100:00000001:0.0:1788880392.831335:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:2.0:1788880392.831336:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000100:00000001:0.0:1788880392.831338:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880392.831343:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880392.831347:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880392.831347:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880392.831350:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880392.831352:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880392.831355:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880392.831359:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880392.831360:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880392.831363:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880392.831364:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880392.831374:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a131edae 00000400:00000010:2.0:1788880392.831377:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a131edae. 00000100:00000001:2.0:1788880392.831381:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880392.831387:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880392.831396:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880392.831399:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880392.831405:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880392.831407:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880392.831445:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880392.831448:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000400:00000200:2.0:1788880392.831452:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880392.831455:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880392.831458:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005c796512 00000400:00000010:2.0:1788880392.831460:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005c796512. 00000100:00000001:2.0:1788880392.831463:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880392.831465:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880392.831473:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880392.831475:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880392.831480:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880392.831482:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880392.832356:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880392.832367:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880392.832370:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880392.832372:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880392.832379:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880392.832393:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b05500 00000400:00000200:2.0:1788880392.832399:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x675 [1] + 224 00000800:00000001:2.0:1788880392.832406:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880392.832411:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880392.832419:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880392.832424:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880392.832430:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880392.832434:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880392.832444:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880392.832459:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880392.832468:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880392.832471:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880392.832486:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880392.832489:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880392.832495:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880392.832497:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880392.832499:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000cbf1a475. 00000400:00000200:3.0:1788880392.832503:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005b101ac6 00000400:00000010:3.0:1788880392.832505:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005b101ac6. 00000100:00000001:3.0:1788880392.832508:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880392.832512:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880392.832520:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880392.832591:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880392.832593:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880392.832596:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880392.832599:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880392.832602:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880392.832604:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880392.832607:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880392.832614:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=70 v=5 (5 5 0 0) 00000800:00000001:0.0:1788880392.832615:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:3.0:1788880392.832620:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=70 v=5 (1 1 0 0) 00000800:00000001:0.0:1788880392.832623:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880392.832626:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880392.832628:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880392.832634:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880392.832642:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05580 00000400:00000200:0.0:1788880392.832647:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x67d [1] + 224 00000800:00000001:0.0:1788880392.832651:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880392.832656:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880392.832658:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880392.832661:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880392.832665:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880392.832669:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880392.832679:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880392.832707:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880392.832714:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880392.832718:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880392.832721:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880392.832724:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880392.832729:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880392.832735:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05600 00000400:00000200:0.0:1788880392.832741:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x685 [1] + 224 00000100:00000001:1.0:1788880392.832744:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:0.0:1788880392.832745:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880392.832750:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880392.832751:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880392.832755:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:1.0:1788880392.832756:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880392.832760:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:0.0:1788880392.832761:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:1.0:1788880392.832765:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005972c901. 00000400:00000200:1.0:1788880392.832770:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000400:00000010:1.0:1788880392.832778:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:0.0:1788880392.832782:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880392.832782:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880392.832788:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880392.832789:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880392.832808:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880392.832812:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880392.832817:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880392.832821:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880392.832824:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880392.832825:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880392.832827:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880392.832846:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880392.832870:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880392.832871:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880392.832873:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880392.832875:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880392.832877:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880392.832877:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880392.832878:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880392.832881:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:0.0:1788880392.832884:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880392.832884:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:3.0:1788880392.832885:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880392.832887:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880392.832890:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880392.832893:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880392.832902:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=60 v=5 (5 5 0 0) 00000100:00100000:3.0:1788880392.832903:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780653824:192.168.204.111@tcp:400: 00000100:00001000:1.0:1788880392.832906:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=60 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880392.832909:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880392.832911:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880392.832914:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:1.0:1788880392.832917:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880392.832918:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880392.832919:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880392.832920:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788880392.832923:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880392.832923:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880392.832924:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880392.832928:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:2.0:1788880392.832928:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:3.0:1788880392.832928:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000802a821 x1875776780653824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880392.832932:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033488580. 00000100:00000001:1.0:1788880392.832933:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880392.832935:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880392.832939:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000100:00000040:1.0:1788880392.832943:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880392.832943:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:3.0:1788880392.832946:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880392.832949:0:201287:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1788880392.832951:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880392.832955:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788880392.832955:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880392.832955:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000063883087. 00000100:00000001:1.0:1788880392.832960:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:3.0:1788880392.832962:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880392.832963:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880392.832967:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:1.0:1788880392.832971:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:3.0:1788880392.832971:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000200:2.0:1788880392.832972:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880392.832976:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880392.832981:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000131c1a55. 02000000:00000001:3.0:1788880392.832987:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880392.832991:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880392.832994:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000802a821. 00000100:00000001:2.0:1788880392.832997:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:1.0:1788880392.833000:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780653952:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880392.833001:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880392.833001:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880392.833005:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880392.833006:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880392.833008:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880392.833009:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880392.833010:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880392.833011:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880392.833014:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880392.833016:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:2.0:1788880392.833017:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880392.833017:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880392.833020:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880392.833021:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b8846eac x1875776780653952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880392.833026:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:2.0:1788880392.833029:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=70 v=5 (5 5 0 0) 00000100:00000001:3.0:1788880392.833029:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880392.833033:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00001000:2.0:1788880392.833036:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=44 v=5 (1 1 0 0) 02000000:00000001:1.0:1788880392.833037:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880392.833039:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000001be0e14c. 00000100:00000001:2.0:1788880392.833042:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880392.833043:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880392.833044:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00010000:00000001:2.0:1788880392.833046:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:1.0:1788880392.833047:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880392.833049:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880392.833053:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00080000:2.0:1788880392.833053:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880392.833055:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000a371c7dc. 02000000:00000001:1.0:1788880392.833059:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880392.833060:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880392.833061:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b8846eac. 00000100:00000001:1.0:1788880392.833063:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880392.833063:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880392.833064:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880392.833069:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:2.0:1788880392.833070:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880392.833071:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880392.833074:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:1.0:1788880392.833075:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880392.833078:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880392.833078:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880392.833082:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880392.833084:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880392.833086:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880392.833091:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880392.833093:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880392.833093:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880392.833095:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880392.833099:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780654080:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880392.833102:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880392.833103:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880392.833104:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000a2475c3c x1875776780654080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880408 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880392.833108:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880392.833109:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880392.833110:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005dd6b9fd. 02000000:00000001:2.0:1788880392.833113:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880392.833114:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880392.833115:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880392.833117:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880392.833121:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000030024c2c. 02000000:00000001:2.0:1788880392.833123:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880392.833124:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880392.833125:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000a2475c3c. 00000100:00000001:2.0:1788880392.833127:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880392.833128:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880392.833129:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880392.833132:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880392.833133:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880392.833134:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880392.833135:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880392.894366:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880392.894374:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880392.894380:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880392.894380:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880392.894444:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880392.894452:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880392.894455:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880393.918352:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880393.918362:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880393.918364:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880393.918367:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880393.918371:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880393.918373:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880393.918375:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880393.918377:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880393.918381:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880393.918382:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880393.918442:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880393.918446:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880393.918449:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880394.942367:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880394.942378:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880394.942414:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880394.942416:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880394.942458:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880394.942463:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880394.942467:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880395.966368:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880395.966378:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880395.966386:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880395.966390:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880395.966392:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880395.966396:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880395.966400:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880395.966403:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880395.966404:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880395.966407:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880395.966449:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880395.966453:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880395.966456:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880396.990375:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880396.990382:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880396.990388:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880396.990388:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880396.990452:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880396.990456:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880396.990458:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880397.950383:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306029007, at 4306029056 00000100:00000040:1.0:1788880397.950383:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306029007, at 4306029056 00000100:00000040:2.0:1788880397.950383:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880397.950397:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880397.950404:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000eeb663f. 00000100:00000010:0.0:1788880397.950409:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009636b993. 00000100:00000010:2.0:1788880397.950410:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 00000020:00000040:2.0:1788880397.950418:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880397.950424:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880397.950429:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880397.950436:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880397.950446:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5d1e8ed. 00000100:00000040:3.0:1788880397.950451:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306029007, at 4306029056 00000100:00000001:2.0:1788880397.950456:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880397.950471:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b0b8cfdf. 00000100:00000040:2.0:1788880397.950474:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880397.950486:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880397.950505:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880397.950512:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_03+3] req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880397.950537:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880397.950539:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880397.950542:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880397.950544:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880397.950546:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880397.950548:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880397.950549:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880397.950551:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880397.950553:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000001:2.0:1788880397.950557:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880397.950560:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880397.950565:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880397.950565:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880397.950568:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880397.950571:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000040:2.0:1788880397.950572:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_00+0] req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880397.950576:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880397.950588:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880397.950591:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880397.950594:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880397.950596:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880397.950598:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880397.950599:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880397.950604:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880397.950606:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880397.950610:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880397.950615:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000020:00000040:2.0:1788880397.950619:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880397.950626:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880397.950629:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880397.950629:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880397.950634:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880397.950635:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880397.950635:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880397.950635:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880397.950637:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880397.950642:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:2.0:1788880397.950647:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008eccda10. 02000000:00000001:0.0:1788880397.950651:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880397.950652:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880397.950653:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880397.950653:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880397.950656:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880397.950664:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880397.950664:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880397.950668:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880397.950670:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880397.950673:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780654208:192.168.204.111@tcp:400: 02000000:00000001:1.0:1788880397.950675:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880397.950678:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880397.950682:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880397.950687:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880397.950689:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:1.0:1788880397.950691:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780654336:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880397.950691:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880397.950693:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880397.950695:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_01+1] req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880397.950696:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880397.950698:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880397.950700:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880397.950702:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880397.950705:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:1.0:1788880397.950705:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880397.950707:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880397.950710:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:1.0:1788880397.950712:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000039ebda7d. 02000000:00000010:0.0:1788880397.950713:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b476d059. 02000000:00000001:1.0:1788880397.950715:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880397.950715:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:0.0:1788880397.950718:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880397.950723:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11366) 00000400:00000010:0.0:1788880397.950728:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007695b617. 00000400:00000010:1.0:1788880397.950728:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005972c901. 00000400:00000010:0.0:1788880397.950748:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 00000400:00000010:1.0:1788880397.950749:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 00000100:00000040:2.0:1788880397.950749:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306029007, at 4306029056 00000100:00000010:2.0:1788880397.950759:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f093018f. 00000100:00000200:0.0:1788880397.950761:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780654208, portal 25 00000100:00000200:1.0:1788880397.950761:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780654336, portal 10 00000100:00000001:1.0:1788880397.950768:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880397.950770:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000100:00000001:0.0:1788880397.950774:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880397.950777:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938435456 : -126823771116160 : ffff8ca78a2c6d80) 00000100:00000040:1.0:1788880397.950781:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880397.950786:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880413 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880397.950804:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880397.950804:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880397.950804:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880397.950808:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880397.950811:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880397.950811:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880397.950813:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880397.950816:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009822ffbc. 00000400:00000010:0.0:1788880397.950817:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e9812fb0. 00000100:00000040:2.0:1788880397.950818:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880397.950819:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780654336, offset 0 00000100:00000200:0.0:1788880397.950821:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780654208, offset 0 00000400:00000200:1.0:1788880397.950828:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880397.950829:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880397.950831:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880397.950832:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880397.950834:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880397.950837:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880397.950838:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880397.950843:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780654464:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880397.950845:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880397.950846:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880397.950848:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880397.950850:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880397.950851:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:2.0:1788880397.950851:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880397.950855:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002f5492f5. 00000400:00000200:1.0:1788880397.950856:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:2.0:1788880397.950856:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880397.950860:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:2.0:1788880397.950860:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007593939c. 00000400:00000010:2.0:1788880397.950863:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008f790e4a. 00000400:00000200:1.0:1788880397.950864:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880397.950868:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [181:181:256:4294967295] 192.168.204.111@tcp LPNI seq info [181:181:8:4294967295] 00000100:00000200:2.0:1788880397.950871:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780654464, portal 10 00000100:00000001:2.0:1788880397.950872:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880397.950873:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 00000100:00000040:2.0:1788880397.950878:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880397.950879:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880397.950885:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:1.0:1788880397.950886:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880397.950889:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:2.0:1788880397.950890:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880397.950892:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000400:00000010:2.0:1788880397.950894:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001c5b8d0f. 00000100:00000200:2.0:1788880397.950897:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780654464, offset 0 00000400:00000200:0.0:1788880397.950898:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:1.0:1788880397.950903:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880397.950905:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880397.950905:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880397.950911:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:1.0:1788880397.950914:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880397.950918:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880397.950921:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880397.950925:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [182:182:255:4294967295] 192.168.204.111@tcp LPNI seq info [182:182:7:4294967295] 00000400:00000200:0.0:1788880397.950934:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880397.950938:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880397.950944:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 500620 usecs (501 jiffies) 00000800:00000200:0.0:1788880397.950947:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880397.950948:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880397.950950:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306029557 00000800:00000010:0.0:1788880397.950953:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000100:00000010:1.0:1788880397.950954:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000400:00000200:2.0:1788880397.950954:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1788880397.950960:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880397.950961:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000400:00000200:2.0:1788880397.950961:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880397.950966:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880397.950969:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306029557 00000800:00000200:0.0:1788880397.950970:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880397.950971:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788880397.950977:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000100:00000010:1.0:1788880397.950978:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000400:00000200:2.0:1788880397.950978:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [183:183:254:4294967295] 192.168.204.111@tcp LPNI seq info [183:183:6:4294967295] 00000100:00000001:1.0:1788880397.950983:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880397.950986:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306029557 00000400:00000200:2.0:1788880397.950987:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880397.950990:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880397.950990:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000800:00000200:2.0:1788880397.950991:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880397.950993:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000800:00000010:2.0:1788880397.950995:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f9812869. 00000100:00000001:0.0:1788880397.951000:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306034127 00000100:00000001:1.0:1788880397.951001:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306029557 00000800:00000200:2.0:1788880397.951002:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:1.0:1788880397.951007:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000800:00000200:3.0:1788880397.951009:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:2.0:1788880397.951010:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880397.951011:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306029557 00000100:00000001:1.0:1788880397.951011:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000800:00000200:2.0:1788880397.951018:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f9812869 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880397.951021:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:3.0:1788880397.951022:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000001:0.0:1788880397.951023:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306034127 00000100:00000001:2.0:1788880397.951026:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880397.951034:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 30813 usecs (31 jiffies) 00000100:00000001:1.0:1788880397.951036:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880397.951038:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306029557 00000100:00000001:2.0:1788880397.951038:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306029087 00000100:00000001:1.0:1788880397.951040:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880397.951041:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306034127 00000100:00000001:1.0:1788880397.951043:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880397.951045:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306029087 00000100:00000001:0.0:1788880397.951047:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306034127 00000100:00000001:1.0:1788880397.951048:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880397.951048:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306034127 00000100:00000001:1.0:1788880397.951050:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:3.0:1788880397.951051:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880397.951054:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306029087 00000100:00000001:1.0:1788880397.951055:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880397.951056:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306029557 00000100:00000001:1.0:1788880397.951058:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880397.951062:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880397.951063:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306034127 00000100:00000001:2.0:1788880397.951064:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306034127 00000100:00000001:1.0:1788880397.951067:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880397.951073:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880397.951075:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306029087 00000100:00000001:0.0:1788880397.951077:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306029557 00000100:00000001:0.0:1788880397.951085:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880397.951086:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306034127 00000400:00000200:3.0:1788880397.951087:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009822ffbc 00000100:00000001:0.0:1788880397.951092:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880397.951094:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009822ffbc. 00000100:00000001:0.0:1788880397.951098:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880397.951100:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880397.951102:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880397.951105:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880397.951109:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880397.951116:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880397.951123:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880397.951124:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880397.951126:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880397.951130:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880397.951131:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880397.951132:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880397.951134:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880397.951137:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880397.951159:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880397.951165:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880397.951173:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880397.951175:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880397.951192:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880397.951194:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000400:00000200:3.0:1788880397.951198:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880397.951201:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880397.951203:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e9812fb0 00000400:00000010:3.0:1788880397.951205:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e9812fb0. 00000100:00000001:3.0:1788880397.951208:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880397.951211:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880413 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880397.951218:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880397.951220:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880413 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880397.951226:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880397.951228:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880397.951290:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880397.951293:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f9812869. 00000400:00000200:3.0:1788880397.951296:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880397.951299:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880397.951302:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001c5b8d0f 00000400:00000010:3.0:1788880397.951304:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001c5b8d0f. 00000100:00000001:3.0:1788880397.951306:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880397.951309:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880397.951317:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880397.951319:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880397.951331:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880397.951333:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880397.952827:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880397.952841:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880397.952844:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880397.952845:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880397.952854:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880397.952866:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05700 00000400:00000200:2.0:1788880397.952874:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x691 [1] + 224 00000800:00000001:2.0:1788880397.952880:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880397.952882:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880397.952884:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880397.952887:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880397.952891:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880397.952896:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880397.952903:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880397.952916:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880397.952925:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880397.952928:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880397.952938:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880397.952940:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880397.952944:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880397.952950:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b05680 00000100:00000001:1.0:1788880397.952952:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880397.952955:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880397.952955:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x68d [1] + 224 00000800:00000001:2.0:1788880397.952959:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880397.952961:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880397.952962:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880397.952963:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:1.0:1788880397.952964:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880397.952966:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005972c901. 00000400:00000200:2.0:1788880397.952966:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880397.952970:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000400:00000010:1.0:1788880397.952972:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:1.0:1788880397.952975:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880397.952976:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880397.952978:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880397.952979:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880397.952986:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880397.952987:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880397.952994:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880397.952995:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880397.952997:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880397.953002:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880397.953006:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788880397.953006:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880397.953008:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880397.953009:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880397.953010:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880397.953011:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880397.953012:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880397.953014:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880397.953016:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880397.953021:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05780 00000100:00001000:1.0:1788880397.953023:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=65 v=5 (5 5 0 0) 00000400:00000200:2.0:1788880397.953025:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x69d [1] + 224 00000100:00001000:1.0:1788880397.953027:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=65 v=5 (1 1 0 0) 00000800:00000001:2.0:1788880397.953029:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880397.953030:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880397.953031:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880397.953032:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880397.953034:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:1.0:1788880397.953037:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788880397.953037:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880397.953039:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880397.953040:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:0.0:1788880397.953041:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00010000:00000001:1.0:1788880397.953042:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880397.953045:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007695b617. 00000400:00000200:2.0:1788880397.953046:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880397.953047:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000200:0.0:1788880397.953050:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8884ddc 00000100:00000040:1.0:1788880397.953051:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000010:0.0:1788880397.953054:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8884ddc. 00000100:00000001:1.0:1788880397.953055:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880397.953057:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880397.953058:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880397.953059:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880397.953063:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880397.953065:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880397.953065:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880397.953086:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880397.953087:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880397.953090:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880397.953092:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880397.953096:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880397.953102:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880397.953113:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780654336:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880397.953122:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880397.953127:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880397.953131:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880397.953131:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780654336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880397.953134:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880397.953138:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880397.953141:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880397.953143:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880397.953145:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880397.953145:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880397.953146:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880397.953149:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:0.0:1788880397.953150:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880397.953154:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=75 v=5 (5 5 0 0) 02000000:00000010:1.0:1788880397.953154:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000039ebda7d. 00000800:00000001:2.0:1788880397.953154:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880397.953159:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880397.953160:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=75 v=5 (1 1 0 0) 02000000:00000001:1.0:1788880397.953162:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880397.953166:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000020:00000001:1.0:1788880397.953166:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880397.953167:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880397.953171:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00010000:00000001:0.0:1788880397.953172:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880397.953174:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880397.953177:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880397.953177:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000040:0.0:1788880397.953183:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880397.953186:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 02000000:00000001:1.0:1788880397.953193:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880397.953194:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880397.953196:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880397.953198:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000010:1.0:1788880397.953200:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:0.0:1788880397.953201:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880397.953204:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880397.953208:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:0.0:1788880397.953209:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880397.953210:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880397.953212:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880397.953215:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880397.953225:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00100000:0.0:1788880397.953226:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780654208:192.168.204.111@tcp:400: 00000400:00000001:3.0:1788880397.953226:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880397.953227:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880397.953264:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880397.953268:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880397.953270:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880397.953272:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880397.953273:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880397.953277:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000175072aa x1875776780654208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880397.953285:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000010:3.0:1788880397.953286:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007593939c. 02000000:00000001:0.0:1788880397.953289:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880397.953293:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b476d059. 02000000:00000001:0.0:1788880397.953296:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880397.953298:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000200:3.0:1788880397.953298:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008f790e4a 00000020:00000040:0.0:1788880397.953302:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000010:3.0:1788880397.953305:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008f790e4a. 00000020:00000001:0.0:1788880397.953313:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880397.953315:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5d1e8ed. 00000100:00000001:3.0:1788880397.953316:0:201286:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:0.0:1788880397.953318:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880397.953321:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880397.953323:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000175072aa. 00000100:00000001:0.0:1788880397.953326:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880397.953327:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880397.953329:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880397.953331:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880397.953334:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880397.953337:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880397.953341:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880397.953345:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880397.953359:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880397.953368:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880397.953370:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880397.953373:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880397.953376:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880397.953379:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880397.953381:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880397.953384:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880397.953393:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=75 v=5 (5 5 0 0) 00000100:00001000:3.0:1788880397.953397:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=49 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880397.953399:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880397.953401:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880397.953403:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880397.953408:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880397.953414:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880397.953418:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880397.953419:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880397.953421:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880397.953422:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880397.953434:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880397.953441:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880397.953445:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880397.953447:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880397.953450:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880397.953457:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780654464:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880397.953463:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880397.953473:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880397.953475:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780654464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880413 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880397.953481:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880397.953483:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880397.953486:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002f5492f5. 02000000:00000001:3.0:1788880397.953490:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880397.953492:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880397.953494:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880397.953496:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880397.953498:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008eccda10. 02000000:00000001:3.0:1788880397.953502:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880397.953504:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880397.953505:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000001:3.0:1788880397.953509:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880397.953510:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880397.953512:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880397.953519:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880397.953521:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880397.953523:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880397.953526:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880398.014370:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880398.014380:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880398.014390:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880398.014391:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880398.014433:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880398.014438:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880398.014442:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880399.038473:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880399.038483:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880399.038491:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880399.038492:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880399.038589:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880399.038596:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880399.038601:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880400.062352:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880400.062362:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880400.062369:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880400.062371:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880400.062417:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880400.062425:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880400.062430:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880401.086495:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880401.086503:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880401.086509:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880401.086510:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880401.086555:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880401.086559:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880401.086562:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880402.110333:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880402.110338:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880402.110343:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880402.110344:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880402.110395:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880402.110398:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880402.110400:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788880403.070359:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306034127, at 4306034176 00000100:00000040:1.0:1788880403.070360:0:182955:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306034127, at 4306034176 00000100:00000010:1.0:1788880403.070378:0:182955:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000010:3.0:1788880403.070386:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000040:0.0:1788880403.070470:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306034127, at 4306034176 00000100:00000010:0.0:1788880403.070485:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000040:2.0:1788880403.070597:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306034127, at 4306034176 00000100:00000010:2.0:1788880403.070605:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000040:0.0:1788880403.070615:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880403.070622:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880403.070634:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000007f91037. 00000020:00000040:0.0:1788880403.070650:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880403.070657:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880403.070660:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880403.070667:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880403.070678:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000a371c7dc. 00000100:00000001:0.0:1788880403.070687:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880403.070700:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880403.070712:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880403.070715:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880403.070722:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000007f91037] to pc [ptlrpcd_00_02+2] req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880403.070753:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880403.070756:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880403.070760:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880403.070763:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000937cf312. 00000100:00000001:2.0:1788880403.070779:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880403.070785:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:0.0:1788880403.070786:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880403.070790:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880403.070791:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880403.070793:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880403.070795:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880403.070799:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880403.070800:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880403.070805:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000131c1a55. 00000100:00000040:2.0:1788880403.070805:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880403.070813:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880403.070821:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:0.0:1788880403.070822:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880403.070825:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880403.070833:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880403.070834:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880403.070836:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880403.070837:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880403.070842:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000937cf312] to pc [ptlrpcd_00_03+3] req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880403.070843:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880403.070850:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780654592:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880403.070859:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880403.070866:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880403.070868:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880403.070869:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880403.070874:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880403.070878:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:2.0:1788880403.070881:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:0.0:1788880403.070882:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000451fe31a. 00000020:00000040:0.0:1788880403.070887:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000010:2.0:1788880403.070889:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000c4cba31. 00000100:00000001:0.0:1788880403.070890:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880403.070896:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880403.070896:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880403.070896:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880403.070903:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880403.070908:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:0.0:1788880403.070911:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006df406c6. 00000400:00000010:2.0:1788880403.070911:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000011bf8993. 00000100:00000001:3.0:1788880403.070913:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880403.070917:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880403.070929:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:2.0:1788880403.070929:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001e963647. 00000100:00000040:3.0:1788880403.070933:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880403.070948:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780654592, portal 25 00000100:00000001:2.0:1788880403.070958:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880403.070962:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849371264 : -126822860180352 : ffff8ca7c0783480) 00000100:00000001:0.0:1788880403.070976:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:2.0:1788880403.070977:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880419 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880403.070982:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880403.070995:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000451fe31a] to pc [ptlrpcd_00_00+0] req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880403.071002:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880403.071011:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880403.071014:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880403.071026:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880403.071027:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880403.071033:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880403.071034:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c3a5d4ae. 00000100:00000200:2.0:1788880403.071042:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780654592, offset 0 02000000:00000001:3.0:1788880403.071043:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880403.071053:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880403.071059:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780654720:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880403.071060:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:2.0:1788880403.071063:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880403.071077:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:0.0:1788880403.071078:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:3.0:1788880403.071087:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1788880403.071088:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:0.0:1788880403.071089:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11371) 02000000:00000001:3.0:1788880403.071093:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880403.071099:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880403.071100:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:3.0:1788880403.071109:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a4a3549f. 00000400:00000200:2.0:1788880403.071117:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880403.071119:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880403.071126:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880403.071128:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880403.071129:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000cbf1a475. 00000100:00100000:0.0:1788880403.071133:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000400:00000200:2.0:1788880403.071133:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880403.071145:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [184:184:256:4294967295] 192.168.204.111@tcp LPNI seq info [184:184:8:4294967295] 00000400:00000010:3.0:1788880403.071147:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005b101ac6. 00000100:00000001:0.0:1788880403.071149:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880403.071165:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780654720, portal 10 00000400:00000200:2.0:1788880403.071169:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880403.071172:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880403.071173:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880403.071182:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849367680 : -126822860183936 : ffff8ca7c0782680) 00000100:00000001:0.0:1788880403.071183:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880403.071190:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:2.0:1788880403.071190:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880403.071194:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880403.071197:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880403.071202:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880403.071207:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:2.0:1788880403.071207:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003139f6ba. 00000800:00000200:2.0:1788880403.071271:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880403.071291:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880403.071292:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880403.071303:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880403.071317:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1788880403.071320:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003139f6ba type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880403.071325:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880403.071329:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880403.071334:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880403.071339:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880403.071343:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880403.071346:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880403.071350:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780654848:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880403.071351:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880403.071363:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:3.0:1788880403.071364:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ec90f18d. 00000100:00000001:2.0:1788880403.071368:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306039248 02000000:00000001:0.0:1788880403.071369:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880403.071376:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880403.071377:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780654720, offset 0 00000100:00000010:2.0:1788880403.071380:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006738d3fe. 02000000:00000001:0.0:1788880403.071382:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880403.071395:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:2.0:1788880403.071405:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306039248 00000400:00000200:3.0:1788880403.071405:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000010:0.0:1788880403.071408:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000c9a81c5. 00000100:00000010:2.0:1788880403.071415:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e91f0731. 02000000:00000001:0.0:1788880403.071418:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880403.071423:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000010:0.0:1788880403.071425:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007695b617. 00000100:00000001:2.0:1788880403.071433:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306039248 00000400:00000200:3.0:1788880403.071437:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:0.0:1788880403.071438:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 00000100:00000200:0.0:1788880403.071450:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780654848, portal 10 00000100:00000010:2.0:1788880403.071452:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ccadc63b. 00000400:00000200:3.0:1788880403.071456:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880403.071457:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880403.071459:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880403.071466:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849366784 : -126822860184832 : ffff8ca7c0782300) 00000100:00000001:2.0:1788880403.071470:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306039248 00000400:00000200:3.0:1788880403.071474:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:0.0:1788880403.071477:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788880403.071478:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000076a63e17. 00000400:00000200:3.0:1788880403.071486:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:1.0:1788880403.071489:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880403.071489:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788880403.071502:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880403.071507:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:1.0:1788880403.071508:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003139f6ba. 00000100:00000001:0.0:1788880403.071509:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880403.071517:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880403.071518:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [185:185:255:4294967295] 192.168.204.111@tcp LPNI seq info [185:185:7:4294967295] 00000400:00000200:1.0:1788880403.071521:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:0.0:1788880403.071525:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880403.071532:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f34d0caa. 00000100:00000001:2.0:1788880403.071535:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880403.071543:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880403.071544:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780654848, offset 0 00000400:00000200:1.0:1788880403.071558:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880403.071559:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:3.0:1788880403.071560:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:0.0:1788880403.071569:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880403.071569:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880403.071578:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880403.071581:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880403.071582:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880403.071585:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880403.071662:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880403.071664:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c3a5d4ae 00000400:00000200:0.0:1788880403.071667:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:1.0:1788880403.071676:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c3a5d4ae. 00000800:00000010:3.0:1788880403.071676:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000400:00000200:0.0:1788880403.071682:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880403.071689:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:0.0:1788880403.071692:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:1.0:1788880403.071701:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880419 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880403.071701:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880403.071704:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880403.071715:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788880403.071725:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880403.071734:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [186:186:254:4294967295] 192.168.204.111@tcp LPNI seq info [186:186:6:4294967295] 00000800:00000200:3.0:1788880403.071742:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880403.071755:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880403.071756:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880403.071766:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 491050 usecs (492 jiffies) 00000100:00000040:1.0:1788880403.071768:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880419 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880403.071772:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880403.071778:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306034669 00000100:00000001:3.0:1788880403.071791:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306039248 00000800:00000200:0.0:1788880403.071794:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880403.071801:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306034669 00000100:00000001:1.0:1788880403.071805:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880403.071806:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000100:00000001:1.0:1788880403.071809:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880403.071815:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306039248 00000800:00000200:0.0:1788880403.071818:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880403.071824:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306034669 00000800:00000200:0.0:1788880403.071830:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880403.071830:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306039248 00000800:00000200:0.0:1788880403.071836:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880403.071840:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306034669 00000100:00000001:0.0:1788880403.071843:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880403.071847:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 30241 usecs (31 jiffies) 00000100:00000001:3.0:1788880403.071849:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306039248 00000100:00000001:0.0:1788880403.071853:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306034208 00000100:00000001:3.0:1788880403.071855:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880403.071861:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306039248 00000100:00000001:3.0:1788880403.071862:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880403.071867:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306034208 00000800:00000200:1.0:1788880403.071869:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880403.071870:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880403.071871:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306039248 00000800:00000010:1.0:1788880403.071874:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000100:00000001:0.0:1788880403.071876:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306034208 00000100:00000001:0.0:1788880403.071883:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306039248 00000400:00000200:1.0:1788880403.071886:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880403.071886:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880403.071887:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306034208 00000100:00000001:0.0:1788880403.071898:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306039248 00000100:00000001:3.0:1788880403.071903:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880403.071905:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880403.071907:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880403.071909:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880403.071916:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880403.071918:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880403.071919:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ec90f18d 00000100:00000001:3.0:1788880403.071922:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:1.0:1788880403.071923:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ec90f18d. 00000100:00000001:3.0:1788880403.071927:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880403.071931:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880403.071933:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880403.071933:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880403.071938:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880403.071945:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880403.071957:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880403.071960:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880403.071966:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880403.071968:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880403.071981:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880403.071983:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000400:00000200:1.0:1788880403.071987:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880403.071990:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880403.071992:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f34d0caa 00000400:00000010:1.0:1788880403.071994:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f34d0caa. 00000100:00000001:1.0:1788880403.071996:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880403.071999:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880403.072012:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880403.072015:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880403.072020:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880403.072021:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880403.073453:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880403.073462:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880403.073464:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880403.073466:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880403.073472:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880403.073513:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05880 00000400:00000200:3.0:1788880403.073531:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x6ad [1] + 224 00000800:00000001:3.0:1788880403.073538:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880403.073545:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880403.073547:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880403.073550:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880403.073555:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880403.073560:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880403.073569:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880403.073593:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880403.073630:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880403.073635:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880403.073690:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880403.073695:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880403.073697:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880403.073699:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880403.073704:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:2.0:1788880403.073708:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880403.073712:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:3.0:1788880403.073712:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05900 00000400:00000010:2.0:1788880403.073716:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000cbf1a475. 00000400:00000200:2.0:1788880403.073722:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005b101ac6 00000400:00000010:2.0:1788880403.073723:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005b101ac6. 00000100:00000001:2.0:1788880403.073726:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:3.0:1788880403.073731:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x6b5 [1] + 224 00000100:00000200:2.0:1788880403.073732:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880403.073740:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:2.0:1788880403.073747:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880403.073763:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880403.073765:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880403.073770:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880403.073773:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880403.073776:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880403.073778:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880403.073781:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880403.073784:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880403.073790:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=71 v=5 (5 5 0 0) 00000400:00000200:3.0:1788880403.073796:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00001000:2.0:1788880403.073804:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=71 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880403.073808:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880403.073809:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880403.073812:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:3.0:1788880403.073814:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00010000:00000001:2.0:1788880403.073816:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880403.073822:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880403.073823:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:2.0:1788880403.073824:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880403.073828:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880403.073830:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880403.073831:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880403.073839:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880403.073850:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880403.073851:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880403.073853:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880403.073855:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880403.073860:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880403.073872:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780654720:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880403.073875:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880403.073877:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880403.073878:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000937cf312 x1875776780654720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880403.073883:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880403.073883:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:2.0:1788880403.073885:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880403.073889:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a4a3549f. 00000800:00000001:3.0:1788880403.073891:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:2.0:1788880403.073893:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000800:00000001:3.0:1788880403.073896:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788880403.073897:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880403.073900:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880403.073903:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880403.073905:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000131c1a55. 02000000:00000001:2.0:1788880403.073910:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880403.073912:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880403.073914:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000937cf312. 00000100:00000001:2.0:1788880403.073918:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880403.073925:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880403.073927:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880403.073929:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880403.073933:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880403.073933:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880403.073936:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880403.073937:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880403.073938:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000001:0.0:1788880403.073942:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880403.073943:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880403.073945:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007695b617. 00000400:00000200:0.0:1788880403.073955:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8884ddc 00000400:00000010:0.0:1788880403.073957:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8884ddc. 00000100:00000001:0.0:1788880403.073960:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880403.073965:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880403.073973:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880403.073981:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880403.073989:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880403.073991:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880403.073993:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880403.073994:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880403.073996:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880403.073998:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880403.074003:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=81 v=5 (5 5 0 0) 00000100:00001000:0.0:1788880403.074006:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=55 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880403.074008:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880403.074010:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880403.074012:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788880403.074015:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00080000:0.0:1788880403.074016:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880403.074022:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880403.074025:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:0.0:1788880403.074027:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880403.074028:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880403.074033:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880403.074036:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880403.074039:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880403.074040:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880403.074043:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880403.074044:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880403.074056:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b05800 00000400:00000200:2.0:1788880403.074067:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x6a5 [1] + 224 00000100:00000001:0.0:1788880403.074070:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:2.0:1788880403.074073:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880403.074074:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880403.074077:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880403.074081:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880403.074088:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780654848:192.168.204.111@tcp:400: 00000800:00000001:2.0:1788880403.074090:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880403.074093:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880403.074094:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880403.074098:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788880403.074103:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:0.0:1788880403.074104:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000451fe31a x1875776780654848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880403.074110:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880403.074115:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:2.0:1788880403.074116:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880403.074118:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880403.074121:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000c9a81c5. 02000000:00000001:0.0:1788880403.074126:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880403.074129:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880403.074131:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:2.0:1788880403.074132:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880403.074135:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880403.074138:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006df406c6. 02000000:00000001:0.0:1788880403.074143:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880403.074146:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880403.074148:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000451fe31a. 00000100:00000001:0.0:1788880403.074150:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880403.074151:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880403.074153:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880403.074156:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880403.074158:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880403.074161:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880403.074163:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880403.074164:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880403.074168:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880403.074169:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880403.074231:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880403.074255:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880403.074259:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880403.074260:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880403.074262:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000011bf8993. 00000400:00000200:3.0:1788880403.074265:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001e963647 00000400:00000010:3.0:1788880403.074267:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001e963647. 00000100:00000001:3.0:1788880403.074270:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880403.074274:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880403.074282:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880403.074289:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880403.074290:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880403.074292:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880403.074293:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880403.074295:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880403.074296:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880403.074298:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880403.074301:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=81 v=5 (5 5 0 0) 00000100:00001000:3.0:1788880403.074303:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=81 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880403.074305:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880403.074307:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880403.074308:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880403.074315:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880403.074317:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880403.074320:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880403.074327:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880403.074329:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880403.074330:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880403.074333:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880403.074339:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780654592:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880403.074344:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880403.074345:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880403.074348:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000007f91037 x1875776780654592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880419 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880403.074354:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880403.074355:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880403.074357:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000c4cba31. 02000000:00000001:3.0:1788880403.074360:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880403.074361:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880403.074362:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880403.074364:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880403.074365:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000a371c7dc. 02000000:00000001:3.0:1788880403.074368:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880403.074369:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880403.074370:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000007f91037. 00000100:00000001:3.0:1788880403.074373:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880403.074373:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880403.074375:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880403.074378:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880403.074379:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880403.074381:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880403.074382:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880403.134346:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880403.134352:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880403.134358:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880403.134359:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880403.134381:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880403.134385:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880403.134406:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880404.158371:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880404.158380:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880404.158383:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880404.158386:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880404.158391:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880404.158394:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880404.158396:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880404.158399:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880404.158404:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880404.158406:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880404.158464:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880404.158467:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880404.158469:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880405.182308:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880405.182316:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880405.182334:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880405.182335:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880405.182358:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880405.182363:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880405.182366:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880406.206354:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880406.206361:0:182955:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880406.206368:0:182955:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880406.206371:0:182955:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880406.206374:0:182955:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880406.206377:0:182955:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880406.206380:0:182955:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880406.206383:0:182955:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880406.206385:0:182955:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880406.206387:0:182955:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880406.206430:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880406.206434:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880406.206437:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880407.230335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880407.230341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880407.230367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880407.230369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880407.230400:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880407.230403:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880407.230405:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880408.190379:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306039248, at 4306039296 00000100:00000040:1.0:1788880408.190379:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306039248, at 4306039296 00000100:00000040:3.0:1788880408.190383:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306039248, at 4306039296 00000100:00000010:1.0:1788880408.190400:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e91f0731. 00000100:00000010:0.0:1788880408.190402:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006738d3fe. 00000100:00000010:3.0:1788880408.190415:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000076a63e17. 00000100:00000040:2.0:1788880408.190491:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880408.190505:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880408.190514:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002084a2b4. 00000020:00000040:2.0:1788880408.190519:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880408.190523:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880408.190526:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880408.190531:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880408.190549:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006aca3ff7. 00000100:00000001:2.0:1788880408.190557:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880408.190567:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880408.190577:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880408.190580:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880408.190586:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002084a2b4] to pc [ptlrpcd_00_01+1] req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880408.190611:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880408.190614:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880408.190619:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880408.190622:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000020:00000040:2.0:1788880408.190627:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880408.190629:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880408.190632:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880408.190632:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880408.190636:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880408.190639:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 00000100:00000001:3.0:1788880408.190642:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880408.190643:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880408.190650:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880408.190651:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880408.190655:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880408.190661:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:3.0:1788880408.190669:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880408.190698:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880408.190703:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880408.190707:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880408.190707:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880408.190719:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880408.190723:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_02+2] req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880408.190727:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880408.190737:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880408.190749:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780654976:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880408.190762:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880408.190771:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880408.190774:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880408.190777:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880408.190778:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880408.190783:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:3.0:1788880408.190785:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d9c9c3cb. 00000100:00000001:2.0:1788880408.190789:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:3.0:1788880408.190790:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880408.190795:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 00000100:00000001:0.0:1788880408.190797:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880408.190801:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000010:3.0:1788880408.190802:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000e0ef94e8. 00000100:00000001:0.0:1788880408.190803:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880408.190812:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880408.190814:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000010:3.0:1788880408.190816:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 02000000:00000001:2.0:1788880408.190817:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880408.190818:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:2.0:1788880408.190829:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000100:00000200:3.0:1788880408.190831:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780654976, portal 25 00000100:00000040:0.0:1788880408.190836:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880408.190836:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880408.190840:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880408.190845:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938429184 : -126823771122432 : ffff8ca78a2c5500) 00000100:00000040:2.0:1788880408.190847:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:3.0:1788880408.190863:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880424 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880408.190880:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880408.190892:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880408.190897:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880408.190897:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880408.190900:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880408.190908:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880408.190911:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_03+3] req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880408.190915:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880408.190917:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788880408.190922:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000035d99bc2. 02000000:00000001:0.0:1788880408.190927:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880408.190929:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780654976, offset 0 00000100:00100000:0.0:1788880408.190940:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780655104:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880408.190947:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880408.190947:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880408.190952:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880408.190956:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880408.190957:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880408.190961:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880408.190962:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:0.0:1788880408.190965:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880408.190970:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11376) 02000000:00000010:0.0:1788880408.190972:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000005842920. 00000400:00000200:3.0:1788880408.190976:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880408.190979:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880408.190986:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880408.190990:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880408.190992:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880408.190996:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007695b617. 00000100:00000040:2.0:1788880408.190997:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880408.191006:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 00000100:00000001:2.0:1788880408.191006:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880408.191009:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880408.191014:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780655104, portal 10 00000100:00000001:0.0:1788880408.191019:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:2.0:1788880408.191021:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880408.191022:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 02000000:00000001:2.0:1788880408.191025:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880408.191028:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880408.191032:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880408.191033:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780655232:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880408.191041:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880408.191045:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880408.191048:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880408.191049:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:2.0:1788880408.191053:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880408.191057:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:2.0:1788880408.191058:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000784e3b9. 00000400:00000010:0.0:1788880408.191062:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f34d0caa. 02000000:00000001:2.0:1788880408.191062:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880408.191067:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780655104, offset 0 00000400:00000010:2.0:1788880408.191069:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000deb55c04. 00000400:00000200:0.0:1788880408.191076:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880408.191091:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880408.191098:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880408.191105:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880408.191110:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880408.191117:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [187:187:256:4294967295] 192.168.204.111@tcp LPNI seq info [187:187:8:4294967295] 00000400:00000200:3.0:1788880408.191131:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880408.191137:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880408.191143:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000023cb0718. 00000800:00000010:3.0:1788880408.191144:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000100:00000200:2.0:1788880408.191150:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780655232, portal 10 00000100:00000001:2.0:1788880408.191155:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:3.0:1788880408.191158:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880408.191160:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000400:00000200:0.0:1788880408.191165:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880408.191175:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:2.0:1788880408.191175:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880408.191183:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880408.191193:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880408.191196:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880408.191203:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788880408.191211:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1788880408.191212:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880408.191217:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880408.191222:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [188:188:255:4294967295] 192.168.204.111@tcp LPNI seq info [188:188:7:4294967295] 00000100:00000001:3.0:1788880408.191222:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880408.191231:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306044367 00000100:00000010:3.0:1788880408.191266:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001632a9ba. 00000100:00000001:3.0:1788880408.191273:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880408.191280:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880408.191283:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306044368 00000800:00000200:0.0:1788880408.191289:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788880408.191289:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004800b45d. 00000100:00000001:3.0:1788880408.191297:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000010:0.0:1788880408.191299:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000100:00000001:3.0:1788880408.191304:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306044368 00000800:00000200:0.0:1788880408.191311:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880408.191312:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306039248 00000100:00000001:3.0:1788880408.191321:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306044368 00000800:00000200:0.0:1788880408.191329:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:3.0:1788880408.191329:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d6fe56. 00000800:00000200:0.0:1788880408.191339:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880408.191340:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880408.191347:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880408.191350:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880408.191353:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880408.191354:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880408.191357:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 481853 usecs (482 jiffies) 00000800:00000010:1.0:1788880408.191359:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000100:00000001:3.0:1788880408.191361:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880408.191365:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306039779 00000100:00000001:3.0:1788880408.191367:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880408.191369:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880408.191371:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306044367 00000100:00000001:0.0:1788880408.191380:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306039779 00000100:00000001:0.0:1788880408.191384:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306044368 00000400:00000200:1.0:1788880408.191386:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880408.191386:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880408.191388:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306039779 00000100:00000001:0.0:1788880408.191394:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306044368 00000100:00000001:0.0:1788880408.191397:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306039779 00000400:00000200:1.0:1788880408.191397:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000035d99bc2 00000100:00000001:3.0:1788880408.191399:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000010:1.0:1788880408.191402:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000035d99bc2. 00000100:00000001:0.0:1788880408.191403:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306044368 00000100:00000001:0.0:1788880408.191412:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880408.191412:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880408.191415:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880408.191416:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880408.191421:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880408.191422:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880424 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880408.191424:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880408.191434:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880408.191436:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880408.191437:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:1.0:1788880408.191442:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880424 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880408.191443:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880408.191445:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880408.191447:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880408.191452:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880408.191455:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880408.191457:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880408.191472:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880408.191475:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000400:00000200:1.0:1788880408.191481:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880408.191485:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880408.191487:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f34d0caa 00000400:00000010:1.0:1788880408.191488:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f34d0caa. 00000100:00000001:1.0:1788880408.191491:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880408.191493:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880408.191500:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880408.191502:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880408.191507:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880408.191508:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880408.191550:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880408.191553:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880408.191557:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880408.191588:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cc594c. 00000100:00000200:2.0:1788880408.191594:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780655232, offset 0 00000400:00000200:2.0:1788880408.191601:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880408.191607:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880408.191609:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880408.191612:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880408.191614:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880408.191616:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880408.191618:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [189:189:256:4294967295] 192.168.204.111@tcp LPNI seq info [189:189:8:4294967295] 00000400:00000200:2.0:1788880408.191627:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880408.191632:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880408.191635:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000800:00000200:2.0:1788880408.191639:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880408.191648:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880408.191651:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880408.191662:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880408.191665:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 29700 usecs (30 jiffies) 00000100:00000001:2.0:1788880408.191668:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306039327 00000100:00000001:2.0:1788880408.191670:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306044367 00000100:00000001:2.0:1788880408.191672:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306039327 00000100:00000001:2.0:1788880408.191675:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306044368 00000100:00000001:2.0:1788880408.191677:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306039327 00000100:00000001:2.0:1788880408.191679:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306044368 00000100:00000001:2.0:1788880408.191682:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306039327 00000100:00000001:2.0:1788880408.191683:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306044368 00000100:00000001:2.0:1788880408.191685:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880408.191687:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880408.191689:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880408.191692:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880408.191698:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880408.191700:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880408.191702:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880408.191703:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880408.191705:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880408.191707:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880408.191748:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880408.191753:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000400:00000200:3.0:1788880408.191758:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880408.191764:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880408.191769:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cc594c 00000400:00000010:3.0:1788880408.191771:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cc594c. 00000100:00000001:3.0:1788880408.191775:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880408.191779:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880408.191822:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880408.191826:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880408.191833:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880408.191834:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880408.193085:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880408.193108:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880408.193110:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880408.193113:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880408.193122:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880408.193134:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b05980 00000400:00000200:2.0:1788880408.193142:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x6bd [1] + 224 00000800:00000001:2.0:1788880408.193148:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880408.193158:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880408.193161:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880408.193165:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880408.193169:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880408.193174:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880408.193183:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880408.193202:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880408.193210:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880408.193213:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880408.193215:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880408.193216:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880408.193220:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880408.193225:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05a00 00000400:00000200:2.0:1788880408.193231:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x6c5 [1] + 224 00000100:00000001:3.0:1788880408.193267:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880408.193268:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880408.193273:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880408.193276:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880408.193280:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880408.193284:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880408.193290:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:3.0:1788880408.193293:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:2.0:1788880408.193294:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:3.0:1788880408.193300:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880408.193304:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000e0ef94e8. 00000400:00000200:3.0:1788880408.193312:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000400:00000010:3.0:1788880408.193316:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:3.0:1788880408.193323:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:2.0:1788880408.193324:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880408.193333:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880408.193361:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880408.193370:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880408.193370:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880408.193375:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880408.193378:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880408.193379:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880408.193385:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788880408.193386:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880408.193390:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880408.193391:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880408.193394:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880408.193397:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880408.193405:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880408.193413:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880408.193415:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05a80 02000000:00000001:3.0:1788880408.193415:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880408.193418:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880408.193426:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00001000:3.0:1788880408.193426:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=86 v=5 (5 5 0 0) 00000400:00000001:0.0:1788880408.193430:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:3.0:1788880408.193435:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=86 v=5 (1 1 0 0) 00000400:00000010:0.0:1788880408.193436:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007695b617. 00000100:00000001:3.0:1788880408.193448:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:0.0:1788880408.193449:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8884ddc 00000100:00000001:3.0:1788880408.193452:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880408.193454:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8884ddc. 00000100:00000001:0.0:1788880408.193458:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880408.193464:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x6cd [1] + 224 00010000:00000001:3.0:1788880408.193465:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000200:0.0:1788880408.193471:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880408.193472:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880408.193473:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880408.193483:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880408.193493:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880408.193497:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:3.0:1788880408.193498:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880408.193499:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880408.193515:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880408.193524:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880408.193555:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880408.193566:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880408.193579:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880408.193582:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880408.193583:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880408.193591:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880408.193593:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880408.193595:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880408.193602:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880408.193608:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880408.193618:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880408.193624:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880408.193636:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880408.193643:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780654976:192.168.204.111@tcp:400: 00000100:00001000:0.0:1788880408.193650:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=76 v=5 (5 5 0 0) 00000100:00000001:3.0:1788880408.193662:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880408.193663:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=76 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880408.193670:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880408.193672:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880408.193675:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880408.193676:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880408.193679:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002084a2b4 x1875776780654976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880408.193683:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788880408.193684:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880408.193691:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880408.193692:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880408.193696:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880408.193700:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880408.193704:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d9c9c3cb. 00000100:00000001:0.0:1788880408.193707:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:3.0:1788880408.193712:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:0.0:1788880408.193713:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880408.193717:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:3.0:1788880408.193719:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880408.193720:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000020:00000040:3.0:1788880408.193725:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880408.193726:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880408.193727:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000001:3.0:1788880408.193736:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000001:1.0:1788880408.193737:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:0.0:1788880408.193740:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880408.193740:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006aca3ff7. 00000400:00000001:1.0:1788880408.193742:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880408.193746:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000deb55c04. 02000000:00000001:3.0:1788880408.193751:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000200:1.0:1788880408.193753:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000023cb0718 02000000:00000001:3.0:1788880408.193757:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000010:1.0:1788880408.193762:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000023cb0718. 00000100:00000010:3.0:1788880408.193763:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002084a2b4. 00000100:00000001:1.0:1788880408.193773:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880408.193775:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880408.193775:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880408.193780:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880408.193782:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880408.193790:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880408.193795:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880408.193798:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880408.193808:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880408.193809:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880408.193813:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880408.193818:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:1.0:1788880408.193821:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880408.193824:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880408.193850:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:0.0:1788880408.193851:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780655104:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880408.193853:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880408.193856:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880408.193860:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880408.193862:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880408.193862:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880408.193865:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880408.193867:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:1.0:1788880408.193868:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880408.193873:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780655104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880408.193874:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=86 v=5 (5 5 0 0) 00000100:00001000:1.0:1788880408.193881:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=60 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880408.193884:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880408.193886:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880408.193887:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880408.193892:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880408.193901:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880408.193905:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00010000:00000001:1.0:1788880408.193905:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880408.193909:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000005842920. 00000100:00000001:1.0:1788880408.193910:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880408.193912:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880408.193915:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880408.193916:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880408.193917:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:1.0:1788880408.193921:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880408.193922:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880408.193926:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880408.193931:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880408.193933:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000010:0.0:1788880408.193935:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 00000100:00000001:1.0:1788880408.193936:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:0.0:1788880408.193940:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880408.193942:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880408.193944:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:0.0:1788880408.193945:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000100:00000040:1.0:1788880408.193950:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880408.193951:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880408.193953:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880408.193958:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1788880408.193965:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780655232:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880408.193966:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880408.193970:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880408.193971:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880408.193974:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880408.193975:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880408.193977:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880408.193980:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780655232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880424 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880408.193987:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880408.193990:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880408.193992:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000784e3b9. 02000000:00000001:1.0:1788880408.193995:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880408.193996:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880408.193998:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880408.193999:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880408.194001:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 02000000:00000001:1.0:1788880408.194003:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880408.194005:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880408.194006:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 00000100:00000001:1.0:1788880408.194008:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880408.194009:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880408.194011:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880408.194014:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880408.194015:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880408.194017:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880408.194019:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880408.254351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880408.254358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880408.254365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880408.254367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880408.254406:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880408.254410:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880408.254413:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880409.278373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880409.278384:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880409.278393:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880409.278395:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880409.278452:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880409.278458:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880409.278462:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880410.302370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880410.302402:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880410.302410:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880410.302411:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880410.302481:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880410.302488:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880410.302493:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880411.326355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880411.326362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880411.326368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880411.326369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880411.326401:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880411.326405:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880411.326408:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880412.350332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880412.350339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880412.350345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880412.350346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880412.350377:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880412.350381:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880412.350385:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880413.310401:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306044368, at 4306044416 00000100:00000040:2.0:1788880413.310401:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306044368, at 4306044416 00000100:00000040:3.0:1788880413.310401:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306044368, at 4306044416 00000100:00000040:0.0:1788880413.310404:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306044367, at 4306044416 00000100:00000010:0.0:1788880413.310429:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001632a9ba. 00000100:00000010:3.0:1788880413.310433:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d6fe56. 00000100:00000010:2.0:1788880413.310440:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ccadc63b. 00000100:00000010:1.0:1788880413.310452:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004800b45d. 00000100:00000040:3.0:1788880413.310470:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880413.310476:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880413.310484:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788880413.310490:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880413.310496:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880413.310499:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880413.310504:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880413.310513:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000055d7d92. 00000100:00000001:3.0:1788880413.310521:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880413.310533:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880413.310544:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880413.310547:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880413.310553:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_00+0] req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880413.310588:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880413.310591:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880413.310595:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880413.310601:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000100:00000001:0.0:1788880413.310603:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:3.0:1788880413.310604:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880413.310609:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880413.310611:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880413.310613:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880413.310617:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880413.310622:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:3.0:1788880413.310622:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:1.0:1788880413.310623:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880413.310634:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880413.310635:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880413.310640:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880413.310652:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880413.310655:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880413.310670:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880413.310675:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880413.310680:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880413.310689:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880413.310690:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880413.310695:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880413.310699:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780655360:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880413.310705:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_01+1] req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880413.310709:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880413.310722:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880413.310726:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880413.310729:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880413.310733:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880413.310734:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000026ba42a. 00000100:00000040:3.0:1788880413.310739:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880413.310740:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880413.310750:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000400:00000010:0.0:1788880413.310752:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007695b617. 00000100:00000010:3.0:1788880413.310754:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880413.310759:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880413.310762:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:0.0:1788880413.310763:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 02000000:00000001:3.0:1788880413.310765:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880413.310768:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880413.310770:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880413.310773:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:0.0:1788880413.310775:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780655360, portal 25 00000100:00000001:1.0:1788880413.310775:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:3.0:1788880413.310776:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:0.0:1788880413.310785:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:1.0:1788880413.310785:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880413.310786:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880413.310791:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000040:3.0:1788880413.310798:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:0.0:1788880413.310810:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880429 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880413.310829:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880413.310833:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880413.310835:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880413.310838:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880413.310845:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880413.310849:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880413.310853:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_02+2] req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880413.310859:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880413.310860:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880413.310874:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788880413.310879:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780655488:192.168.204.111@tcp:400: 00000400:00000010:0.0:1788880413.310880:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e9812fb0. 00000100:00000200:0.0:1788880413.310887:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780655360, offset 0 00000100:00000001:1.0:1788880413.310889:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880413.310897:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880413.310898:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880413.310900:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880413.310903:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880413.310907:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880413.310912:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:1.0:1788880413.310915:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880413.310920:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11382) 00000100:00000001:2.0:1788880413.310921:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880413.310926:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000010:1.0:1788880413.310926:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000094e066a1. 00000100:00000001:2.0:1788880413.310931:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880413.310935:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788880413.310937:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880413.310939:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880413.310951:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:1.0:1788880413.310951:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005972c901. 00000100:00000040:2.0:1788880413.310955:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880413.310963:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:1.0:1788880413.310964:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 00000400:00000200:0.0:1788880413.310973:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:1.0:1788880413.310977:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780655488, portal 10 00000400:00000200:0.0:1788880413.310981:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [190:190:256:4294967295] 192.168.204.111@tcp LPNI seq info [190:190:8:4294967295] 00000100:00000001:2.0:1788880413.310982:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880413.310988:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880413.310995:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:2.0:1788880413.310996:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880413.311000:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 02000000:00000001:2.0:1788880413.311001:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880413.311010:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880413.311012:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00100000:2.0:1788880413.311018:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780655616:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880413.311019:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880413.311030:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:0.0:1788880413.311035:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880413.311037:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880413.311041:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880413.311048:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 02000000:00000001:2.0:1788880413.311049:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880413.311056:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000dc03ef0a. 00000100:00000001:1.0:1788880413.311060:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880413.311062:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:2.0:1788880413.311063:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880413.311070:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880413.311072:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000002869d45. 00000400:00000010:1.0:1788880413.311078:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005377d7ad. 00000400:00000010:2.0:1788880413.311081:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000006bff322d. 00000100:00000200:1.0:1788880413.311086:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780655488, offset 0 00000100:00000200:2.0:1788880413.311088:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780655616, portal 10 00000800:00000200:0.0:1788880413.311094:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880413.311096:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:0.0:1788880413.311100:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880413.311100:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000400:00000200:1.0:1788880413.311108:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880413.311117:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880413.311123:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880413.311126:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880413.311131:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306049487 00000100:00000010:0.0:1788880413.311144:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000975d5882. 00000100:00000001:2.0:1788880413.311145:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880413.311147:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880413.311154:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:1.0:1788880413.311159:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:2.0:1788880413.311163:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880413.311172:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306049487 00000400:00000010:2.0:1788880413.311172:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000400:00000200:1.0:1788880413.311176:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:2.0:1788880413.311182:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780655616, offset 0 00000100:00000010:0.0:1788880413.311186:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001a2ff840. 00000400:00000200:1.0:1788880413.311186:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880413.311201:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:2.0:1788880413.311207:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880413.311211:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [191:191:255:4294967295] 192.168.204.111@tcp LPNI seq info [191:191:7:4294967295] 00000100:00000001:0.0:1788880413.311213:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306049487 00000100:00000010:0.0:1788880413.311222:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000085563e6f. 00000100:00000001:0.0:1788880413.311227:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880413.311261:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306049487 00000100:00000010:0.0:1788880413.311264:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014321469. 00000100:00000001:0.0:1788880413.311268:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880413.311276:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880413.311284:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880413.311285:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880413.311295:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880413.311299:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:1.0:1788880413.311303:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880413.311307:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880413.311308:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788880413.311312:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880413.311313:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:1.0:1788880413.311315:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000400:00000200:2.0:1788880413.311318:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880413.311326:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880413.311330:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000010:3.0:1788880413.311332:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000100:00000001:0.0:1788880413.311333:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880413.311335:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880413.311342:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880413.311347:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880413.311348:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880413.311349:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880413.311356:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880413.311366:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880413.311371:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [192:192:254:4294967295] 192.168.204.111@tcp LPNI seq info [192:192:6:4294967295] 00000800:00000200:1.0:1788880413.311375:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880413.311375:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880413.311385:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880413.311385:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880413.311393:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880413.311399:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 472996 usecs (473 jiffies) 00000100:00000001:1.0:1788880413.311404:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306044890 00000100:00000001:1.0:1788880413.311406:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306049487 00000100:00000001:1.0:1788880413.311415:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306044890 00000100:00000001:1.0:1788880413.311421:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306049487 00000400:00000200:2.0:1788880413.311421:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880413.311432:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306044890 00000100:00000001:1.0:1788880413.311437:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306049487 00000800:00000200:2.0:1788880413.311438:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880413.311457:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 00000100:00000001:1.0:1788880413.311465:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306044890 00000800:00000200:2.0:1788880413.311468:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880413.311472:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306049487 00000100:00000001:1.0:1788880413.311478:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880413.311479:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880413.311485:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880413.311487:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880413.311492:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880413.311494:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880413.311498:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e9812fb0 00000100:00000040:2.0:1788880413.311499:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 29178 usecs (30 jiffies) 00000100:00000001:1.0:1788880413.311501:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880413.311505:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e9812fb0. 00000100:00000001:2.0:1788880413.311512:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306044447 00000100:00000001:1.0:1788880413.311517:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880413.311517:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880413.311518:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306049487 00000100:00000001:1.0:1788880413.311524:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880413.311525:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306044447 00000100:00000200:3.0:1788880413.311530:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880429 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880413.311534:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306049487 00000100:00000001:1.0:1788880413.311535:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880413.311539:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880413.311545:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306044447 00000100:00000001:1.0:1788880413.311546:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880413.311550:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880413.311555:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880413.311555:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306049487 00000100:00000040:3.0:1788880413.311557:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880429 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880413.311567:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306044447 00000100:00000001:2.0:1788880413.311576:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306049487 00000100:00000001:3.0:1788880413.311580:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880413.311581:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880413.311594:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880413.311595:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880413.311601:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880413.311604:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880413.311610:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880413.311611:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880413.311615:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880413.311617:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:3.0:1788880413.311618:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880413.311620:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880413.311623:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000100:00000001:2.0:1788880413.311625:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880413.311630:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880413.311637:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880413.311640:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005377d7ad 00000400:00000010:3.0:1788880413.311642:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005377d7ad. 00000100:00000001:3.0:1788880413.311646:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880413.311649:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880413.311656:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880413.311659:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880413.311664:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880413.311666:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880413.311694:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880413.311696:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000400:00000200:3.0:1788880413.311700:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880413.311703:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880413.311705:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:3.0:1788880413.311707:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:3.0:1788880413.311709:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880413.311712:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880413.311724:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880413.311727:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880413.311732:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880413.311734:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880413.312986:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880413.312995:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880413.312998:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880413.313000:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880413.313006:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880413.313016:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b05b00 00000400:00000200:3.0:1788880413.313023:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x6d5 [1] + 224 00000800:00000001:3.0:1788880413.313063:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880413.313069:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880413.313071:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880413.313074:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880413.313079:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880413.313082:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880413.313090:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880413.313108:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880413.313114:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880413.313116:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880413.313139:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880413.313142:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880413.313147:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880413.313148:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880413.313149:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007695b617. 00000400:00000200:0.0:1788880413.313152:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8884ddc 00000400:00000010:0.0:1788880413.313153:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8884ddc. 00000100:00000001:0.0:1788880413.313161:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880413.313163:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880413.313169:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880413.313173:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880413.313174:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880413.313176:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880413.313178:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880413.313180:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880413.313181:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880413.313183:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880413.313188:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=91 v=5 (5 5 0 0) 00000100:00001000:0.0:1788880413.313190:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=91 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880413.313192:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880413.313193:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880413.313195:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880413.313197:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880413.313198:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880413.313200:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880413.313204:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880413.313206:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880413.313207:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880413.313209:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880413.313213:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780655360:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880413.313217:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880413.313219:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880413.313221:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780655360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880413.313227:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880413.313260:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880413.313263:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000026ba42a. 02000000:00000001:0.0:1788880413.313267:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880413.313269:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880413.313271:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880413.313273:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880413.313275:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000055d7d92. 02000000:00000001:0.0:1788880413.313279:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880413.313281:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880413.313282:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:0.0:1788880413.313286:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880413.313287:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880413.313289:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880413.313294:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880413.313295:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880413.313297:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880413.313299:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880413.313589:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880413.313604:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880413.313607:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880413.313620:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880413.313629:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880413.313642:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05b80 00000400:00000200:2.0:1788880413.313650:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x6dd [1] + 224 00000800:00000001:2.0:1788880413.313655:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880413.313659:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880413.313661:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880413.313667:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880413.313673:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880413.313678:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880413.313689:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880413.313708:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880413.313716:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880413.313720:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880413.313722:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880413.313724:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880413.313728:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880413.313735:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05c00 00000400:00000200:2.0:1788880413.313740:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x6e5 [1] + 224 00000800:00000001:2.0:1788880413.313743:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880413.313755:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880413.313758:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880413.313760:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880413.313764:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880413.313765:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880413.313770:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:1.0:1788880413.313772:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880413.313776:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000200:2.0:1788880413.313776:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880413.313780:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005972c901. 00000400:00000200:1.0:1788880413.313786:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000100:00000040:2.0:1788880413.313790:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880413.313791:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:1.0:1788880413.313797:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880413.313804:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880413.313825:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880413.313834:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880413.313835:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880413.313838:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880413.313838:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880413.313842:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880413.313844:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880413.313846:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880413.313847:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880413.313854:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880413.313860:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880413.313861:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880413.313866:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880413.313871:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880413.313873:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:1.0:1788880413.313874:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=81 v=5 (5 5 0 0) 00000400:00000010:0.0:1788880413.313876:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000002869d45. 00000100:00001000:1.0:1788880413.313883:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=81 v=5 (1 1 0 0) 00000400:00000200:0.0:1788880413.313885:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000006bff322d 00000100:00000001:1.0:1788880413.313893:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880413.313897:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880413.313900:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000006bff322d. 00000100:00000001:0.0:1788880413.313905:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:1.0:1788880413.313911:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880413.313917:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880413.313919:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880413.313924:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880413.313928:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880413.313938:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000200:0.0:1788880413.313943:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880413.313943:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880413.313957:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880413.313966:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880413.313968:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880413.313970:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880413.313974:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880413.313982:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780655488:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880413.313986:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880413.313989:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880413.313990:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880413.313997:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880413.313998:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880413.314003:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:1.0:1788880413.314008:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780655488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880413.314009:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880413.314010:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880413.314013:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880413.314019:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=91 v=5 (5 5 0 0) 00000100:00000001:1.0:1788880413.314027:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880413.314031:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880413.314035:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000094e066a1. 00000100:00001000:0.0:1788880413.314041:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=65 v=5 (1 1 0 0) 02000000:00000001:1.0:1788880413.314042:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880413.314047:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880413.314050:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880413.314051:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000020:00000001:1.0:1788880413.314054:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880413.314056:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880413.314058:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 00010000:00000001:0.0:1788880413.314062:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:1.0:1788880413.314065:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880413.314068:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880413.314069:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00010000:00080000:0.0:1788880413.314074:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880413.314077:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880413.314080:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880413.314087:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880413.314096:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:0.0:1788880413.314097:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880413.314100:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880413.314102:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880413.314107:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880413.314108:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880413.314111:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880413.314114:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880413.314117:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880413.314119:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880413.314128:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880413.314130:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880413.314131:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880413.314133:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880413.314141:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780655616:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880413.314147:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880413.314149:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880413.314152:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780655616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880429 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880413.314158:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880413.314160:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880413.314162:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000dc03ef0a. 02000000:00000001:0.0:1788880413.314165:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880413.314167:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880413.314168:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880413.314170:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880413.314172:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 02000000:00000001:0.0:1788880413.314182:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880413.314183:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880413.314185:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:0.0:1788880413.314187:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880413.314189:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880413.314190:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880413.314194:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880413.314196:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880413.314198:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880413.314199:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880413.374342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880413.374348:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880413.374351:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880413.374352:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880413.374357:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880413.374359:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880413.374360:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880413.374361:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880413.374365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880413.374366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880413.374397:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880413.374401:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880413.374403:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880414.398304:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880414.398313:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880414.398319:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880414.398320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880414.398357:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880414.398361:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880414.398364:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880415.422332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880415.422337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880415.422341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880415.422342:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880415.422381:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880415.422385:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880415.422388:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880416.446305:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880416.446312:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880416.446319:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880416.446322:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880416.446324:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880416.446327:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880416.446331:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880416.446333:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880416.446334:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880416.446336:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880416.446371:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880416.446375:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880416.446377:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880417.470353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880417.470361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880417.470378:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880417.470379:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880417.470418:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880417.470422:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880417.470425:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880418.430403:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306049487, at 4306049536 00000100:00000040:2.0:1788880418.430403:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306049487, at 4306049536 00000100:00000040:0.0:1788880418.430404:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306049487, at 4306049536 00000100:00000010:1.0:1788880418.430429:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001a2ff840. 00000100:00000010:2.0:1788880418.430434:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000085563e6f. 00000100:00000010:0.0:1788880418.430439:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000975d5882. 00000100:00000040:3.0:1788880418.430484:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306049487, at 4306049536 00000100:00000040:2.0:1788880418.430487:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880418.430497:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880418.430497:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014321469. 00000100:00000010:2.0:1788880418.430506:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000020:00000040:2.0:1788880418.430513:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880418.430519:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880418.430523:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880418.430530:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880418.430539:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:2.0:1788880418.430547:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880418.430560:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880418.430578:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880418.430598:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880418.430607:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_03+3] req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880418.430653:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880418.430655:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880418.430659:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880418.430662:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000884e74b8. 00000020:00000040:2.0:1788880418.430665:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880418.430667:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880418.430669:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880418.430671:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880418.430674:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c54a5c6f. 00000100:00000001:2.0:1788880418.430677:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880418.430682:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880418.430688:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880418.430689:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880418.430692:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000884e74b8] to pc [ptlrpcd_00_00+0] req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880418.430714:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880418.430716:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880418.430719:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880418.430722:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000629edf55. 00000020:00000040:2.0:1788880418.430724:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880418.430726:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880418.430728:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880418.430731:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880418.430734:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 00000100:00000001:2.0:1788880418.430739:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880418.430744:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880418.430748:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880418.430755:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880418.430756:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880418.430760:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880418.430760:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880418.430762:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880418.430767:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000629edf55] to pc [ptlrpcd_00_01+1] req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880418.430770:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880418.430784:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:2.0:1788880418.430793:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880418.430795:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880418.430798:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880418.430804:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880418.430806:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880418.430812:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880418.430814:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11387) 00000100:00000001:1.0:1788880418.430824:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880418.430826:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880418.430829:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880418.430831:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880418.430833:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880418.430839:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880418.430844:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880418.430849:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880418.430852:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880418.430862:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780655872:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880418.430872:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880418.430875:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880418.430879:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880418.430882:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880418.430883:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880418.430887:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880418.430889:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880418.430896:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:1.0:1788880418.430898:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880418.430905:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780655744:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880418.430910:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000010:3.0:1788880418.430912:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000006010d39b. 02000000:00000001:1.0:1788880418.430916:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880418.430918:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880418.430922:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880418.430925:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:3.0:1788880418.430928:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000e0ef94e8. 02000000:00000010:1.0:1788880418.430932:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009a0ddd94. 02000000:00000001:1.0:1788880418.430936:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880418.430939:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 00000400:00000010:1.0:1788880418.430943:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005972c901. 00000100:00000200:3.0:1788880418.430950:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780655872, portal 10 00000400:00000010:1.0:1788880418.430951:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 00000100:00000001:3.0:1788880418.430956:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880418.430959:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938423808 : -126823771127808 : ffff8ca78a2c4000) 00000100:00000200:1.0:1788880418.430960:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780655744, portal 25 00000100:00000001:1.0:1788880418.430966:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880418.430968:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 00000100:00000040:3.0:1788880418.430972:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880418.430980:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880434 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880418.431004:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880418.431007:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880418.431013:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880418.431020:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880418.431021:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000999bcd04. 00000400:00000010:1.0:1788880418.431026:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000994ad42e. 00000100:00000200:3.0:1788880418.431027:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780655872, offset 0 00000100:00000200:1.0:1788880418.431034:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780655744, offset 0 00000400:00000200:3.0:1788880418.431043:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880418.431062:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880418.431066:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880418.431074:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880418.431080:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880418.431083:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880418.431086:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880418.431087:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880418.431094:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880418.431104:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880418.431108:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880418.431109:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [193:193:256:4294967295] 192.168.204.111@tcp LPNI seq info [193:193:8:4294967295] 00000100:00000001:0.0:1788880418.431110:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880418.431117:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880418.431124:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880418.431126:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880418.431129:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880418.431133:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880418.431138:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880418.431142:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00100000:0.0:1788880418.431144:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780656000:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880418.431147:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880418.431149:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880418.431153:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880418.431155:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880418.431157:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d2602e97. 02000000:00000001:0.0:1788880418.431159:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880418.431161:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007695b617. 00000400:00000010:0.0:1788880418.431169:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 00000100:00000200:0.0:1788880418.431173:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780656000, portal 10 00000100:00000001:0.0:1788880418.431177:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:3.0:1788880418.431181:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880418.431182:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880418.431185:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938432768 : -126823771118848 : ffff8ca78a2c6300) 00000800:00000010:3.0:1788880418.431195:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000749ed09e. 00000100:00000040:0.0:1788880418.431198:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880418.431198:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880418.431207:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880418.431214:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880418.431215:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880418.431220:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788880418.431223:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:0.0:1788880418.431224:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880418.431229:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000200:0.0:1788880418.431253:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780656000, offset 0 00000400:00000200:0.0:1788880418.431259:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880418.431261:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [194:194:255:4294967295] 192.168.204.111@tcp LPNI seq info [194:194:7:4294967295] 00000800:00000200:3.0:1788880418.431274:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880418.431280:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880418.431287:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000749ed09e type 1, nob 320 niov 1 nkiov 1 00000800:00000200:1.0:1788880418.431288:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880418.431291:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880418.431299:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000400:00000200:0.0:1788880418.431302:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880418.431309:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880418.431312:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880418.431314:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880418.431317:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880418.431318:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880418.431325:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:3.0:1788880418.431330:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 464467 usecs (465 jiffies) 00000400:00000200:0.0:1788880418.431331:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [195:195:254:4294967295] 192.168.204.111@tcp LPNI seq info [195:195:6:4294967295] 00000800:00000200:1.0:1788880418.431332:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880418.431360:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880418.431360:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306050002 00000400:00000200:0.0:1788880418.431367:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880418.431373:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306054608 00000100:00000010:3.0:1788880418.431374:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d6fe56. 00000800:00000200:0.0:1788880418.431375:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880418.431385:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b016e4f4. 00000100:00000001:3.0:1788880418.431388:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000800:00000200:0.0:1788880418.431390:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880418.431403:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880418.431403:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306050002 00000100:00000001:1.0:1788880418.431407:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306050002 00000800:00000200:2.0:1788880418.431408:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:0.0:1788880418.431409:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b016e4f4 type 1, nob 320 niov 1 nkiov 1 00000100:00000010:3.0:1788880418.431414:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008ea3b290. 00000100:00000001:1.0:1788880418.431418:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306054608 00000100:00000001:0.0:1788880418.431421:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880418.431421:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000749ed09e. 00000100:00000001:3.0:1788880418.431426:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000040:0.0:1788880418.431429:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 28682 usecs (29 jiffies) 00000100:00000001:0.0:1788880418.431436:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306049566 00000400:00000200:2.0:1788880418.431437:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880418.431438:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306050002 00000100:00000001:3.0:1788880418.431438:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306050002 00000100:00000001:0.0:1788880418.431445:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306054608 00000100:00000010:3.0:1788880418.431455:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d54626ec. 00000100:00000001:0.0:1788880418.431457:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306049566 00000100:00000001:1.0:1788880418.431458:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306054608 00000400:00000200:2.0:1788880418.431463:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880418.431470:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306054608 00000100:00000001:3.0:1788880418.431472:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880418.431474:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306049566 00000400:00000200:2.0:1788880418.431477:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000999bcd04 00000400:00000010:2.0:1788880418.431483:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000999bcd04. 00000100:00000001:3.0:1788880418.431485:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306050002 00000100:00000001:1.0:1788880418.431489:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306050002 00000100:00000001:2.0:1788880418.431492:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000010:3.0:1788880418.431492:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001bac11fc. 00000100:00000001:0.0:1788880418.431495:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306054608 00000100:00000001:1.0:1788880418.431495:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306054608 00000100:00000001:3.0:1788880418.431499:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000200:2.0:1788880418.431502:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880418.431504:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306049566 00000100:00000001:3.0:1788880418.431510:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880418.431511:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306050002 00000100:00000001:3.0:1788880418.431527:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880418.431529:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880418.431530:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880418.431531:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306054608 00000100:00000001:3.0:1788880418.431535:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880418.431536:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880418.431539:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880418.431540:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880418.431550:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880418.431553:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880418.431554:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880418.431568:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880418.431569:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880418.431572:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880418.431572:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880418.431574:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880418.431580:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880418.431580:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880418.431586:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880418.431586:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880418.431593:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880418.431594:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880418.431596:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880418.431600:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:2.0:1788880418.431603:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880418.431603:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880418.431605:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000010:2.0:1788880418.431610:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000100:00000001:0.0:1788880418.431614:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880418.431618:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880418.431620:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880418.431628:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880418.431633:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000994ad42e 00000400:00000010:2.0:1788880418.431636:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000994ad42e. 00000100:00000001:2.0:1788880418.431640:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880418.431645:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880434 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880418.431659:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880418.431662:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880434 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880418.431668:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880418.431669:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880418.431711:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880418.431714:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b016e4f4. 00000400:00000200:2.0:1788880418.431717:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880418.431720:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880418.431722:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:2.0:1788880418.431724:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:2.0:1788880418.431726:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880418.431729:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880418.431735:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880418.431736:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880418.431741:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880418.431743:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880418.433476:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880418.433497:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880418.433501:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880418.433502:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880418.433548:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880418.433564:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05d00 00000400:00000200:3.0:1788880418.433573:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x6ed [1] + 224 00000800:00000001:3.0:1788880418.433581:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880418.433585:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880418.433588:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880418.433591:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880418.433598:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880418.433603:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880418.433613:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880418.433641:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880418.433651:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880418.433655:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880418.433658:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880418.433661:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880418.433666:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880418.433676:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b05c80 00000400:00000200:3.0:1788880418.433683:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x6f1 [1] + 224 00000800:00000001:3.0:1788880418.433689:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880418.433693:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880418.433695:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880418.433699:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880418.433703:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880418.433705:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:3.0:1788880418.433709:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880418.433711:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880418.433721:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:3.0:1788880418.433723:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880418.433724:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880418.433728:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000e0ef94e8. 00000400:00000200:0.0:1788880418.433739:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000400:00000010:0.0:1788880418.433745:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:0.0:1788880418.433757:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880418.433759:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:0.0:1788880418.433768:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880418.433768:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880418.433776:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880418.433781:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880418.433786:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880418.433792:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880418.433798:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880418.433805:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880418.433806:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880418.433807:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000001:1.0:1788880418.433811:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880418.433812:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880418.433815:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880418.433818:0:201285:0:(client.c:1580:after_reply()) Process entered 00000400:00000010:1.0:1788880418.433820:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005972c901. 02000000:00000001:0.0:1788880418.433827:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000200:3.0:1788880418.433827:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05d80 00000400:00000200:1.0:1788880418.433829:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000100:00000001:0.0:1788880418.433833:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880418.433837:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880418.433840:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880418.433843:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:1.0:1788880418.433848:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:0.0:1788880418.433850:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=86 v=5 (5 5 0 0) 00000400:00000200:3.0:1788880418.433853:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x6fd [1] + 224 00000100:00000200:1.0:1788880418.433857:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880418.433870:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=86 v=5 (1 1 0 0) 00000800:00000001:3.0:1788880418.433870:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880418.433882:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:3.0:1788880418.433884:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880418.433885:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880418.433887:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880418.433890:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00010000:00000001:0.0:1788880418.433893:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:3.0:1788880418.433903:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00010000:00000001:0.0:1788880418.433908:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880418.433916:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880418.433918:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880418.433922:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880418.433923:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:3.0:1788880418.433925:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880418.433928:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880418.433928:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880418.433933:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880418.433937:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880418.433940:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:0.0:1788880418.433943:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880418.433946:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880418.433950:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880418.433963:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880418.433973:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=96 v=5 (5 5 0 0) 00000100:00000040:3.0:1788880418.433987:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880418.433988:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=96 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880418.433989:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880418.434002:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880418.434005:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880418.434008:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880418.434010:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880418.434018:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880418.434021:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880418.434027:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880418.434030:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880418.434043:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880418.434054:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:0.0:1788880418.434055:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780655872:192.168.204.111@tcp:400: 00000800:00000001:3.0:1788880418.434062:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880418.434069:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880418.434070:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880418.434071:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880418.434079:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880418.434080:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880418.434084:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880418.434087:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000884e74b8 x1875776780655872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880418.434096:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880418.434099:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:1.0:1788880418.434101:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880418.434102:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880418.434105:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880418.434110:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000006010d39b. 00000400:00000001:2.0:1788880418.434112:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880418.434115:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880418.434119:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007695b617. 02000000:00000001:0.0:1788880418.434120:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000200:2.0:1788880418.434125:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8884ddc 00000100:00100000:1.0:1788880418.434126:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780655744:192.168.204.111@tcp:400: 00000020:00000001:0.0:1788880418.434127:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000010:2.0:1788880418.434129:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8884ddc. 00000020:00000040:0.0:1788880418.434132:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880418.434137:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000001:0.0:1788880418.434143:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880418.434145:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880418.434146:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880418.434149:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c54a5c6f. 00000100:00000001:1.0:1788880418.434156:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:0.0:1788880418.434162:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880418.434166:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000200:2.0:1788880418.434167:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880418.434168:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780655744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880418.434174:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000884e74b8. 00000100:00000001:0.0:1788880418.434187:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880418.434194:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880418.434196:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880418.434201:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880418.434205:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880418.434209:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880418.434211:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880418.434212:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880418.434213:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880418.434219:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880418.434220:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880418.434229:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880418.434274:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880418.434277:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880418.434278:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880418.434282:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880418.434286:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009a0ddd94. 00000100:00001000:2.0:1788880418.434289:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=96 v=5 (5 5 0 0) 02000000:00000001:1.0:1788880418.434292:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:2.0:1788880418.434294:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=70 v=5 (1 1 0 0) 00000020:00000001:1.0:1788880418.434296:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880418.434297:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880418.434301:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880418.434302:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00010000:00000001:2.0:1788880418.434304:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:1.0:1788880418.434310:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00080000:2.0:1788880418.434314:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880418.434315:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 02000000:00000001:1.0:1788880418.434323:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880418.434329:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00010000:00000001:2.0:1788880418.434332:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880418.434333:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000100:00000001:2.0:1788880418.434338:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:1.0:1788880418.434342:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:2.0:1788880418.434352:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880418.434358:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880418.434358:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880418.434362:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880418.434365:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880418.434372:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880418.434374:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880418.434378:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880418.434386:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880418.434391:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880418.434403:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880418.434407:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880418.434410:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880418.434417:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880418.434428:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780656000:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880418.434434:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880418.434436:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880418.434439:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000629edf55 x1875776780656000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880434 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880418.434446:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880418.434448:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880418.434457:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d2602e97. 02000000:00000001:2.0:1788880418.434461:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880418.434463:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880418.434465:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880418.434467:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880418.434469:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 02000000:00000001:2.0:1788880418.434472:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880418.434473:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880418.434475:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000629edf55. 00000100:00000001:2.0:1788880418.434478:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880418.434479:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880418.434481:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880418.434487:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880418.434489:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880418.434491:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880418.434492:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880418.494430:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880418.494439:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880418.494447:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880418.494449:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880418.494486:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880418.494493:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880418.494498:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880419.519127:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880419.519133:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880419.519139:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880419.519140:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880419.519184:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880419.519189:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880419.519192:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880420.542368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880420.542378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880420.542385:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880420.542386:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880420.542459:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880420.542467:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880420.542472:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880421.566357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880421.566364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880421.566370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880421.566371:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880421.566402:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880421.566405:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880421.566407:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880422.590394:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880422.590400:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880422.590404:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880422.590420:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880422.590475:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880422.590479:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880422.590482:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788880423.550385:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306054608, at 4306054656 00000100:00000040:1.0:1788880423.550390:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306054608, at 4306054656 00000100:00000040:0.0:1788880423.550391:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306054608, at 4306054656 00000100:00000010:1.0:1788880423.550415:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008ea3b290. 00000100:00000010:3.0:1788880423.550429:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001bac11fc. 00000100:00000010:0.0:1788880423.550433:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d6fe56. 00000100:00000040:2.0:1788880423.550438:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306054608, at 4306054656 00000100:00000010:2.0:1788880423.550452:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d54626ec. 00000100:00000040:1.0:1788880423.550467:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880423.550475:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880423.550485:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e2be23b2. 00000020:00000040:1.0:1788880423.550491:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880423.550496:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880423.550499:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880423.550504:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880423.550513:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:1.0:1788880423.550520:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880423.550531:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880423.550540:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880423.550542:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880423.550548:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e2be23b2] to pc [ptlrpcd_00_02+2] req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880423.550595:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880423.550598:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880423.550601:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880423.550605:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002980f9ff. 00000100:00000001:0.0:1788880423.550607:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:1.0:1788880423.550608:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880423.550612:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880423.550614:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880423.550614:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880423.550618:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880423.550619:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:1.0:1788880423.550625:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000040:0.0:1788880423.550629:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880423.550630:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880423.550636:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880423.550646:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880423.550652:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880423.550665:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880423.550668:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880423.550668:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788880423.550674:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880423.550677:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880423.550680:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880423.550685:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880423.550686:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780656128:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880423.550693:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002980f9ff] to pc [ptlrpcd_00_03+3] req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880423.550694:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880423.550701:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880423.550705:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880423.550707:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880423.550708:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880423.550711:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:0.0:1788880423.550714:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a581a743. 00000100:00000001:1.0:1788880423.550714:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880423.550717:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002b387587. 02000000:00000001:0.0:1788880423.550718:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880423.550722:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880423.550725:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880423.550727:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000010:0.0:1788880423.550728:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006e8d9c3b. 02000000:00000001:1.0:1788880423.550729:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880423.550734:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:2.0:1788880423.550738:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788880423.550739:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005c796512. 00000100:00000001:1.0:1788880423.550739:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880423.550745:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:0.0:1788880423.550751:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780656128, portal 25 00000100:00000040:1.0:1788880423.550751:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880423.550753:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880423.550762:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880423.550764:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880423.550765:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055936 : -126823771495680 : ffff8ca78a26a300) 00000100:00000001:1.0:1788880423.550768:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880423.550772:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880423.550780:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002b387587] to pc [ptlrpcd_00_00+0] req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880423.550782:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880439 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880423.550822:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880423.550827:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880423.550838:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880423.550846:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880423.550850:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:0.0:1788880423.550852:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e0ace8ca. 00000100:00000001:1.0:1788880423.550852:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880423.550855:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880423.550857:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:0.0:1788880423.550859:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780656128, offset 0 02000000:00000001:2.0:1788880423.550865:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880423.550869:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788880423.550876:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11392) 00000100:00100000:2.0:1788880423.550876:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780656256:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880423.550877:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880423.550889:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880423.550896:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880423.550899:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880423.550902:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880423.550904:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880423.550906:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00100000:1.0:1788880423.550909:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000400:00000200:0.0:1788880423.550913:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:2.0:1788880423.550915:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000009e54fce. 00000400:00000200:0.0:1788880423.550921:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:2.0:1788880423.550923:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880423.550928:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880423.550930:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880423.550936:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:2.0:1788880423.550937:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000087a6361a. 00000100:00000001:1.0:1788880423.550941:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880423.550943:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [196:196:256:4294967295] 192.168.204.111@tcp LPNI seq info [196:196:8:4294967295] 00000100:00000001:1.0:1788880423.550946:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880423.550950:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005071f3f0. 00000100:00000001:1.0:1788880423.550953:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880423.550962:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:2.0:1788880423.550966:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780656256, portal 10 00000100:00000001:1.0:1788880423.550967:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880423.550973:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880423.550974:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880423.550980:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938061312 : -126823771490304 : ffff8ca78a26b800) 00000400:00000200:0.0:1788880423.550982:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880423.550982:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880423.550983:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880423.550998:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880423.551004:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880423.551020:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ac392bfe. 00000800:00000200:0.0:1788880423.551046:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880423.551063:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880423.551065:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880423.551069:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880423.551080:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880423.551081:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:1.0:1788880423.551084:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:2.0:1788880423.551087:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ac9b6c42. 00000800:00000200:0.0:1788880423.551091:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ac392bfe type 1, nob 320 niov 1 nkiov 1 00000100:00000200:2.0:1788880423.551093:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780656256, offset 0 02000000:00000001:1.0:1788880423.551100:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880423.551110:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880423.551111:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880423.551113:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880423.551120:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306059727 00000100:00100000:1.0:1788880423.551122:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780656384:192.168.204.111@tcp:400: 00000100:00000010:0.0:1788880423.551126:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000975d5882. 00000400:00000200:2.0:1788880423.551130:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880423.551132:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880423.551136:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 02000000:00000001:1.0:1788880423.551138:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880423.551142:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880423.551144:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880423.551145:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306059727 02000000:00000001:1.0:1788880423.551145:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880423.551154:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000006c03106. 00000100:00000010:0.0:1788880423.551158:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ffa8a4fd. 02000000:00000001:1.0:1788880423.551158:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880423.551162:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880423.551166:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000010:1.0:1788880423.551169:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005972c901. 00000400:00000200:2.0:1788880423.551174:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880423.551176:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306059727 00000400:00000010:1.0:1788880423.551178:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 00000400:00000200:2.0:1788880423.551184:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:0.0:1788880423.551187:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000a4f85af. 00000100:00000200:1.0:1788880423.551188:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780656384, portal 10 00000100:00000001:1.0:1788880423.551195:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788880423.551196:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [197:197:255:4294967295] 192.168.204.111@tcp LPNI seq info [197:197:7:4294967295] 00000100:00000001:0.0:1788880423.551198:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880423.551208:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938057728 : -126823771493888 : ffff8ca78a26aa00) 00000100:00000001:0.0:1788880423.551209:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306059727 00000100:00000010:0.0:1788880423.551219:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000029f028ab. 00000100:00000040:1.0:1788880423.551227:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880423.551227:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880423.551271:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880423.551277:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880423.551289:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880423.551290:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880423.551295:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880423.551303:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880423.551304:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880423.551305:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880423.551310:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000007206624. 00000100:00000001:0.0:1788880423.551316:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:1.0:1788880423.551319:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780656384, offset 0 00000800:00000010:2.0:1788880423.551319:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000781b5970. 00000100:00000001:0.0:1788880423.551325:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880423.551332:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880423.551334:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880423.551338:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880423.551339:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880423.551346:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880423.551346:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880423.551357:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880423.551365:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880423.551373:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880423.551380:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880423.551384:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1788880423.551390:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000781b5970 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880423.551394:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880423.551405:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [198:198:254:4294967295] 192.168.204.111@tcp LPNI seq info [198:198:6:4294967295] 00000100:00000001:2.0:1788880423.551406:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880423.551422:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 456243 usecs (457 jiffies) 00000100:00000001:2.0:1788880423.551434:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306055114 00000800:00000200:3.0:1788880423.551440:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880423.551441:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306059727 00000100:00000001:2.0:1788880423.551450:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306055114 00000800:00000010:3.0:1788880423.551457:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ac392bfe. 00000400:00000200:1.0:1788880423.551460:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880423.551460:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306059727 00000400:00000200:3.0:1788880423.551466:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880423.551469:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306055114 00000100:00000001:2.0:1788880423.551479:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306059727 00000100:00000001:2.0:1788880423.551484:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306055114 00000100:00000001:2.0:1788880423.551486:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306059727 00000400:00000200:3.0:1788880423.551490:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880423.551492:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880423.551497:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880423.551504:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000400:00000200:3.0:1788880423.551504:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e0ace8ca 00000100:00000001:2.0:1788880423.551514:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880423.551521:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880423.551521:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880423.551529:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:1.0:1788880423.551538:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880423.551543:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880423.551543:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880423.551548:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880423.551552:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880423.551554:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880423.551554:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e0ace8ca. 00000100:00000001:2.0:1788880423.551560:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880423.551561:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 28213 usecs (29 jiffies) 00000100:00000001:2.0:1788880423.551564:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880423.551565:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880423.551568:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306054686 00000100:00000001:2.0:1788880423.551571:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880423.551575:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306059727 00000100:00000200:3.0:1788880423.551576:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880439 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880423.551584:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306054686 00000100:00000001:1.0:1788880423.551594:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306059727 00000100:00000001:1.0:1788880423.551603:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306054686 00000100:00000001:1.0:1788880423.551610:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306059727 00000100:00000001:3.0:1788880423.551611:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880423.551618:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306054686 00000100:00000040:3.0:1788880423.551620:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880439 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880423.551624:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306059727 00000100:00000001:1.0:1788880423.551629:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880423.551638:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880423.551646:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880423.551649:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880423.551653:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880423.551654:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880423.551682:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880423.551685:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000781b5970. 00000400:00000200:3.0:1788880423.551689:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880423.551693:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880423.551695:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ac9b6c42 00000400:00000010:3.0:1788880423.551697:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ac9b6c42. 00000100:00000001:3.0:1788880423.551700:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880423.551703:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880423.551711:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880423.551714:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880423.551720:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880423.551722:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880423.551755:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880423.551757:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:3.0:1788880423.551761:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880423.551764:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880423.551767:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000007206624 00000400:00000010:3.0:1788880423.551769:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000007206624. 00000100:00000001:3.0:1788880423.551778:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880423.551782:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880423.551789:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880423.551791:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880423.551797:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880423.551799:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880423.552849:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880423.552865:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880423.552868:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880423.552871:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880423.552879:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880423.552893:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b05e00 00000400:00000200:2.0:1788880423.552903:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x705 [1] + 224 00000800:00000001:2.0:1788880423.552910:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880423.552925:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880423.552927:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880423.552931:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880423.552937:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880423.552941:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880423.552952:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880423.552975:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880423.552985:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880423.552990:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880423.553019:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880423.553024:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880423.553030:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880423.553033:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880423.553036:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006e8d9c3b. 00000400:00000200:1.0:1788880423.553041:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005c796512 00000400:00000010:1.0:1788880423.553042:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005c796512. 00000100:00000001:1.0:1788880423.553046:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880423.553053:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880423.553075:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880423.553081:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880423.553083:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880423.553086:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880423.553088:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880423.553090:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880423.553092:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880423.553095:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880423.553100:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=101 v=5 (5 5 0 0) 00000100:00001000:1.0:1788880423.553104:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=101 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880423.553106:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880423.553107:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880423.553108:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880423.553110:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880423.553112:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880423.553115:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880423.553119:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880423.553121:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880423.553122:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880423.553124:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880423.553128:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780656128:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880423.553131:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880423.553133:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880423.553134:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e2be23b2 x1875776780656128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880423.553138:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880423.553139:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880423.553141:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a581a743. 02000000:00000001:1.0:1788880423.553149:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880423.553150:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880423.553152:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880423.553153:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880423.553155:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:1.0:1788880423.553157:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880423.553159:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880423.553160:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e2be23b2. 00000100:00000001:1.0:1788880423.553162:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880423.553163:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880423.553164:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880423.553168:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880423.553169:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880423.553170:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880423.553171:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880423.554145:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880423.554154:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880423.554157:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880423.554159:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880423.554168:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880423.554178:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05e80 00000400:00000200:3.0:1788880423.554185:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x70d [1] + 224 00000800:00000001:3.0:1788880423.554190:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880423.554195:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880423.554199:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880423.554202:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880423.554206:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880423.554211:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880423.554227:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880423.554460:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880423.554468:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880423.554472:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880423.554474:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880423.554476:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880423.554481:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880423.554488:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b05f00 00000400:00000200:3.0:1788880423.554494:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x715 [1] + 224 00000100:00000001:2.0:1788880423.554499:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880423.554504:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880423.554544:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:2.0:1788880423.554545:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880423.554549:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:3.0:1788880423.554549:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880423.554552:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000087a6361a. 00000400:00000200:3.0:1788880423.554552:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880423.554560:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005071f3f0 00000400:00000200:3.0:1788880423.554561:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:2.0:1788880423.554568:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005071f3f0. 00000100:00000001:2.0:1788880423.554572:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880423.554574:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880423.554578:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880423.554581:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880423.554600:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880423.554604:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880423.554620:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880423.554624:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880423.554627:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880423.554630:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880423.554633:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880423.554634:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880423.554635:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880423.554640:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880423.554643:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880423.554648:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880423.554649:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=91 v=5 (5 5 0 0) 00000100:00001000:2.0:1788880423.554664:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=91 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880423.554668:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880423.554671:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880423.554680:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880423.554685:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880423.554688:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880423.554690:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880423.554695:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880423.554699:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880423.554700:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880423.554705:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880423.554707:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880423.554717:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880423.554717:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000001:1.0:1788880423.554723:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880423.554725:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000010:1.0:1788880423.554730:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005972c901. 00000100:00000001:2.0:1788880423.554731:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880423.554740:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000100:00000040:2.0:1788880423.554741:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880423.554747:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:1.0:1788880423.554752:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880423.554762:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880423.554775:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780656256:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880423.554790:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880423.554795:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:1.0:1788880423.554798:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880423.554804:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002980f9ff x1875776780656256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880423.554816:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880423.554819:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880423.554827:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880423.554831:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880423.554834:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880423.554836:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:2.0:1788880423.554838:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880423.554840:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880423.554843:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000009e54fce. 02000000:00000001:1.0:1788880423.554846:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880423.554850:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:1.0:1788880423.554855:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=101 v=5 (5 5 0 0) 00000020:00000001:2.0:1788880423.554855:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880423.554862:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00001000:1.0:1788880423.554865:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=75 v=5 (1 1 0 0) 00000020:00000001:2.0:1788880423.554867:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880423.554872:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000010:2.0:1788880423.554873:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:2.0:1788880423.554879:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880423.554881:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880423.554882:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002980f9ff. 00000100:00000001:2.0:1788880423.554886:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880423.554887:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880423.554890:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880423.554897:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880423.554897:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:1.0:1788880423.554901:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880423.554908:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880423.554908:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880423.554916:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880423.554920:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880423.554929:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880423.554932:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880423.554934:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880423.554938:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880423.554939:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880423.554943:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880423.554949:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880423.554952:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880423.554953:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880423.554956:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880423.554963:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780656384:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880423.554968:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880423.554970:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880423.554973:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002b387587 x1875776780656384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880439 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880423.554979:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880423.554993:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880423.554995:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000006c03106. 02000000:00000001:1.0:1788880423.554999:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880423.555000:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880423.555002:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880423.555004:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880423.555006:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 02000000:00000001:1.0:1788880423.555008:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880423.555010:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880423.555011:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002b387587. 00000100:00000001:1.0:1788880423.555021:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880423.555022:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880423.555024:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880423.555029:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880423.555030:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880423.555032:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880423.555034:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880423.614387:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880423.614397:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880423.614399:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880423.614403:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880423.614409:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880423.614411:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880423.614413:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880423.614415:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880423.614420:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880423.614421:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880423.614512:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880423.614530:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880423.614535:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880424.638390:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880424.638399:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880424.638406:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880424.638407:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880424.638439:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880424.638444:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880424.638448:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880425.662371:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880425.662381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880425.662389:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880425.662390:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880425.662433:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880425.662438:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880425.662442:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880426.686356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880426.686363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880426.686374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880426.686377:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880426.686380:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880426.686383:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880426.686386:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880426.686388:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880426.686390:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880426.686392:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880426.686428:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880426.686432:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880426.686434:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880427.710377:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880427.710389:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880427.710398:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880427.710421:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880427.710476:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880427.710482:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880427.710485:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880428.670342:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306059727, at 4306059776 00000100:00000040:3.0:1788880428.670342:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306059727, at 4306059776 00000100:00000040:1.0:1788880428.670348:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306059727, at 4306059776 00000100:00000010:3.0:1788880428.670365:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000029f028ab. 00000100:00000010:2.0:1788880428.670369:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000a4f85af. 00000100:00000010:1.0:1788880428.670373:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ffa8a4fd. 00000100:00000040:0.0:1788880428.670376:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880428.670389:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880428.670397:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000451fe31a. 00000020:00000040:0.0:1788880428.670403:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880428.670407:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880428.670410:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880428.670415:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880428.670425:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000a371c7dc. 00000100:00000001:0.0:1788880428.670431:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880428.670439:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880428.670446:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880428.670448:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880428.670452:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000451fe31a] to pc [ptlrpcd_00_01+1] req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880428.670472:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880428.670476:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880428.670479:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880428.670480:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000009ee4c6bf. 00000020:00000040:0.0:1788880428.670482:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880428.670484:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880428.670485:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880428.670487:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880428.670490:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000131c1a55. 00000100:00000001:0.0:1788880428.670492:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880428.670499:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880428.670504:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880428.670509:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880428.670511:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880428.670516:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880428.670516:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880428.670532:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880428.670534:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880428.670537:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880428.670550:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000009ee4c6bf] to pc [ptlrpcd_00_02+2] req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880428.670591:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880428.670595:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880428.670602:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880428.670606:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880428.670609:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880428.670611:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880428.670613:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788880428.670616:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780656512:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880428.670622:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880428.670628:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006b32fd51. 00000100:00000001:1.0:1788880428.670630:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000020:00000040:0.0:1788880428.670633:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880428.670634:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880428.670636:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880428.670640:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880428.670641:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880428.670642:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880428.670649:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880428.670650:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880428.670651:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880428.670652:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880428.670662:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880428.670663:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000030024c2c. 02000000:00000001:1.0:1788880428.670664:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880428.670667:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880428.670671:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880428.670680:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002974b115. 00000100:00000040:0.0:1788880428.670686:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880428.670689:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880428.670700:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880428.670705:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880428.670708:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005972c901. 02000000:00000001:2.0:1788880428.670712:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880428.670720:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880428.670725:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788880428.670726:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880428.670729:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880428.670729:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 00000100:00000040:0.0:1788880428.670740:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006b32fd51] to pc [ptlrpcd_00_03+3] req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880428.670740:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780656640:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880428.670750:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780656512, portal 25 00000100:00000001:1.0:1788880428.670762:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880428.670764:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880428.670769:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849366784 : -126822860184832 : ffff8ca7c0782300) 02000000:00000001:2.0:1788880428.670773:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880428.670779:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880428.670789:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880428.670789:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880444 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880428.670796:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880428.670797:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:2.0:1788880428.670804:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000fde20984. 00000100:00000001:3.0:1788880428.670815:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880428.670816:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880428.670818:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880428.670822:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:2.0:1788880428.670828:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000087a6361a. 00000100:00000001:3.0:1788880428.670829:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880428.670833:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11397) 00000100:00000001:1.0:1788880428.670837:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880428.670839:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880428.670847:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880428.670848:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005071f3f0. 00000400:00000010:1.0:1788880428.670854:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000100:00000200:2.0:1788880428.670855:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780656640, portal 10 00000100:00000001:3.0:1788880428.670855:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:0.0:1788880428.670856:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306059727, at 4306059776 00000100:00000001:3.0:1788880428.670859:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880428.670860:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000200:1.0:1788880428.670865:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780656512, offset 0 00000100:00000001:2.0:1788880428.670865:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849358720 : -126822860192896 : ffff8ca7c0780380) 02000000:00000001:3.0:1788880428.670873:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880428.670878:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:0.0:1788880428.670879:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000975d5882. 02000000:00000001:3.0:1788880428.670882:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880428.670883:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880428.670889:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:3.0:1788880428.670892:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780656768:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880428.670900:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880428.670904:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880428.670910:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880428.670911:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880428.670912:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880428.670914:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880428.670915:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a3b2cf4a. 02000000:00000001:3.0:1788880428.670916:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880428.670921:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008545eb31. 00000100:00000200:2.0:1788880428.670922:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780656640, offset 0 02000000:00000001:3.0:1788880428.670925:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880428.670928:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:3.0:1788880428.670931:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004100a205. 00000400:00000200:2.0:1788880428.670935:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880428.670936:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880428.670941:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:3.0:1788880428.670943:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cf82a3fa. 00000400:00000200:1.0:1788880428.670946:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:3.0:1788880428.670948:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780656768, portal 10 00000400:00000200:1.0:1788880428.670953:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [199:199:256:4294967295] 192.168.204.111@tcp LPNI seq info [199:199:8:4294967295] 00000100:00000001:3.0:1788880428.670954:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880428.670956:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849362304 : -126822860189312 : ffff8ca7c0781180) 00000100:00000040:3.0:1788880428.670967:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880428.670978:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880428.670984:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880428.670989:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880428.670991:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880428.670992:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880428.670995:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 00000400:00000200:2.0:1788880428.670997:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:1.0:1788880428.671001:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000400:00000200:2.0:1788880428.671002:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:3.0:1788880428.671002:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780656768, offset 0 00000400:00000200:2.0:1788880428.671007:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880428.671011:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880428.671016:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [200:200:255:4294967295] 192.168.204.111@tcp LPNI seq info [200:200:7:4294967295] 00000800:00000200:1.0:1788880428.671017:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880428.671024:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:3.0:1788880428.671026:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880428.671030:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880428.671034:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880428.671037:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000800:00000010:2.0:1788880428.671039:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000045330cf1. 00000400:00000200:3.0:1788880428.671042:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:2.0:1788880428.671047:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880428.671047:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880428.671054:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880428.671056:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880428.671060:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880428.671061:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880428.671061:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000045330cf1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880428.671065:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880428.671068:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880428.671070:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306064847 00000400:00000200:3.0:1788880428.671070:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [201:201:254:4294967295] 192.168.204.111@tcp LPNI seq info [201:201:6:4294967295] 00000100:00000040:2.0:1788880428.671073:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 448325 usecs (449 jiffies) 00000100:00000010:1.0:1788880428.671079:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000100:00000001:2.0:1788880428.671079:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306060225 00000400:00000200:3.0:1788880428.671086:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880428.671091:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:3.0:1788880428.671092:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880428.671098:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000100:00000001:1.0:1788880428.671104:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306064847 00000100:00000001:2.0:1788880428.671104:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306064847 00000800:00000200:3.0:1788880428.671105:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880428.671110:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306060225 00000100:00000010:1.0:1788880428.671112:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000800:00000200:3.0:1788880428.671115:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880428.671118:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880428.671119:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880428.671122:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880428.671126:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306064847 00000100:00000001:2.0:1788880428.671126:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306064847 00000100:00000040:3.0:1788880428.671126:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 27775 usecs (28 jiffies) 00000100:00000001:3.0:1788880428.671133:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306059804 00000100:00000010:1.0:1788880428.671135:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000800:00000200:0.0:1788880428.671139:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880428.671143:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880428.671143:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306060225 00000100:00000001:3.0:1788880428.671143:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306064847 00000100:00000001:3.0:1788880428.671165:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306059804 00000800:00000010:0.0:1788880428.671167:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:1.0:1788880428.671171:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306064847 00000100:00000001:2.0:1788880428.671171:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306064847 00000100:00000001:3.0:1788880428.671172:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306064847 00000400:00000200:0.0:1788880428.671177:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880428.671179:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306060225 00000100:00000010:1.0:1788880428.671180:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000100:00000001:3.0:1788880428.671180:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306059804 00000100:00000001:1.0:1788880428.671187:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880428.671187:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306064847 00000400:00000200:0.0:1788880428.671192:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880428.671193:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306059804 00000100:00000001:2.0:1788880428.671196:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306064847 00000100:00000001:1.0:1788880428.671197:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880428.671202:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000100:00000001:1.0:1788880428.671203:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880428.671203:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880428.671203:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306064847 00000400:00000010:0.0:1788880428.671214:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:2.0:1788880428.671214:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880428.671215:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880428.671215:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880428.671226:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880428.671227:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880428.671228:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880428.671232:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880428.671260:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880428.671264:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880428.671266:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880428.671267:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880428.671268:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880428.671270:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880428.671273:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880428.671275:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880428.671277:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:0.0:1788880428.671281:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880444 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880428.671281:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880428.671282:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880428.671283:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880428.671287:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880428.671290:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880428.671291:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880428.671292:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880428.671295:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880428.671296:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880428.671300:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880428.671308:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880428.671309:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880428.671325:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880428.671332:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880444 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880428.671340:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880428.671342:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880428.671358:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880428.671360:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000045330cf1. 00000400:00000200:0.0:1788880428.671363:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880428.671365:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880428.671367:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a3b2cf4a 00000400:00000010:0.0:1788880428.671369:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a3b2cf4a. 00000100:00000001:0.0:1788880428.671371:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880428.671374:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880428.671380:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880428.671382:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880428.671387:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880428.671389:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880428.671444:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880428.671446:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000400:00000200:0.0:1788880428.671450:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880428.671453:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880428.671455:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 00000400:00000010:0.0:1788880428.671457:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000100:00000001:0.0:1788880428.671466:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880428.671470:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880428.671477:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880428.671480:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880428.671486:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880428.671487:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880428.673010:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880428.673026:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880428.673029:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880428.673031:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880428.673042:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880428.673055:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b05f80 00000400:00000200:2.0:1788880428.673062:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x71d [1] + 224 00000800:00000001:2.0:1788880428.673070:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880428.673074:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880428.673076:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880428.673080:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880428.673087:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880428.673093:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880428.673102:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880428.673121:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880428.673137:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880428.673141:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880428.673143:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880428.673146:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880428.673164:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880428.673165:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880428.673170:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880428.673173:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06000 00000400:00000001:1.0:1788880428.673179:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880428.673182:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880428.673184:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005972c901. 00000400:00000200:1.0:1788880428.673189:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000400:00000010:1.0:1788880428.673191:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:1.0:1788880428.673194:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880428.673198:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x721 [1] + 224 00000100:00000200:1.0:1788880428.673200:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880428.673206:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880428.673211:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880428.673214:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:1.0:1788880428.673216:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880428.673221:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880428.673228:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880428.673230:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:2.0:1788880428.673253:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880428.673258:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880428.673263:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880428.673266:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:2.0:1788880428.673267:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880428.673269:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880428.673272:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880428.673276:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880428.673284:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=106 v=5 (5 5 0 0) 00000100:00001000:1.0:1788880428.673288:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=106 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880428.673290:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880428.673291:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880428.673294:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880428.673294:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:1.0:1788880428.673298:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880428.673300:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880428.673301:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788880428.673305:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880428.673306:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880428.673311:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880428.673315:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880428.673316:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880428.673320:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880428.673323:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880428.673324:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000040:1.0:1788880428.673328:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880428.673330:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880428.673334:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06080 00000100:00000001:3.0:1788880428.673335:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880428.673340:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880428.673343:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880428.673346:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000087a6361a. 00000100:00100000:1.0:1788880428.673348:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780656512:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880428.673352:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005071f3f0 00000400:00000010:3.0:1788880428.673355:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005071f3f0. 00000100:00000001:1.0:1788880428.673356:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880428.673359:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880428.673360:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788880428.673363:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x72d [1] + 224 00000100:00000200:3.0:1788880428.673367:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880428.673368:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000451fe31a x1875776780656512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880428.673371:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880428.673380:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880428.673385:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880428.673392:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880428.673396:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880428.673397:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880428.673402:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880428.673406:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880428.673412:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880428.673415:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002974b115. 00000100:00000001:3.0:1788880428.673420:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880428.673423:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880428.673424:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880428.673427:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:2.0:1788880428.673428:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880428.673429:0:201287:0:(client.c:1580:after_reply()) Process entered 00000020:00000040:1.0:1788880428.673431:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880428.673434:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880428.673439:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000a371c7dc. 02000000:00000001:3.0:1788880428.673439:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880428.673443:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880428.673447:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880428.673448:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880428.673451:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880428.673453:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880428.673456:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000451fe31a. 00000100:00001000:3.0:1788880428.673458:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=96 v=5 (5 5 0 0) 00000100:00000001:1.0:1788880428.673460:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880428.673461:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:3.0:1788880428.673462:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=96 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880428.673466:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880428.673468:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880428.673468:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880428.673470:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880428.673470:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880428.673474:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880428.673475:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880428.673476:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880428.673479:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880428.673480:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880428.673481:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880428.673486:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880428.673487:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880428.673491:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880428.673495:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880428.673497:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880428.673503:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880428.673511:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880428.673512:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880428.673515:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880428.673518:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880428.673519:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880428.673525:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880428.673529:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880428.673533:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00100000:3.0:1788880428.673541:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780656640:192.168.204.111@tcp:400: 00000400:00000010:0.0:1788880428.673543:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004100a205. 00000100:00000001:3.0:1788880428.673549:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880428.673554:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cf82a3fa 00000100:00000001:3.0:1788880428.673555:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000010:0.0:1788880428.673558:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cf82a3fa. 00000100:00000040:3.0:1788880428.673561:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000009ee4c6bf x1875776780656640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880428.673566:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880428.673572:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880428.673575:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000200:0.0:1788880428.673577:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880428.673579:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000fde20984. 02000000:00000001:3.0:1788880428.673585:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880428.673587:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880428.673590:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880428.673598:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880428.673605:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000131c1a55. 02000000:00000001:3.0:1788880428.673611:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880428.673614:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880428.673617:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000009ee4c6bf. 00000100:00000200:0.0:1788880428.673620:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880428.673623:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880428.673626:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880428.673631:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880428.673639:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880428.673642:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880428.673647:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880428.673648:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880428.673650:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880428.673652:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880428.673658:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880428.673661:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880428.673663:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880428.673667:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880428.673670:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880428.673678:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=106 v=5 (5 5 0 0) 00000100:00001000:0.0:1788880428.673683:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=80 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880428.673685:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880428.673687:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880428.673689:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880428.673694:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880428.673716:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880428.673720:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880428.673722:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880428.673724:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880428.673725:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880428.673729:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880428.673735:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880428.673738:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880428.673740:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880428.673743:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880428.673750:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780656768:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880428.673755:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880428.673757:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880428.673759:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006b32fd51 x1875776780656768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880444 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880428.673766:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880428.673768:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880428.673771:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008545eb31. 02000000:00000001:0.0:1788880428.673774:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880428.673776:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880428.673778:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880428.673780:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880428.673782:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000030024c2c. 02000000:00000001:0.0:1788880428.673785:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880428.673787:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880428.673788:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006b32fd51. 00000100:00000001:0.0:1788880428.673791:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880428.673792:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880428.673799:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880428.673806:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880428.673808:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880428.673810:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880428.673812:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880428.734381:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880428.734396:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880428.734403:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880428.734404:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880428.734484:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880428.734489:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880428.734492:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880429.758334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880429.758342:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880429.758348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880429.758349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880429.758542:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880429.758547:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880429.758551:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880430.782367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880430.782375:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880430.782382:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880430.782399:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880430.782453:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880430.782457:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880430.782460:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880431.806363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880431.806372:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880431.806379:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880431.806381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880431.806453:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880431.806460:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880431.806464:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880432.830340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880432.830348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880432.830355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880432.830356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880432.830412:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880432.830416:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880432.830419:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880433.790356:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306064847, at 4306064896 00000100:00000040:2.0:1788880433.790357:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306064847, at 4306064896 00000100:00000040:3.0:1788880433.790367:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306064847, at 4306064896 00000100:00000040:0.0:1788880433.790368:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:1.0:1788880433.790371:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000010:2.0:1788880433.790373:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000001:0.0:1788880433.790386:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880433.790386:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:0.0:1788880433.790412:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006b32fd51. 00000020:00000040:0.0:1788880433.790418:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880433.790423:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880433.790426:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880433.790430:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880433.790438:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000030024c2c. 00000100:00000001:0.0:1788880433.790445:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880433.790454:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880433.790466:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880433.790468:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880433.790474:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006b32fd51] to pc [ptlrpcd_00_00+0] req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880433.790504:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880433.790507:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880433.790511:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880433.790514:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000118779b0. 00000020:00000040:0.0:1788880433.790518:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880433.790521:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880433.790522:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880433.790524:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880433.790525:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880433.790530:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:0.0:1788880433.790532:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000131c1a55. 00000100:00000001:2.0:1788880433.790534:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880433.790537:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880433.790542:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880433.790545:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880433.790567:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880433.790570:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880433.790578:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880433.790580:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788880433.790582:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880433.790583:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880433.790589:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880433.790593:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000118779b0] to pc [ptlrpcd_00_01+1] req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880433.790596:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780656896:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880433.790598:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880433.790603:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880433.790608:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880433.790619:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880433.790622:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880433.790625:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880433.790628:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008c6f5069. 02000000:00000001:2.0:1788880433.790633:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880433.790634:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880433.790638:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:2.0:1788880433.790639:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8697e0a. 00000100:00000001:1.0:1788880433.790641:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880433.790645:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880433.790647:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880433.790650:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000010:0.0:1788880433.790652:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001d70df98. 00000400:00000010:2.0:1788880433.790653:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000078d4a4ff. 00000020:00000040:0.0:1788880433.790656:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:1.0:1788880433.790657:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880433.790662:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000200:2.0:1788880433.790663:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780656896, portal 25 02000000:00000001:0.0:1788880433.790666:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880433.790670:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:0.0:1788880433.790671:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880433.790673:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849362304 : -126822860189312 : ffff8ca7c0781180) 00000100:00000001:1.0:1788880433.790676:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000010:0.0:1788880433.790679:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000a371c7dc. 00000100:00000001:1.0:1788880433.790680:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880433.790684:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880433.790685:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880449 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880433.790686:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880433.790691:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880433.790695:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880433.790698:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880433.790707:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000118779b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780657024:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880433.790707:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880433.790717:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880433.790717:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880433.790718:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880433.790722:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880433.790724:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 02000000:00000001:1.0:1788880433.790725:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880433.790729:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880433.790731:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780656896, offset 0 00000100:00000040:0.0:1788880433.790732:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001d70df98] to pc [ptlrpcd_00_02+2] req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880433.790735:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880433.790741:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000005842920. 00000400:00000200:2.0:1788880433.790746:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880433.790747:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880433.790753:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005972c901. 00000100:00000001:0.0:1788880433.790757:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880433.790759:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 00000100:00000040:0.0:1788880433.790761:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:2.0:1788880433.790764:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:1.0:1788880433.790765:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780657024, portal 10 00000100:00000040:0.0:1788880433.790767:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880433.790771:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880433.790773:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849360512 : -126822860191104 : ffff8ca7c0780a80) 00000100:00000040:0.0:1788880433.790775:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11402) 00000400:00000200:2.0:1788880433.790783:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788880433.790787:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880433.790790:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880433.790790:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880433.790795:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880433.790798:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880433.790800:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880433.790801:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880433.790808:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880433.790809:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:1.0:1788880433.790812:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000100:00000200:1.0:1788880433.790817:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780657024, offset 0 00000400:00000200:2.0:1788880433.790817:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [202:202:256:4294967295] 192.168.204.111@tcp LPNI seq info [202:202:8:4294967295] 00000100:00000040:0.0:1788880433.790820:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880433.790829:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880433.790849:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880433.790853:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880433.790856:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880433.790860:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880433.790861:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880433.790863:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880433.790865:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880433.790869:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788880433.790871:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880433.790877:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000001d70df98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780657152:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880433.790877:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880433.790884:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880433.790885:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880433.790890:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1788880433.790893:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788880433.790894:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880433.790898:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880433.790901:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [203:203:255:4294967295] 192.168.204.111@tcp LPNI seq info [203:203:7:4294967295] 02000000:00000010:0.0:1788880433.790903:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000099f3ac40. 02000000:00000001:0.0:1788880433.790908:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880433.790911:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dcd03aa7. 00000400:00000200:1.0:1788880433.790917:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:0.0:1788880433.790919:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009fe01c6d. 00000100:00000200:0.0:1788880433.790925:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780657152, portal 10 00000800:00000200:1.0:1788880433.790926:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880433.790929:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880433.790931:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849368576 : -126822860183040 : ffff8ca7c0782a00) 00000800:00000010:1.0:1788880433.790933:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000040:0.0:1788880433.790940:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880433.790942:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880433.790952:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:1.0:1788880433.790956:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:0.0:1788880433.790960:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880433.790964:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000400:00000010:0.0:1788880433.790965:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a08ac001. 00000100:00000200:0.0:1788880433.790971:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780657152, offset 0 00000400:00000200:0.0:1788880433.790977:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880433.790982:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880433.790984:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:1.0:1788880433.790989:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 440656 usecs (441 jiffies) 00000400:00000200:0.0:1788880433.790990:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880433.790994:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880433.790996:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880433.791000:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880433.791002:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306065337 00000400:00000200:0.0:1788880433.791006:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [204:204:254:4294967295] 192.168.204.111@tcp LPNI seq info [204:204:6:4294967295] 00000800:00000010:2.0:1788880433.791011:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e53a8f34. 00000100:00000001:1.0:1788880433.791013:0:201286:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, simply extend its deadline from 4306064847 00000100:00000001:1.0:1788880433.791021:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306065337 00000400:00000200:0.0:1788880433.791025:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880433.791025:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:1.0:1788880433.791029:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000800:00000200:2.0:1788880433.791035:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880433.791036:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000800:00000200:2.0:1788880433.791041:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e53a8f34 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880433.791043:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306065337 00000100:00000001:2.0:1788880433.791047:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880433.791049:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000100:00000001:2.0:1788880433.791064:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306069967 00000100:00000001:1.0:1788880433.791067:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000800:00000200:0.0:1788880433.791068:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880433.791073:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306065337 00000800:00000010:0.0:1788880433.791074:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ac392bfe. 00000100:00000001:1.0:1788880433.791079:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306065337 00000100:00000010:1.0:1788880433.791087:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000100:00000001:2.0:1788880433.791089:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306069967 00000800:00000200:0.0:1788880433.791091:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880433.791094:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880433.791095:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306065337 00000800:00000200:0.0:1788880433.791100:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880433.791101:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880433.791103:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306069967 00000100:00000001:3.0:1788880433.791105:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:0.0:1788880433.791106:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ac392bfe type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880433.791108:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880433.791108:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306065337 00000100:00000001:3.0:1788880433.791113:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880433.791116:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880433.791116:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880433.791116:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306069967 00000100:00000001:2.0:1788880433.791123:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306065337 00000100:00000040:0.0:1788880433.791124:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 27327 usecs (28 jiffies) 00000100:00000001:1.0:1788880433.791124:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880433.791126:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880433.791130:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880433.791131:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306064924 00000100:00000001:2.0:1788880433.791133:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880433.791134:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880433.791135:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306069967 00000100:00000001:2.0:1788880433.791136:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880433.791138:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880433.791141:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306064924 00000100:00000001:1.0:1788880433.791143:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880433.791143:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880433.791145:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306069967 00000100:00000001:1.0:1788880433.791147:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880433.791148:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880433.791149:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306064924 00000100:00000001:1.0:1788880433.791152:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880433.791153:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306069967 00000100:00000001:2.0:1788880433.791153:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880433.791157:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880433.791158:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306064924 00000100:00000001:2.0:1788880433.791160:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880433.791162:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306069967 00000100:00000001:2.0:1788880433.791164:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880433.791166:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880433.791169:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880433.791170:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880433.791175:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880433.791177:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880433.791182:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880433.791183:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880433.791186:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880433.791187:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880433.791189:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880433.791191:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880433.791204:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880433.791209:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:1.0:1788880433.791213:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880433.791218:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880433.791263:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:1.0:1788880433.791266:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:1.0:1788880433.791271:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880433.791275:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880433.791285:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880433.791290:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880433.791297:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880433.791300:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880433.791316:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880433.791319:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e53a8f34. 00000400:00000200:1.0:1788880433.791323:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880433.791327:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880433.791331:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:1.0:1788880433.791333:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:1.0:1788880433.791336:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880433.791347:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880449 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880433.791354:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880433.791358:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880449 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880433.791364:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880433.791365:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880433.791421:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880433.791424:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ac392bfe. 00000400:00000200:1.0:1788880433.791427:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880433.791431:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880433.791433:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a08ac001 00000400:00000010:1.0:1788880433.791435:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a08ac001. 00000100:00000001:1.0:1788880433.791437:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880433.791440:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880433.791447:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880433.791449:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880433.791454:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880433.791455:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880433.793906:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880433.793923:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880433.793929:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880433.793931:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880433.794012:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880433.794047:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06180 00000400:00000200:0.0:1788880433.794068:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x73d [1] + 224 00000800:00000001:0.0:1788880433.794077:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880433.794082:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880433.794085:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880433.794089:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880433.794097:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880433.794104:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880433.794116:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880433.794136:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880433.794146:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880433.794150:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880433.794152:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880433.794154:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880433.794159:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880433.794165:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b06100 00000400:00000200:0.0:1788880433.794172:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x735 [1] + 224 00000800:00000001:0.0:1788880433.794176:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880433.794179:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880433.794180:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880433.794182:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880433.794186:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880433.794188:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880433.794193:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:1.0:1788880433.794197:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:0.0:1788880433.794198:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880433.794202:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880433.794206:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005972c901. 00000100:00000040:0.0:1788880433.794211:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880433.794214:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000400:00000010:1.0:1788880433.794218:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:1.0:1788880433.794225:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880433.794231:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880433.794265:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880433.794273:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788880433.794274:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880433.794285:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880433.794297:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880433.794298:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880433.794302:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880433.794303:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880433.794309:0:201286:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:0.0:1788880433.794313:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 02000000:00000001:1.0:1788880433.794315:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880433.794319:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:0.0:1788880433.794322:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06200 00000100:00000001:1.0:1788880433.794323:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880433.794329:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880433.794330:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x745 [1] + 224 02000000:00000001:1.0:1788880433.794330:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880433.794332:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880433.794336:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:2.0:1788880433.794337:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880433.794339:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880433.794340:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8697e0a. 00000800:00000001:0.0:1788880433.794342:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880433.794343:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=101 v=5 (5 5 0 0) 00000400:00000200:0.0:1788880433.794344:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880433.794344:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000078d4a4ff 00000400:00000010:2.0:1788880433.794347:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000078d4a4ff. 00000400:00000200:0.0:1788880433.794350:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00001000:1.0:1788880433.794352:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=101 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880433.794352:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880433.794353:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880433.794358:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000200:2.0:1788880433.794358:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880433.794359:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880433.794362:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880433.794369:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:0.0:1788880433.794371:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880433.794371:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880433.794380:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880433.794380:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880433.794381:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880433.794386:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880433.794388:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:2.0:1788880433.794388:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880433.794390:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880433.794390:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:1.0:1788880433.794393:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:0.0:1788880433.794395:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880433.794397:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000800:00000001:0.0:1788880433.794399:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880433.794399:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880433.794400:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880433.794403:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880433.794408:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=111 v=5 (5 5 0 0) 00000100:00000040:1.0:1788880433.794409:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:2.0:1788880433.794413:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=111 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880433.794416:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880433.794417:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880433.794419:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880433.794421:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880433.794424:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880433.794425:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880433.794429:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880433.794434:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:3.0:1788880433.794436:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880433.794438:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000001:3.0:1788880433.794440:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880433.794444:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dcd03aa7. 00000100:00000001:1.0:1788880433.794445:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880433.794448:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880433.794451:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009fe01c6d 00000100:00000001:2.0:1788880433.794452:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788880433.794455:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009fe01c6d. 00000100:00000040:1.0:1788880433.794456:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880433.794457:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880433.794461:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880433.794462:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880433.794471:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880433.794473:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000118779b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780657024:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880433.794478:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880433.794483:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880433.794490:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880433.794496:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000118779b0 x1875776780657024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880433.794501:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880433.794522:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880433.794523:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:2.0:1788880433.794524:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780656896:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880433.794527:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880433.794532:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880433.794534:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880433.794534:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880433.794535:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880433.794539:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880433.794539:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880433.794541:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880433.794545:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880433.794546:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006b32fd51 x1875776780656896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880433.794549:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000005842920. 00000100:00001000:3.0:1788880433.794551:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=111 v=5 (5 5 0 0) 00000100:00000001:2.0:1788880433.794556:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880433.794557:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:3.0:1788880433.794558:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=85 v=5 (1 1 0 0) 02000000:00000001:2.0:1788880433.794559:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000020:00000001:1.0:1788880433.794562:0:201286:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000010:2.0:1788880433.794562:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008c6f5069. 02000000:00000001:2.0:1788880433.794565:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000040:1.0:1788880433.794566:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880433.794569:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880433.794573:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880433.794573:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000020:00000001:1.0:1788880433.794574:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880433.794577:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880433.794578:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880433.794579:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000131c1a55. 02000000:00000010:2.0:1788880433.794581:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000030024c2c. 00010000:00000001:3.0:1788880433.794583:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:1.0:1788880433.794587:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880433.794587:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880433.794590:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:1.0:1788880433.794591:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00010000:00080000:3.0:1788880433.794591:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788880433.794594:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006b32fd51. 00000100:00000010:1.0:1788880433.794595:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000118779b0. 00000100:00000001:2.0:1788880433.794600:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880433.794602:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00010000:00000001:3.0:1788880433.794605:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880433.794606:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880433.794609:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880433.794612:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880433.794612:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880433.794613:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880433.794623:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880433.794626:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880433.794626:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880433.794629:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880433.794630:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880433.794632:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880433.794633:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880433.794635:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880433.794638:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880433.794639:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880433.794641:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880433.794647:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880433.794653:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880433.794655:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880433.794657:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880433.794660:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880433.794667:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000001d70df98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780657152:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880433.794671:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880433.794678:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880433.794680:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000001d70df98 x1875776780657152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880449 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880433.794686:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880433.794687:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880433.794689:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000099f3ac40. 02000000:00000001:3.0:1788880433.794692:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880433.794693:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880433.794694:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880433.794696:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880433.794697:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000a371c7dc. 02000000:00000001:3.0:1788880433.794700:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880433.794707:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880433.794708:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000001d70df98. 00000100:00000001:3.0:1788880433.794711:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880433.794712:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880433.794713:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880433.794718:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880433.794719:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880433.794720:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880433.794722:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880433.854343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880433.854357:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880433.854360:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880433.854365:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880433.854372:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880433.854375:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880433.854377:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880433.854380:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880433.854386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880433.854388:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880433.854438:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880433.854442:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880433.854445:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880434.878417:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880434.878430:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880434.878438:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880434.878439:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880434.878593:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880434.878599:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880434.878603:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880435.902431:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880435.902437:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880435.902441:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880435.902442:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880435.902516:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880435.902520:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880435.902523:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880436.928207:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880436.928214:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880436.928219:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880436.928222:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880436.928255:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880436.928257:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880436.928260:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880436.928262:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880436.928264:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880436.928266:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880436.928333:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880436.928337:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880436.928339:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880437.950347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880437.950355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880437.950362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880437.950364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880437.950423:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880437.950428:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880437.950430:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880438.910377:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306069967, at 4306070016 00000100:00000040:2.0:1788880438.910378:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306069967, at 4306070016 00000100:00000040:1.0:1788880438.910379:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306069967, at 4306070016 00000100:00000040:3.0:1788880438.910387:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880438.910400:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880438.910405:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000010:2.0:1788880438.910407:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000010:1.0:1788880438.910412:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000010:3.0:1788880438.910414:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000020:00000040:3.0:1788880438.910424:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880438.910431:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880438.910436:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880438.910443:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880438.910464:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5b428a0. 00000100:00000001:3.0:1788880438.910474:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880438.910490:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880438.910508:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880438.910511:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880438.910518:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_03+3] req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880438.910570:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880438.910573:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880438.910577:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880438.910580:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880438.910582:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880438.910584:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880438.910585:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880438.910587:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880438.910589:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:3.0:1788880438.910593:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880438.910631:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880438.910639:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880438.910639:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880438.910642:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_00+0] req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880438.910658:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880438.910659:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880438.910661:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880438.910664:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000020:00000040:3.0:1788880438.910666:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880438.910667:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880438.910668:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880438.910669:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880438.910672:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:3.0:1788880438.910674:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880438.910677:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880438.910682:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880438.910684:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880438.910686:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_01+1] req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880438.910691:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880438.910692:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880438.910695:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880438.910695:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1788880438.910699:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11407) 00000100:00000001:2.0:1788880438.910703:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880438.910707:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880438.910715:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306069967, at 4306070016 00000100:00000040:2.0:1788880438.910722:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:3.0:1788880438.910723:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000001:0.0:1788880438.910737:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880438.910740:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880438.910742:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880438.910744:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880438.910745:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880438.910750:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:0.0:1788880438.910754:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880438.910755:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880438.910761:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880438.910766:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880438.910769:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880438.910769:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780657408:192.168.204.111@tcp:400: 02000000:00000001:0.0:1788880438.910773:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880438.910777:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880438.910778:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880438.910779:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880438.910782:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780657280:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880438.910784:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880438.910799:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880438.910799:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880438.910803:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880438.910803:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880438.910809:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:2.0:1788880438.910809:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880438.910816:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005bd099ee. 02000000:00000010:2.0:1788880438.910816:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008d432156. 02000000:00000001:0.0:1788880438.910821:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880438.910821:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880438.910831:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8ada50c. 00000400:00000010:2.0:1788880438.910832:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8697e0a. 00000400:00000010:0.0:1788880438.910842:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000400:00000010:2.0:1788880438.910842:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000078d4a4ff. 00000100:00000200:2.0:1788880438.910850:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780657408, portal 10 00000100:00000001:2.0:1788880438.910854:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880438.910856:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000200:0.0:1788880438.910858:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780657280, portal 25 00000100:00000001:1.0:1788880438.910858:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880438.910864:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880438.910864:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880438.910865:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880438.910867:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000100:00000001:1.0:1788880438.910875:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880438.910879:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880438.910880:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880438.910881:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880438.910884:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880454 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880438.910888:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880438.910891:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880438.910894:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f779aa21. 00000100:00000200:2.0:1788880438.910900:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780657408, offset 0 00000100:00000001:0.0:1788880438.910901:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880438.910908:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880438.910913:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000033f4d237. 00000100:00000001:1.0:1788880438.910914:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:2.0:1788880438.910914:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880438.910917:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880438.910920:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:0.0:1788880438.910921:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780657280, offset 0 02000000:00000001:1.0:1788880438.910922:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880438.910927:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880438.910930:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780657536:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880438.910932:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880438.910934:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880438.910938:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880438.910942:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1788880438.910943:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880438.910945:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880438.910948:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880438.910951:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000010:1.0:1788880438.910964:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000006010d39b. 02000000:00000001:1.0:1788880438.910968:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880438.910969:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880438.910973:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:1.0:1788880438.910974:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005972c901. 00000400:00000200:2.0:1788880438.910979:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [205:205:256:4294967295] 192.168.204.111@tcp LPNI seq info [205:205:8:4294967295] 00000400:00000010:1.0:1788880438.910980:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 00000100:00000200:1.0:1788880438.910986:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780657536, portal 10 00000100:00000001:1.0:1788880438.910989:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880438.910998:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000400:00000200:2.0:1788880438.910999:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:1.0:1788880438.911007:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880438.911010:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880438.911012:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880438.911019:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:2.0:1788880438.911020:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000eaa15d72. 00000400:00000200:0.0:1788880438.911024:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880438.911029:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880438.911031:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788880438.911035:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788880438.911035:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:1.0:1788880438.911040:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880438.911045:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [206:206:255:4294967295] 192.168.204.111@tcp LPNI seq info [206:206:7:4294967295] 00000400:00000010:1.0:1788880438.911045:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000800:00000200:2.0:1788880438.911051:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:1.0:1788880438.911052:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780657536, offset 0 00000800:00000200:2.0:1788880438.911057:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000eaa15d72 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880438.911064:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880438.911072:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880438.911078:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880438.911080:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880438.911082:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b016e4f4. 00000100:00000040:2.0:1788880438.911087:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 433266 usecs (434 jiffies) 00000800:00000200:0.0:1788880438.911090:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880438.911093:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880438.911094:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306070450 00000800:00000200:0.0:1788880438.911099:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880438.911099:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1788880438.911106:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b016e4f4 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880438.911106:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:2.0:1788880438.911108:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ccadc63b. 00000400:00000200:1.0:1788880438.911112:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880438.911116:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880438.911118:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880438.911118:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880438.911120:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306075087 00000400:00000200:1.0:1788880438.911129:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [207:207:254:4294967295] 192.168.204.111@tcp LPNI seq info [207:207:6:4294967295] 00000100:00000001:2.0:1788880438.911130:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306070450 00000100:00000001:0.0:1788880438.911132:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306070450 00000100:00000010:2.0:1788880438.911136:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003dccfd54. 00000100:00000001:2.0:1788880438.911144:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880438.911145:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306075087 00000400:00000200:1.0:1788880438.911149:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880438.911152:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306070450 00000100:00000001:0.0:1788880438.911153:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306070450 00000800:00000200:1.0:1788880438.911158:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880438.911159:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045d7c07c. 00000100:00000001:0.0:1788880438.911161:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306075087 00000800:00000010:1.0:1788880438.911164:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:2.0:1788880438.911171:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000800:00000200:1.0:1788880438.911175:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880438.911180:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880438.911181:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306070450 00000100:00000001:0.0:1788880438.911184:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306070450 00000800:00000200:1.0:1788880438.911185:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:2.0:1788880438.911189:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005fb7b615. 00000100:00000001:0.0:1788880438.911191:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306075087 00000800:00000200:1.0:1788880438.911191:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000800:00000010:3.0:1788880438.911194:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000eaa15d72. 00000100:00000001:2.0:1788880438.911198:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880438.911203:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880438.911206:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880438.911208:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 26916 usecs (27 jiffies) 00000400:00000200:3.0:1788880438.911208:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880438.911209:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306070450 00000100:00000001:2.0:1788880438.911214:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880438.911220:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306070043 00000100:00000001:1.0:1788880438.911226:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306075087 00000100:00000001:0.0:1788880438.911227:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880438.911227:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880438.911264:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306070044 00000100:00000001:1.0:1788880438.911266:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306075087 00000100:00000001:1.0:1788880438.911268:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306070044 00000100:00000001:1.0:1788880438.911272:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306075087 00000100:00000001:1.0:1788880438.911274:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306070044 00000100:00000001:0.0:1788880438.911281:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880438.911282:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880438.911284:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306075087 00000400:00000200:3.0:1788880438.911284:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880438.911286:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880438.911292:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880438.911295:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880438.911295:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880438.911302:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880438.911303:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880438.911303:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880438.911306:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:3.0:1788880438.911308:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f779aa21 00000100:00000001:2.0:1788880438.911310:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880438.911311:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880438.911311:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880438.911315:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880438.911315:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880438.911316:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880438.911320:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880438.911320:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880438.911323:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880438.911325:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880438.911326:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f779aa21. 00000100:00000001:1.0:1788880438.911327:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880438.911328:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880438.911338:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880438.911342:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880438.911343:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880438.911349:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880438.911350:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880438.911359:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880438.911374:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880438.911378:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880438.911385:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880438.911394:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880438.911415:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880438.911417:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b016e4f4. 00000400:00000200:3.0:1788880438.911421:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880438.911425:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880438.911428:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000033f4d237 00000400:00000010:3.0:1788880438.911430:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000033f4d237. 00000100:00000001:3.0:1788880438.911432:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880438.911436:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880454 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880438.911443:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880438.911445:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880454 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880438.911451:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880438.911453:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880438.911560:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880438.911562:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:3.0:1788880438.911566:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880438.911570:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880438.911572:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:3.0:1788880438.911574:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:3.0:1788880438.911585:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880438.911597:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880438.911605:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880438.911607:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880438.911613:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880438.911615:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880438.914878:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880438.914894:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880438.914898:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880438.914900:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880438.914910:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880438.914931:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b06280 00000400:00000200:1.0:1788880438.914940:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x751 [1] + 224 00000800:00000001:1.0:1788880438.914946:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880438.914950:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880438.914952:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880438.914967:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880438.914975:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880438.914981:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880438.914991:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880438.915011:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880438.915021:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880438.915025:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880438.915026:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880438.915028:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880438.915032:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880438.915049:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06300 00000400:00000200:1.0:1788880438.915054:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x74d [1] + 224 00000100:00000001:0.0:1788880438.915056:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:1.0:1788880438.915058:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880438.915062:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:1.0:1788880438.915062:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880438.915066:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880438.915068:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:0.0:1788880438.915071:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880438.915075:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880438.915077:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8ada50c. 00000400:00000200:0.0:1788880438.915082:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000400:00000010:0.0:1788880438.915088:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00000100:00000001:0.0:1788880438.915092:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880438.915093:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880438.915098:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880438.915099:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880438.915127:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880438.915131:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880438.915141:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880438.915142:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880438.915144:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880438.915146:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880438.915147:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880438.915151:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:1.0:1788880438.915151:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880438.915155:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:1.0:1788880438.915155:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880438.915157:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880438.915160:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880438.915164:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880438.915167:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880438.915168:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06380 00000400:00000200:1.0:1788880438.915172:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x75d [1] + 224 00000100:00001000:0.0:1788880438.915174:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=116 v=5 (5 5 0 0) 00000800:00000001:1.0:1788880438.915174:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880438.915179:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=116 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880438.915183:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880438.915183:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:1.0:1788880438.915184:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880438.915186:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880438.915189:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:1.0:1788880438.915189:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880438.915189:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:0.0:1788880438.915194:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788880438.915196:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880438.915197:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880438.915197:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:2.0:1788880438.915201:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880438.915204:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8697e0a. 00000100:00000040:0.0:1788880438.915205:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880438.915210:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000078d4a4ff 00000100:00000001:0.0:1788880438.915212:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880438.915214:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000078d4a4ff. 00000100:00000001:0.0:1788880438.915215:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880438.915217:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880438.915220:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880438.915220:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:0.0:1788880438.915221:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880438.915229:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880438.915230:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880438.915231:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780657280:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880438.915275:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880438.915276:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880438.915281:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880438.915288:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780657280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880438.915288:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880438.915303:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880438.915306:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880438.915308:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:0.0:1788880438.915309:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005bd099ee. 00000100:00000001:2.0:1788880438.915311:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788880438.915314:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880438.915316:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:0.0:1788880438.915317:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000800:00000001:1.0:1788880438.915317:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000040:0.0:1788880438.915321:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880438.915321:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880438.915325:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:0.0:1788880438.915326:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880438.915330:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5b428a0. 00000800:00000001:1.0:1788880438.915330:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880438.915330:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880438.915334:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880438.915336:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880438.915340:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:2.0:1788880438.915340:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880438.915343:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:3.0:1788880438.915345:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880438.915347:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00001000:2.0:1788880438.915348:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=106 v=5 (5 5 0 0) 00000100:00000001:3.0:1788880438.915351:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880438.915357:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880438.915359:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=106 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880438.915360:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880438.915362:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880438.915369:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880438.915370:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000010:3.0:1788880438.915373:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005972c901. 00000100:00000001:0.0:1788880438.915374:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880438.915374:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880438.915378:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880438.915383:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000100:00000001:0.0:1788880438.915384:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:2.0:1788880438.915384:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000010:3.0:1788880438.915392:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:0.0:1788880438.915393:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880438.915394:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880438.915401:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880438.915402:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:2.0:1788880438.915407:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880438.915411:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000200:3.0:1788880438.915416:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880438.915418:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880438.915435:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880438.915441:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880438.915459:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880438.915463:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880438.915464:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880438.915466:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880438.915470:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880438.915471:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880438.915474:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880438.915476:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:2.0:1788880438.915479:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880438.915484:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880438.915491:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880438.915503:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=116 v=5 (5 5 0 0) 00000100:00100000:2.0:1788880438.915504:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780657408:192.168.204.111@tcp:400: 00000100:00001000:3.0:1788880438.915511:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=90 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880438.915516:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880438.915518:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880438.915519:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880438.915520:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880438.915526:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00010000:00080000:3.0:1788880438.915530:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880438.915533:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780657408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880438.915548:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880438.915553:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880438.915562:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008d432156. 02000000:00000001:2.0:1788880438.915567:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880438.915568:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880438.915571:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00010000:00000001:3.0:1788880438.915583:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880438.915586:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880438.915587:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000010:2.0:1788880438.915593:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 00000100:00000040:3.0:1788880438.915593:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880438.915600:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880438.915604:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880438.915605:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880438.915608:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880438.915610:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:2.0:1788880438.915615:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880438.915616:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880438.915618:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880438.915620:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880438.915624:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880438.915628:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880438.915634:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880438.915638:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880438.915641:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880438.915645:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880438.915647:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880438.915650:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880438.915658:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780657536:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880438.915663:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880438.915666:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880438.915667:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780657536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880454 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880438.915672:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880438.915674:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880438.915676:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000006010d39b. 02000000:00000001:3.0:1788880438.915679:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880438.915680:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880438.915681:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880438.915683:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880438.915685:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 02000000:00000001:3.0:1788880438.915688:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880438.915689:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880438.915690:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:3.0:1788880438.915693:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880438.915693:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880438.915700:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880438.915705:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880438.915706:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880438.915714:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880438.915716:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880438.974401:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880438.974409:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880438.974416:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880438.974417:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880438.974533:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880438.974537:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880438.974539:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880439.998376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880439.998386:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880439.998394:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880439.998395:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880439.998450:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880439.998454:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880439.998456:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880441.022313:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880441.022323:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880441.022330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880441.022332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880441.022392:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880441.022396:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880441.022399:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880442.046358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880442.046366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880442.046373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880442.046375:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880442.046448:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880442.046453:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880442.046456:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880443.070340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880443.070347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880443.070354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880443.070356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880443.070409:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880443.070413:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880443.070414:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880444.030343:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306075087, at 4306075136 00000100:00000040:1.0:1788880444.030343:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306075087, at 4306075136 00000100:00000040:2.0:1788880444.030343:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306075087, at 4306075136 00000100:00000040:3.0:1788880444.030344:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306075087, at 4306075136 00000100:00000010:1.0:1788880444.030359:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003dccfd54. 00000100:00000010:0.0:1788880444.030361:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ccadc63b. 00000100:00000010:2.0:1788880444.030363:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045d7c07c. 00000100:00000010:3.0:1788880444.030365:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005fb7b615. 00000100:00000040:3.0:1788880444.030385:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880444.030390:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880444.030398:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000020:00000040:3.0:1788880444.030402:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880444.030406:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880444.030409:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880444.030413:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880444.030426:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:3.0:1788880444.030434:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880444.030443:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880444.030453:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880444.030455:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880444.030461:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_02+2] req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880444.030482:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880444.030485:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880444.030488:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880444.030491:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000020:00000040:3.0:1788880444.030493:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880444.030496:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880444.030497:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880444.030498:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880444.030500:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880444.030503:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:3.0:1788880444.030504:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:1.0:1788880444.030505:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880444.030510:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880444.030510:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880444.030512:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880444.030515:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880444.030524:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880444.030528:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880444.030529:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880444.030533:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880444.030538:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880444.030541:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:1.0:1788880444.030542:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880444.030544:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880444.030549:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780657664:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880444.030551:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_03+3] req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880444.030557:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880444.030568:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880444.030571:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880444.030572:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880444.030574:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880444.030579:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1788880444.030580:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:3.0:1788880444.030583:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 02000000:00000010:1.0:1788880444.030586:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000fa0f7569. 00000100:00000001:0.0:1788880444.030587:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:3.0:1788880444.030587:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880444.030591:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880444.030592:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880444.030593:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880444.030596:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880444.030597:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:1.0:1788880444.030599:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 02000000:00000001:3.0:1788880444.030599:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880444.030605:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880444.030607:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000055d7d92. 00000400:00000010:1.0:1788880444.030609:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 00000100:00000001:3.0:1788880444.030613:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880444.030618:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780657664, portal 25 00000100:00000001:0.0:1788880444.030622:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788880444.030622:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880444.030624:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880444.030627:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880444.030627:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 02000000:00000001:0.0:1788880444.030633:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880444.030637:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880444.030638:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:1.0:1788880444.030639:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880460 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880444.030641:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880444.030642:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880444.030648:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_00+0] req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880444.030651:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780657792:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880444.030663:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880444.030666:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880444.030668:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880444.030671:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880444.030673:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880444.030675:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:1.0:1788880444.030678:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000641230df. 02000000:00000010:0.0:1788880444.030681:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009c91be43. 00000100:00000200:1.0:1788880444.030681:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780657664, offset 0 02000000:00000001:0.0:1788880444.030686:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880444.030687:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880444.030692:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880444.030693:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:0.0:1788880444.030695:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8ada50c. 00000100:00000040:3.0:1788880444.030700:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:0.0:1788880444.030702:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000100:00000001:2.0:1788880444.030704:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880444.030705:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11412) 00000400:00000200:1.0:1788880444.030706:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:0.0:1788880444.030707:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780657792, portal 10 00000100:00000001:2.0:1788880444.030708:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880444.030711:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880444.030713:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880444.030716:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880444.030717:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000040:2.0:1788880444.030718:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880444.030720:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880444.030722:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880444.030725:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880444.030727:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:0.0:1788880444.030730:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880444.030736:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880444.030746:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [208:208:256:4294967295] 192.168.204.111@tcp LPNI seq info [208:208:8:4294967295] 00000100:00000001:2.0:1788880444.030750:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880444.030753:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880444.030766:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880444.030770:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880444.030773:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788880444.030777:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880444.030783:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00100000:2.0:1788880444.030785:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780657920:192.168.204.111@tcp:400: 00000800:00000200:1.0:1788880444.030789:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880444.030792:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880444.030793:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:0.0:1788880444.030798:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 02000000:00000001:2.0:1788880444.030798:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000800:00000010:1.0:1788880444.030799:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 02000000:00000001:2.0:1788880444.030801:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880444.030804:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780657792, offset 0 02000000:00000001:2.0:1788880444.030804:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880444.030808:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000c21a3ff8. 00000800:00000200:1.0:1788880444.030812:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:2.0:1788880444.030813:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880444.030814:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880444.030821:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8697e0a. 00000400:00000200:0.0:1788880444.030825:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880444.030828:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:2.0:1788880444.030828:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000078d4a4ff. 00000400:00000200:0.0:1788880444.030831:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:2.0:1788880444.030834:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780657920, portal 10 00000800:00000200:1.0:1788880444.030835:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880444.030837:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880444.030839:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880444.030841:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000100:00000040:2.0:1788880444.030849:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880444.030851:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880444.030853:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880444.030870:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306080207 00000400:00000200:0.0:1788880444.030871:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880444.030872:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:1.0:1788880444.030876:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000100:00000040:2.0:1788880444.030878:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880444.030881:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [209:209:255:4294967295] 192.168.204.111@tcp LPNI seq info [209:209:7:4294967295] 00000400:00000010:2.0:1788880444.030883:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c6ecbca9. 00000100:00000001:1.0:1788880444.030886:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000200:2.0:1788880444.030887:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780657920, offset 0 00000400:00000200:2.0:1788880444.030895:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880444.030898:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306080207 00000400:00000200:0.0:1788880444.030903:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:1.0:1788880444.030903:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000100:00000001:1.0:1788880444.030908:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:0.0:1788880444.030912:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880444.030914:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306080207 00000400:00000200:2.0:1788880444.030914:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788880444.030919:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000100:00000010:1.0:1788880444.030920:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000400:00000200:2.0:1788880444.030922:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880444.030926:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:2.0:1788880444.030929:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788880444.030930:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880444.030931:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306080207 00000400:00000200:2.0:1788880444.030935:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:1.0:1788880444.030936:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000100:00000001:1.0:1788880444.030941:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:0.0:1788880444.030942:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880444.030942:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880444.030944:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880444.030948:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880444.030949:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880444.030951:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [210:210:254:4294967295] 192.168.204.111@tcp LPNI seq info [210:210:6:4294967295] 00000100:00000001:1.0:1788880444.030953:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880444.030955:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880444.030958:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:3.0:1788880444.030958:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:0.0:1788880444.030963:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 426142 usecs (427 jiffies) 00000100:00000001:1.0:1788880444.030966:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880444.030970:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:3.0:1788880444.030970:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000001:0.0:1788880444.030971:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306075563 00000400:00000200:2.0:1788880444.030971:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880444.030973:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880444.030976:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880444.030977:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880444.030978:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306080207 00000100:00000001:1.0:1788880444.030979:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880444.030984:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880444.030986:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306075563 00000100:00000001:0.0:1788880444.030991:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306080207 00000400:00000200:3.0:1788880444.030991:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000200:2.0:1788880444.030992:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880444.030998:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306075563 00000400:00000200:3.0:1788880444.031000:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000641230df 00000800:00000010:2.0:1788880444.031001:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000eaa15d72. 00000100:00000001:0.0:1788880444.031003:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306080207 00000400:00000010:3.0:1788880444.031004:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000641230df. 00000100:00000001:0.0:1788880444.031008:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306075563 00000800:00000200:2.0:1788880444.031010:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880444.031011:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880444.031014:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306080207 00000100:00000200:3.0:1788880444.031019:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880460 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880444.031020:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880444.031026:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880444.031026:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000eaa15d72 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880444.031032:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880444.031033:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880444.031036:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 26542 usecs (27 jiffies) 00000100:00000001:0.0:1788880444.031038:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880444.031039:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306075163 00000100:00000001:0.0:1788880444.031043:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880444.031043:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306080207 00000100:00000001:2.0:1788880444.031048:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306075163 00000100:00000001:2.0:1788880444.031049:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306080207 00000100:00000001:0.0:1788880444.031051:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880444.031051:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306075163 00000100:00000001:0.0:1788880444.031055:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880444.031055:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306080207 00000100:00000001:2.0:1788880444.031058:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306075163 00000100:00000001:2.0:1788880444.031061:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306080207 00000100:00000001:0.0:1788880444.031062:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880444.031063:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880444.031066:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880444.031067:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880444.031070:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880444.031071:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880444.031076:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880444.031076:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880444.031084:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880444.031085:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880444.031088:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880444.031089:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880444.031090:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880444.031093:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880444.031104:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880444.031108:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880460 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880444.031114:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880444.031116:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880444.031131:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880444.031133:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000400:00000200:3.0:1788880444.031136:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880444.031139:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880444.031141:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000400:00000010:3.0:1788880444.031143:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000001:3.0:1788880444.031145:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880444.031147:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880444.031153:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880444.031154:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880444.031160:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880444.031161:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880444.031192:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880444.031194:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000eaa15d72. 00000400:00000200:3.0:1788880444.031196:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880444.031200:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880444.031202:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c6ecbca9 00000400:00000010:3.0:1788880444.031203:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c6ecbca9. 00000100:00000001:3.0:1788880444.031211:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880444.031214:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880444.031221:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880444.031223:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880444.031228:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880444.031230:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880444.032347:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880444.032356:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880444.032358:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880444.032360:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880444.032366:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880444.032377:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b06400 00000400:00000200:3.0:1788880444.032384:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x765 [1] + 224 00000800:00000001:3.0:1788880444.032390:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880444.032429:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880444.032432:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880444.032435:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880444.032439:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880444.032443:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880444.032451:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880444.032465:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880444.032471:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880444.032473:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880444.032501:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880444.032504:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880444.032509:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880444.032511:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880444.032512:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:1.0:1788880444.032516:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:1.0:1788880444.032517:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:1.0:1788880444.032521:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880444.032525:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880444.032534:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880444.032541:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880444.032547:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880444.032549:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880444.032551:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880444.032552:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880444.032554:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880444.032556:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880444.032560:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=122 v=5 (5 5 0 0) 00000100:00001000:1.0:1788880444.032563:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=122 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880444.032565:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880444.032566:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880444.032568:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880444.032570:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880444.032572:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880444.032574:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880444.032578:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880444.032580:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880444.032582:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880444.032584:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880444.032592:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780657664:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880444.032598:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880444.032598:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880444.032600:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880444.032604:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780657664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880444.032605:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880444.032609:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880444.032612:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880444.032616:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880444.032620:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000200:3.0:1788880444.032621:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 02000000:00000010:1.0:1788880444.032624:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000fa0f7569. 02000000:00000001:1.0:1788880444.032629:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000200:3.0:1788880444.032630:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06480 00000020:00000001:1.0:1788880444.032632:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880444.032633:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880444.032636:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880444.032639:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 02000000:00000001:1.0:1788880444.032645:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880444.032652:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880444.032654:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:1.0:1788880444.032658:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880444.032659:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880444.032660:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880444.032664:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880444.032665:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880444.032667:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880444.032670:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880444.032680:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x76d [1] + 224 00000800:00000001:3.0:1788880444.032686:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880444.032690:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880444.032692:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880444.032695:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880444.032698:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880444.032701:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880444.032709:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880444.032724:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880444.032728:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880444.032731:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880444.032733:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880444.032734:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880444.032737:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880444.032743:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06500 00000400:00000200:3.0:1788880444.032748:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x775 [1] + 224 00000800:00000001:3.0:1788880444.032752:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880444.032754:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880444.032757:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880444.032757:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880444.032761:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:3.0:1788880444.032761:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:0.0:1788880444.032763:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880444.032765:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8ada50c. 00000400:00000200:3.0:1788880444.032769:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880444.032770:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000400:00000010:0.0:1788880444.032774:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00000100:00000001:0.0:1788880444.032776:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880444.032777:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880444.032781:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880444.032784:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880444.032810:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880444.032811:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880444.032832:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880444.032835:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880444.032837:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880444.032839:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880444.032840:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880444.032842:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880444.032842:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788880444.032844:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880444.032849:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=112 v=5 (5 5 0 0) 00000100:00001000:0.0:1788880444.032852:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=112 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880444.032854:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880444.032867:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880444.032867:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:0.0:1788880444.032871:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:3.0:1788880444.032873:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880444.032875:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880444.032879:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880444.032881:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880444.032884:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880444.032887:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:0.0:1788880444.032889:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:2.0:1788880444.032890:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880444.032893:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000010:2.0:1788880444.032894:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8697e0a. 00000100:00000001:0.0:1788880444.032896:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880444.032900:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000078d4a4ff 00000100:00000040:0.0:1788880444.032903:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880444.032911:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880444.032914:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000078d4a4ff. 00000100:00000001:0.0:1788880444.032915:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880444.032918:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880444.032921:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:0.0:1788880444.032923:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880444.032928:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880444.032941:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780657792:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880444.032949:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880444.032951:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880444.032952:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880444.032956:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780657792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880444.032975:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880444.032976:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880444.032976:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880444.032978:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788880444.032980:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009c91be43. 02000000:00000001:0.0:1788880444.032984:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880444.032985:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880444.032987:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880444.032987:0:201285:0:(client.c:1580:after_reply()) Process entered 00000020:00000001:0.0:1788880444.032992:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880444.032993:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 02000000:00000001:2.0:1788880444.032994:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:0.0:1788880444.032997:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880444.032997:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880444.032999:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880444.033000:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880444.033004:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 02000000:00000001:2.0:1788880444.033004:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880444.033010:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880444.033011:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880444.033012:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=122 v=5 (5 5 0 0) 00000100:00000001:0.0:1788880444.033016:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880444.033017:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=96 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880444.033022:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880444.033022:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880444.033026:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880444.033026:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880444.033032:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:2.0:1788880444.033032:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880444.033037:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880444.033043:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880444.033045:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880444.033047:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880444.033048:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880444.033049:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880444.033050:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880444.033053:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880444.033058:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880444.033060:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880444.033061:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880444.033064:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880444.033077:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780657920:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880444.033080:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880444.033081:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880444.033083:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780657920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880460 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880444.033086:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880444.033087:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880444.033088:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000c21a3ff8. 02000000:00000001:2.0:1788880444.033090:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880444.033091:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880444.033092:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880444.033093:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880444.033094:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000055d7d92. 02000000:00000001:2.0:1788880444.033095:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880444.033096:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880444.033097:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000001:2.0:1788880444.033098:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880444.033099:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880444.033100:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880444.033102:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880444.033103:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880444.033104:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880444.033105:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880444.094346:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880444.094354:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880444.094356:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880444.094360:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880444.094364:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880444.094367:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880444.094369:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880444.094371:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880444.094375:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880444.094377:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880444.094430:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880444.094433:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880444.094435:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880445.118369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880445.118378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880445.118386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880445.118388:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880445.118451:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880445.118457:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880445.118460:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880446.142296:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880446.142303:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880446.142309:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880446.142310:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880446.142804:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880446.142807:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880446.142810:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880447.166382:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880447.166391:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880447.166399:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880447.166403:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880447.166407:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880447.166411:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880447.166415:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880447.166419:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880447.166422:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880447.166426:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880447.166488:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880447.166509:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880447.166514:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880448.190349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880448.190372:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880448.190378:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880448.190378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880448.190437:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880448.190441:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880448.190444:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880449.150359:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306080207, at 4306080256 00000100:00000040:1.0:1788880449.150359:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306080207, at 4306080256 00000100:00000010:0.0:1788880449.150377:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000010:1.0:1788880449.150379:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000040:3.0:1788880449.150503:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306080207, at 4306080256 00000100:00000010:3.0:1788880449.150508:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000040:0.0:1788880449.150524:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880449.150530:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880449.150539:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ff0be673. 00000020:00000040:0.0:1788880449.150544:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880449.150548:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880449.150551:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880449.150558:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880449.150569:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006df406c6. 00000100:00000001:0.0:1788880449.150576:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880449.150588:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880449.150598:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880449.150601:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880449.150607:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ff0be673] to pc [ptlrpcd_00_01+1] req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880449.150632:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880449.150635:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880449.150639:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880449.150642:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001d70df98. 00000020:00000040:0.0:1788880449.150645:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880449.150648:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880449.150649:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880449.150651:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880449.150652:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880449.150655:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b016e4f4. 00000100:00000001:0.0:1788880449.150659:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880449.150659:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880449.150665:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880449.150672:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880449.150674:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880449.150677:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880449.150682:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880449.150693:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880449.150696:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880449.150699:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001d70df98] to pc [ptlrpcd_00_02+2] req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880449.150702:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880449.150705:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880449.150712:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880449.150712:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880449.150720:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780658048:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880449.150724:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880449.150726:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880449.150733:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880449.150738:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880449.150738:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880449.150743:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000040:2.0:1788880449.150745:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306080207, at 4306080256 00000100:00000010:0.0:1788880449.150747:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000118779b0. 02000000:00000001:3.0:1788880449.150752:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880449.150754:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880449.150765:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880449.150766:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880449.150769:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880449.150771:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880449.150772:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:3.0:1788880449.150772:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000033d66d82. 00000100:00000010:2.0:1788880449.150773:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 02000000:00000001:0.0:1788880449.150775:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880449.150779:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880449.150781:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880449.150783:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880449.150788:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006f95a419. 02000000:00000010:0.0:1788880449.150792:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f0a0057b. 00000100:00000001:1.0:1788880449.150792:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880449.150795:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880449.150796:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a165c7f9. 00000100:00000001:0.0:1788880449.150798:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880449.150799:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880449.150803:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:3.0:1788880449.150803:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780658048, portal 25 02000000:00000001:1.0:1788880449.150806:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880449.150806:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880449.150807:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880449.150808:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000100:00000040:3.0:1788880449.150819:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880465 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880449.150820:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788880449.150821:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000001d70df98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780658176:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880449.150822:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880449.150828:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880449.150828:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880449.150829:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880449.150835:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880449.150840:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880449.150841:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880449.150843:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880449.150848:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000118779b0] to pc [ptlrpcd_00_03+3] req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880449.150848:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000016c67947. 00000100:00000040:3.0:1788880449.150848:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880449.150850:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880449.150851:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000400:00000010:1.0:1788880449.150856:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000200:3.0:1788880449.150856:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780658048, offset 0 00000400:00000010:1.0:1788880449.150865:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 00000400:00000200:3.0:1788880449.150867:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880449.150869:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880449.150874:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:1.0:1788880449.150875:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780658176, portal 10 00000100:00000001:1.0:1788880449.150881:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880449.150882:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880449.150883:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849368576 : -126822860183040 : ffff8ca7c0782a00) 00000400:00000200:3.0:1788880449.150883:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880449.150886:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880449.150895:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11417) 00000100:00000001:2.0:1788880449.150895:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880449.150896:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880449.150896:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880449.150902:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880449.150909:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788880449.150909:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:1.0:1788880449.150921:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880449.150921:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880449.150921:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:1.0:1788880449.150929:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000974f52f4. 00000400:00000200:3.0:1788880449.150931:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:1.0:1788880449.150935:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780658176, offset 0 00000400:00000200:3.0:1788880449.150940:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [211:211:256:4294967295] 192.168.204.111@tcp LPNI seq info [211:211:8:4294967295] 00000400:00000200:1.0:1788880449.150959:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880449.150968:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880449.150974:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880449.150975:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788880449.150986:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880449.150990:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880449.150997:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880449.150998:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788880449.150998:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880449.151008:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000400:00000200:1.0:1788880449.151009:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:2.0:1788880449.151010:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000118779b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780658304:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880449.151015:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880449.151020:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788880449.151020:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880449.151024:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788880449.151026:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880449.151030:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [212:212:255:4294967295] 192.168.204.111@tcp LPNI seq info [212:212:7:4294967295] 00000800:00000200:3.0:1788880449.151031:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:2.0:1788880449.151032:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880449.151039:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880449.151040:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880449.151040:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880449.151051:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:1.0:1788880449.151052:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880449.151064:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 02000000:00000010:2.0:1788880449.151064:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008e83276d. 00000100:00000001:3.0:1788880449.151066:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880449.151071:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306085327 02000000:00000001:2.0:1788880449.151073:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880449.151076:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002aba003e. 00000800:00000200:1.0:1788880449.151083:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880449.151083:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000010:2.0:1788880449.151084:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8697e0a. 00000100:00000001:3.0:1788880449.151088:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306085327 00000800:00000200:1.0:1788880449.151091:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880449.151094:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000010:3.0:1788880449.151095:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c7c25dfc. 00000400:00000010:2.0:1788880449.151097:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000078d4a4ff. 00000100:00000001:1.0:1788880449.151098:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880449.151098:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880449.151104:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306085327 00000100:00000040:1.0:1788880449.151108:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 419203 usecs (420 jiffies) 00000100:00000010:3.0:1788880449.151109:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000026aaaaf2. 00000100:00000200:2.0:1788880449.151110:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780658304, portal 10 00000100:00000001:1.0:1788880449.151112:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306080676 00000100:00000001:3.0:1788880449.151113:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880449.151117:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880449.151118:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306085327 00000100:00000001:3.0:1788880449.151119:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306085327 00000100:00000001:2.0:1788880449.151120:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849360512 : -126822860191104 : ffff8ca7c0780a80) 00000100:00000001:1.0:1788880449.151122:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306080676 00000100:00000010:3.0:1788880449.151122:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b0b8cfdf. 00000100:00000001:1.0:1788880449.151128:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306085327 00000100:00000001:1.0:1788880449.151131:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306080676 00000100:00000001:3.0:1788880449.151131:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000040:2.0:1788880449.151135:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880449.151137:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880449.151141:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880449.151144:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880449.151145:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306085327 00000100:00000001:3.0:1788880449.151146:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880449.151147:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306080676 00000800:00000200:0.0:1788880449.151156:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880449.151156:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306085327 00000100:00000001:3.0:1788880449.151156:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880449.151160:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880449.151163:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:0.0:1788880449.151167:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000100:00000001:1.0:1788880449.151170:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880449.151170:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880449.151170:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880449.151174:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880449.151177:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880449.151179:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880449.151182:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880449.151184:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880449.151185:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880449.151189:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880449.151191:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000010:2.0:1788880449.151193:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c6ecbca9. 00000100:00000001:1.0:1788880449.151194:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880449.151194:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880449.151195:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880449.151199:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:2.0:1788880449.151205:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780658304, offset 0 00000100:00000001:1.0:1788880449.151206:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880449.151208:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000100:00000001:1.0:1788880449.151214:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880449.151215:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000001:0.0:1788880449.151223:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:2.0:1788880449.151229:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:0.0:1788880449.151263:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880465 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880449.151275:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880449.151286:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880449.151293:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788880449.151294:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880449.151306:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:0.0:1788880449.151307:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880465 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880449.151314:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880449.151323:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [213:213:254:4294967295] 192.168.204.111@tcp LPNI seq info [213:213:6:4294967295] 00000100:00000001:0.0:1788880449.151335:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880449.151338:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:2.0:1788880449.151347:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880449.151354:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880449.151360:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e53a8f34. 00000800:00000200:2.0:1788880449.151367:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880449.151375:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880449.151379:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e53a8f34 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880449.151383:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880449.151387:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 26127 usecs (27 jiffies) 00000100:00000001:2.0:1788880449.151390:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306080284 00000100:00000001:2.0:1788880449.151393:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306085327 00000100:00000001:2.0:1788880449.151395:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306080284 00000100:00000001:2.0:1788880449.151397:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306085327 00000100:00000001:2.0:1788880449.151399:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306080284 00000100:00000001:2.0:1788880449.151401:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306085327 00000100:00000001:2.0:1788880449.151403:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306080284 00000100:00000001:2.0:1788880449.151406:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306085327 00000800:00000200:0.0:1788880449.151413:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880449.151417:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:2.0:1788880449.151421:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880449.151424:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880449.151427:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880449.151432:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880449.151437:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880449.151438:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880449.151442:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000974f52f4 00000400:00000010:0.0:1788880449.151446:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000974f52f4. 00000100:00000001:2.0:1788880449.151449:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880449.151450:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880449.151453:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000200:0.0:1788880449.151457:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880449.151460:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880449.151462:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880449.151467:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880449.151479:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880449.151483:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880449.151487:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880449.151494:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880449.151496:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880449.151509:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880449.151511:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e53a8f34. 00000400:00000200:0.0:1788880449.151514:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880449.151517:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880449.151519:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c6ecbca9 00000400:00000010:0.0:1788880449.151520:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c6ecbca9. 00000100:00000001:0.0:1788880449.151523:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880449.151525:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880449.151531:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880449.151533:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880449.151548:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880449.151550:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880449.152924:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880449.152935:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880449.152938:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880449.152940:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880449.152958:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880449.152969:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06600 00000400:00000200:3.0:1788880449.152976:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x785 [1] + 224 00000800:00000001:3.0:1788880449.152982:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880449.153000:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880449.153003:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880449.153006:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880449.153011:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880449.153015:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880449.153024:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880449.153045:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880449.153052:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880449.153055:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880449.153368:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880449.153371:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880449.153377:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880449.153378:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880449.153380:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:1.0:1788880449.153384:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:1.0:1788880449.153386:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:1.0:1788880449.153388:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880449.153392:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880449.153401:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880449.153406:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880449.153408:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880449.153410:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880449.153412:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880449.153414:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880449.153431:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880449.153434:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880449.153442:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=117 v=5 (5 5 0 0) 00000100:00001000:1.0:1788880449.153446:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=117 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880449.153449:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880449.153450:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880449.153452:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880449.153455:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880449.153458:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880449.153464:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880449.153465:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880449.153467:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880449.153471:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880449.153489:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880449.153492:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880449.153493:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880449.153496:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880449.153502:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000001d70df98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780658176:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880449.153506:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880449.153508:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880449.153516:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000001d70df98 x1875776780658176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880449.153521:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880449.153523:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880449.153525:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000016c67947. 02000000:00000001:1.0:1788880449.153530:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880449.153532:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880449.153533:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880449.153536:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880449.153538:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b016e4f4. 02000000:00000001:1.0:1788880449.153541:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880449.153543:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880449.153544:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000001d70df98. 00000100:00000001:1.0:1788880449.153547:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880449.153548:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880449.153550:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880449.153555:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880449.153557:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880449.153559:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880449.153561:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880449.153913:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880449.153932:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880449.153936:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880449.153938:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880449.153948:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880449.153962:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06680 00000400:00000200:3.0:1788880449.153969:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x78d [1] + 224 00000800:00000001:3.0:1788880449.153975:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880449.153978:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880449.153979:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880449.153982:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880449.154004:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880449.154009:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880449.154018:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880449.154042:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880449.154050:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880449.154054:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880449.154055:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880449.154057:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880449.154061:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880449.154067:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b06580 00000400:00000200:3.0:1788880449.154073:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x77d [1] + 224 00000800:00000001:3.0:1788880449.154076:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880449.154081:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880449.154083:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880449.154085:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880449.154089:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880449.154092:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880449.154100:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880449.154127:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880449.154132:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880449.154135:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880449.154137:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880449.154143:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880449.154150:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880449.154153:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880449.154155:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8697e0a. 00000400:00000200:2.0:1788880449.154160:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000078d4a4ff 00000400:00000010:2.0:1788880449.154162:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000078d4a4ff. 00000100:00000001:0.0:1788880449.154204:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880449.154208:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880449.154295:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:0.0:1788880449.154296:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880449.154300:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000200:2.0:1788880449.154303:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880449.154305:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006f95a419. 00000400:00000200:0.0:1788880449.154315:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a165c7f9 00000400:00000010:0.0:1788880449.154322:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a165c7f9. 00000100:00000001:0.0:1788880449.154329:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880449.154341:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880449.154342:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880449.154372:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880449.154376:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880449.154381:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000200:0.0:1788880449.154386:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880449.154388:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880449.154397:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880449.154399:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880449.154404:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880449.154406:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880449.154408:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880449.154410:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:2.0:1788880449.154416:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880449.154417:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880449.154422:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880449.154425:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880449.154429:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=127 v=5 (5 5 0 0) 00000100:00001000:0.0:1788880449.154430:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=127 v=5 (5 5 0 0) 00000100:00001000:2.0:1788880449.154444:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=101 v=5 (1 1 0 0) 00000100:00001000:0.0:1788880449.154445:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=127 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880449.154457:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880449.154458:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880449.154462:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880449.154464:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880449.154472:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880449.154472:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880449.154479:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880449.154481:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:2.0:1788880449.154484:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880449.154488:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880449.154549:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880449.154551:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880449.154552:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880449.154554:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880449.154581:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780658048:192.168.204.111@tcp:400: 00010000:00000001:2.0:1788880449.154595:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880449.154596:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880449.154606:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880449.154609:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ff0be673 x1875776780658048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880449.154609:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880449.154615:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880449.154625:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880449.154626:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880449.154628:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880449.154630:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880449.154635:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000033d66d82. 00000100:00000040:2.0:1788880449.154640:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880449.154642:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880449.154645:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880449.154647:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880449.154657:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880449.154661:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006df406c6. 00000100:00000001:2.0:1788880449.154663:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880449.154667:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880449.154668:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:0.0:1788880449.154672:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788880449.154672:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:0.0:1788880449.154677:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ff0be673. 00000100:00000040:2.0:1788880449.154684:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880449.154686:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880449.154689:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880449.154694:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:2.0:1788880449.154703:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000118779b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780658304:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880449.154710:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880449.154713:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880449.154714:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880449.154729:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880449.154729:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880449.154735:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880449.154740:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000118779b0 x1875776780658304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880465 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880449.154750:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880449.154754:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880449.154770:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008e83276d. 02000000:00000001:2.0:1788880449.154775:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880449.154777:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880449.154778:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880449.154781:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880449.154783:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f0a0057b. 02000000:00000001:2.0:1788880449.154786:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880449.154788:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880449.154789:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000118779b0. 00000100:00000001:2.0:1788880449.154793:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880449.154794:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880449.154803:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880449.154808:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880449.154810:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880449.154812:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880449.154814:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880449.214356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880449.214365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880449.214374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880449.214375:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880449.214515:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880449.214520:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880449.214523:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880450.238342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880450.238348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880450.238353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880450.238353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880450.238396:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880450.238399:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880450.238401:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880451.262344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880451.262352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880451.262358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880451.262359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880451.262441:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880451.262447:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880451.262450:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880452.286399:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880452.286409:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880452.286418:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880452.286419:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880452.286484:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880452.286489:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880452.286493:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880453.310352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880453.310361:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880453.310364:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880453.310367:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880453.310372:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880453.310374:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880453.310376:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880453.310378:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880453.310382:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880453.310383:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880453.310537:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880453.310545:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880453.310551:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880454.270363:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306085327, at 4306085376 00000100:00000040:2.0:1788880454.270368:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306085327, at 4306085376 00000100:00000040:0.0:1788880454.270371:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306085327, at 4306085376 00000100:00000040:3.0:1788880454.270382:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306085327, at 4306085376 00000100:00000010:2.0:1788880454.270384:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000026aaaaf2. 00000100:00000010:0.0:1788880454.270386:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002aba003e. 00000100:00000010:1.0:1788880454.270390:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c7c25dfc. 00000100:00000010:3.0:1788880454.270405:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b0b8cfdf. 00000100:00000040:2.0:1788880454.270426:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880454.270443:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880454.270451:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000629edf55. 00000020:00000040:2.0:1788880454.270456:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880454.270460:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880454.270462:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880454.270466:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880454.270472:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 00000100:00000001:2.0:1788880454.270479:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880454.270488:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880454.270497:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880454.270512:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880454.270518:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000629edf55] to pc [ptlrpcd_00_00+0] req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880454.270549:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880454.270551:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880454.270554:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880454.270556:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cc13c26b. 00000020:00000040:2.0:1788880454.270559:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880454.270561:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880454.270562:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880454.270563:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880454.270567:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 00000100:00000001:2.0:1788880454.270570:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880454.270574:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880454.270578:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880454.270582:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:0.0:1788880454.270583:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:2.0:1788880454.270584:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880454.270586:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880454.270591:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cc13c26b] to pc [ptlrpcd_00_01+1] req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880454.270594:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880454.270597:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880454.270601:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880454.270604:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880454.270606:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880454.270607:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880454.270612:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880454.270613:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880454.270616:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880454.270619:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000005a8bb1de. 00000020:00000040:2.0:1788880454.270621:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880454.270623:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880454.270624:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880454.270625:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880454.270628:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:2.0:1788880454.270631:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880454.270635:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880454.270652:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880454.270653:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880454.270656:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a8bb1de] to pc [ptlrpcd_00_02+2] req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880454.270692:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880454.270698:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880454.270708:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880454.270708:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880454.270710:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880454.270717:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880454.270718:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:1.0:1788880454.270720:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880454.270721:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11423) 00000100:00000001:3.0:1788880454.270726:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880454.270727:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880454.270729:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880454.270734:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880454.270734:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880454.270738:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1788880454.270740:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880454.270740:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880454.270741:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880454.270745:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880454.270749:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780658432:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880454.270758:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880454.270762:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880454.270768:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880454.270769:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:0.0:1788880454.270771:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880454.270774:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880454.270774:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880454.270776:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880454.270778:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788880454.270782:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880454.270783:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:1.0:1788880454.270786:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000010:0.0:1788880454.270790:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e6a93e02. 02000000:00000001:1.0:1788880454.270793:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880454.270793:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880454.270795:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880454.270808:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780658688:192.168.204.111@tcp:400: 00000400:00000010:0.0:1788880454.270810:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8ada50c. 00000400:00000010:0.0:1788880454.270822:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000100:00000001:1.0:1788880454.270829:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880454.270832:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000200:0.0:1788880454.270833:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780658432, portal 25 00000100:00000001:3.0:1788880454.270837:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880454.270838:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1788880454.270843:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880454.270844:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938432768 : -126823771118848 : ffff8ca78a2c6300) 02000000:00000001:1.0:1788880454.270846:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880454.270848:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880454.270850:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880454.270853:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880470 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880454.270854:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000010:1.0:1788880454.270855:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000064b136ca. 02000000:00000001:1.0:1788880454.270864:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880454.270866:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880454.270872:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000001:0.0:1788880454.270879:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:1.0:1788880454.270880:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 00000100:00100000:3.0:1788880454.270881:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780658560:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880454.270889:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780658688, portal 10 00000100:00000040:0.0:1788880454.270893:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880454.270897:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000010:0.0:1788880454.270902:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000100:00000001:1.0:1788880454.270903:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938427392 : -126823771124224 : ffff8ca78a2c4e00) 00000100:00000001:3.0:1788880454.270904:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000200:0.0:1788880454.270914:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780658432, offset 0 02000000:00000001:3.0:1788880454.270917:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:1.0:1788880454.270923:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880454.270924:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880454.270925:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880454.270933:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880454.270939:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:3.0:1788880454.270942:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000659bd78d. 00000400:00000200:0.0:1788880454.270943:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:1.0:1788880454.270952:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880454.270952:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880454.270957:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:1.0:1788880454.270960:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000038c5ac58. 00000400:00000010:3.0:1788880454.270964:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000ea759cb. 00000100:00000200:1.0:1788880454.270966:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780658688, offset 0 00000400:00000200:0.0:1788880454.270967:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880454.270972:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880454.270974:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:3.0:1788880454.270977:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e78d7264. 00000400:00000200:1.0:1788880454.270984:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880454.270989:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [214:214:256:4294967295] 192.168.204.111@tcp LPNI seq info [214:214:8:4294967295] 00000100:00000200:3.0:1788880454.270995:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780658560, portal 10 00000100:00000001:3.0:1788880454.271000:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880454.271002:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938433664 : -126823771117952 : ffff8ca78a2c6680) 00000400:00000200:0.0:1788880454.271010:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788880454.271015:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880454.271038:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880454.271047:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880454.271053:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880454.271055:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788880454.271058:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000400:00000010:3.0:1788880454.271059:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000200:3.0:1788880454.271066:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780658560, offset 0 00000400:00000200:1.0:1788880454.271067:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880454.271077:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788880454.271085:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880454.271086:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880454.271091:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880454.271098:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788880454.271101:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880454.271107:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880454.271107:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [215:215:255:4294967295] 192.168.204.111@tcp LPNI seq info [215:215:7:4294967295] 00000400:00000200:1.0:1788880454.271119:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880454.271123:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880454.271125:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880454.271127:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:0.0:1788880454.271130:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306090447 00000400:00000200:3.0:1788880454.271134:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:0.0:1788880454.271137:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000975d5882. 00000800:00000200:1.0:1788880454.271141:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880454.271142:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880454.271144:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880454.271155:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306090447 00000400:00000200:3.0:1788880454.271155:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880454.271157:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:0.0:1788880454.271158:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009ac8d70b. 00000800:00000200:1.0:1788880454.271163:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880454.271164:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880454.271174:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306090447 00000100:00000001:1.0:1788880454.271175:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880454.271178:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:1.0:1788880454.271184:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 25753 usecs (26 jiffies) 00000100:00000010:0.0:1788880454.271187:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005b8bebe2. 00000800:00000200:2.0:1788880454.271188:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:3.0:1788880454.271191:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880454.271192:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306085402 00000800:00000010:2.0:1788880454.271196:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000100:00000001:0.0:1788880454.271198:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880454.271209:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306090447 00000400:00000200:2.0:1788880454.271209:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880454.271209:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [216:216:254:4294967295] 192.168.204.111@tcp LPNI seq info [216:216:6:4294967295] 00000100:00000001:0.0:1788880454.271211:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306090447 00000100:00000001:1.0:1788880454.271223:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306085402 00000400:00000200:2.0:1788880454.271225:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000010:0.0:1788880454.271226:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c0bfe00. 00000100:00000001:1.0:1788880454.271274:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306090447 00000100:00000001:0.0:1788880454.271278:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880454.271281:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306085403 00000100:00000001:0.0:1788880454.271289:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880454.271290:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306090447 00000400:00000200:3.0:1788880454.271299:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880454.271301:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880454.271301:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306085403 00000100:00000001:1.0:1788880454.271310:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306090447 00000100:00000001:0.0:1788880454.271312:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880454.271312:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000800:00000200:3.0:1788880454.271317:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880454.271320:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880454.271320:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:0.0:1788880454.271321:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880454.271326:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000100:00000001:0.0:1788880454.271327:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880454.271328:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880454.271329:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880454.271334:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880454.271336:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880454.271338:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880454.271339:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880470 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880454.271344:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880454.271347:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880454.271347:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880454.271352:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880454.271358:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880454.271363:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:3.0:1788880454.271369:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880454.271370:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880454.271374:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880454.271374:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000200:3.0:1788880454.271377:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880454.271380:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880454.271382:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880470 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880454.271391:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880454.271392:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880454.271397:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000040:3.0:1788880454.271398:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 412496 usecs (413 jiffies) 00000100:00000001:3.0:1788880454.271404:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306085790 00000100:00000001:3.0:1788880454.271407:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306090447 00000100:00000001:3.0:1788880454.271411:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306085790 00000100:00000001:3.0:1788880454.271414:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306090447 00000100:00000001:3.0:1788880454.271417:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306085790 00000100:00000001:3.0:1788880454.271419:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306090447 00000100:00000001:3.0:1788880454.271421:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306085790 00000100:00000001:3.0:1788880454.271423:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306090447 00000100:00000001:3.0:1788880454.271425:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880454.271426:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880454.271427:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880454.271430:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880454.271433:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880454.271440:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:2.0:1788880454.271446:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880454.271451:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880454.271454:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000038c5ac58 00000400:00000010:2.0:1788880454.271456:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000038c5ac58. 00000100:00000001:2.0:1788880454.271468:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880454.271471:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880454.271480:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880454.271482:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880454.271488:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880454.271489:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880454.271549:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880454.271553:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:2.0:1788880454.271557:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880454.271561:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880454.271564:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000400:00000010:2.0:1788880454.271565:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000001:2.0:1788880454.271568:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880454.271571:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880454.271578:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880454.271581:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880454.271586:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880454.271588:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880454.273556:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880454.273567:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880454.273569:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880454.273571:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880454.273577:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880454.273587:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b06700 00000400:00000200:3.0:1788880454.273640:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x795 [1] + 224 00000800:00000001:3.0:1788880454.273647:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880454.273651:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880454.273654:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880454.273657:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880454.273670:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880454.273675:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880454.273684:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880454.273697:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880454.273704:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880454.273708:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880454.273709:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880454.273711:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880454.273716:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880454.273722:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06800 00000400:00000200:3.0:1788880454.273728:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x799 [1] + 224 00000800:00000001:3.0:1788880454.273733:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880454.273735:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880454.273736:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880454.273739:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880454.273740:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880454.273746:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:0.0:1788880454.273747:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880454.273751:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880454.273753:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8ada50c. 00000400:00000200:0.0:1788880454.273757:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000400:00000010:0.0:1788880454.273765:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00000100:00000001:0.0:1788880454.273768:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880454.273770:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880454.273774:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880454.273775:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880454.273797:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880454.273814:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880454.273817:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880454.273825:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880454.273828:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880454.273830:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880454.273833:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880454.273836:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880454.273837:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880454.273840:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880454.273845:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=132 v=5 (5 5 0 0) 00000100:00001000:0.0:1788880454.273849:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=132 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880454.273852:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880454.273853:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880454.273856:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880454.273858:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880454.273860:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880454.273864:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880454.273870:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880454.273873:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880454.273874:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880454.273877:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880454.273883:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780658432:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880454.273888:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880454.273890:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880454.273892:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000629edf55 x1875776780658432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880454.273908:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880454.273911:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880454.273912:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880454.273914:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e6a93e02. 00000100:00000001:1.0:1788880454.273917:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880454.273919:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000001:1.0:1788880454.273922:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000001:0.0:1788880454.273923:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000001:1.0:1788880454.273926:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000020:00000040:0.0:1788880454.273927:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000010:1.0:1788880454.273930:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000020:00000001:0.0:1788880454.273931:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880454.273935:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 00000400:00000200:1.0:1788880454.273936:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:1.0:1788880454.273940:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 02000000:00000001:0.0:1788880454.273942:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880454.273944:0:201287:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:0.0:1788880454.273945:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880454.273948:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000629edf55. 00000100:00000001:0.0:1788880454.273952:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880454.273955:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880454.273956:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880454.273958:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880454.273966:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:1.0:1788880454.273968:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880454.273970:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880454.273975:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880454.273978:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880454.273991:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880454.273994:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880454.273996:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880454.274007:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880454.274009:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880454.274010:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880454.274012:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880454.274017:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=132 v=5 (5 5 0 0) 00000100:00001000:1.0:1788880454.274020:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=106 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880454.274021:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880454.274022:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880454.274024:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880454.274027:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880454.274037:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880454.274040:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880454.274041:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880454.274043:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880454.274044:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880454.274046:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880454.274052:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880454.274054:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880454.274055:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880454.274057:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880454.274063:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780658688:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880454.274067:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880454.274069:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880454.274070:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000005a8bb1de x1875776780658688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880454.274075:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880454.274077:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880454.274078:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000064b136ca. 02000000:00000001:1.0:1788880454.274082:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880454.274088:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880454.274089:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880454.274091:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880454.274092:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 02000000:00000001:1.0:1788880454.274094:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880454.274095:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880454.274097:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000005a8bb1de. 00000100:00000001:1.0:1788880454.274099:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880454.274100:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880454.274102:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880454.274105:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880454.274106:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880454.274108:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880454.274109:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880454.274136:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880454.274142:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880454.274148:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880454.274151:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880454.274155:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880454.274171:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06780 00000400:00000200:3.0:1788880454.274176:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x7a5 [1] + 224 00000800:00000001:3.0:1788880454.274180:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880454.274185:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880454.274187:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880454.274190:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880454.274193:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880454.274196:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880454.274205:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880454.274220:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880454.274225:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880454.274227:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880454.274254:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880454.274257:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880454.274261:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880454.274263:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880454.274264:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000ea759cb. 00000400:00000200:2.0:1788880454.274268:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e78d7264 00000400:00000010:2.0:1788880454.274270:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e78d7264. 00000100:00000001:2.0:1788880454.274272:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880454.274276:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880454.274284:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880454.274293:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880454.274294:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880454.274296:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880454.274297:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880454.274298:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880454.274298:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880454.274300:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880454.274303:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=122 v=5 (5 5 0 0) 00000100:00001000:2.0:1788880454.274305:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=122 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880454.274306:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880454.274307:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880454.274308:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880454.274310:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880454.274311:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880454.274312:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880454.274313:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880454.274313:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880454.274315:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880454.274319:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880454.274320:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880454.274321:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880454.274322:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880454.274326:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780658560:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880454.274328:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880454.274329:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880454.274330:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cc13c26b x1875776780658560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880470 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880454.274333:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880454.274334:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880454.274338:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000659bd78d. 02000000:00000001:2.0:1788880454.274340:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880454.274341:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880454.274342:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880454.274343:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880454.274344:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 02000000:00000001:2.0:1788880454.274346:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880454.274346:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880454.274347:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cc13c26b. 00000100:00000001:2.0:1788880454.274349:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880454.274349:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880454.274350:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880454.274353:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880454.274354:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880454.274355:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880454.274356:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880454.334345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880454.334352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880454.334359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880454.334361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880454.334482:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880454.334490:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880454.334495:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880455.358374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880455.358381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880455.358387:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880455.358389:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880455.358502:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880455.358508:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880455.358512:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880456.382758:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880456.382768:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880456.382777:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880456.382780:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880456.382783:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880456.382787:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880456.382799:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880456.382803:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880456.382805:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880456.382807:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880456.382981:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880456.382986:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880456.382989:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880457.406370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880457.406377:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880457.406381:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880457.406382:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880457.406451:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880457.406455:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880457.406458:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880458.430387:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880458.430396:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880458.430426:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880458.430427:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880458.430495:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880458.430501:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880458.430506:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880459.390376:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306090447, at 4306090496 00000100:00000040:2.0:1788880459.390380:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306090447, at 4306090496 00000100:00000040:1.0:1788880459.390386:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306090447, at 4306090496 00000100:00000010:2.0:1788880459.390400:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005b8bebe2. 00000100:00000010:0.0:1788880459.390402:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000975d5882. 00000100:00000010:1.0:1788880459.390408:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009ac8d70b. 00000100:00000040:2.0:1788880459.390425:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880459.390432:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880459.390440:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cc13c26b. 00000100:00000040:3.0:1788880459.390442:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306090447, at 4306090496 00000100:00000010:3.0:1788880459.390452:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c0bfe00. 00000020:00000040:2.0:1788880459.390464:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880459.390471:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880459.390475:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880459.390489:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880459.390501:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 00000100:00000001:2.0:1788880459.390510:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880459.390524:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880459.390535:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880459.390538:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880459.390545:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cc13c26b] to pc [ptlrpcd_00_03+3] req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880459.390587:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880459.390590:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880459.390594:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880459.390598:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000910cb22c. 00000020:00000040:2.0:1788880459.390600:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880459.390603:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880459.390604:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880459.390606:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880459.390610:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c54a5c6f. 00000100:00000001:2.0:1788880459.390613:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880459.390618:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880459.390626:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880459.390627:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880459.390629:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880459.390632:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000910cb22c] to pc [ptlrpcd_00_00+0] req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880459.390636:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:1.0:1788880459.390641:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880459.390652:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880459.390655:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880459.390658:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880459.390661:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880459.390664:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880459.390666:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880459.390670:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880459.390671:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880459.390672:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880459.390674:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880459.390676:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880459.390680:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788880459.390686:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880459.390688:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880459.390692:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000010:2.0:1788880459.390693:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ea101a56. 00000100:00000001:0.0:1788880459.390696:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:2.0:1788880459.390698:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880459.390700:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880459.390700:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880459.390704:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880459.390705:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880459.390706:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880459.390706:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880459.390708:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:0.0:1788880459.390712:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880459.390712:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880459.390713:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880459.390717:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000010:2.0:1788880459.390717:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 02000000:00000001:1.0:1788880459.390724:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880459.390724:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880459.390730:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00100000:1.0:1788880459.390733:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780658816:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880459.390734:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880459.390734:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880459.390741:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880459.390742:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880459.390747:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880459.390747:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:1.0:1788880459.390749:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880459.390750:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880459.390753:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880459.390754:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880459.390757:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ea101a56] to pc [ptlrpcd_00_01+1] req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880459.390758:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880459.390764:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880459.390765:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000010c9710f. 00000100:00000001:3.0:1788880459.390768:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880459.390771:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780658944:192.168.204.111@tcp:400: 02000000:00000001:1.0:1788880459.390771:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880459.390772:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880459.390772:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880459.390776:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000040:2.0:1788880459.390777:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880459.390779:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:1.0:1788880459.390779:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000001:3.0:1788880459.390780:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880459.390784:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880459.390785:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:0.0:1788880459.390787:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880459.390788:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:1.0:1788880459.390789:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 00000100:00000040:2.0:1788880459.390791:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11428) 02000000:00000001:0.0:1788880459.390792:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880459.390792:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880459.390797:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780658816, portal 25 00000100:00000001:3.0:1788880459.390798:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880459.390802:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000115c4ce2. 00000100:00000001:3.0:1788880459.390802:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880459.390803:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880459.390807:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938433664 : -126823771117952 : ffff8ca78a2c6680) 00000100:00000001:3.0:1788880459.390807:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880459.390808:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880459.390813:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8ada50c. 00000100:00000040:3.0:1788880459.390816:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880459.390818:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880475 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880459.390820:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000100:00000200:0.0:1788880459.390825:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780658944, portal 10 00000100:00000001:0.0:1788880459.390830:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880459.390832:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938437248 : -126823771114368 : ffff8ca78a2c7480) 00000100:00000040:0.0:1788880459.390844:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880459.390844:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880459.390851:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880459.390856:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880459.390861:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 00000100:00000040:1.0:1788880459.390861:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000200:0.0:1788880459.390867:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780658944, offset 0 00000400:00000010:1.0:1788880459.390867:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000061cfcb81. 00000100:00000001:3.0:1788880459.390868:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880459.390871:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880459.390873:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780658816, offset 0 02000000:00000001:3.0:1788880459.390874:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788880459.390878:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880459.390879:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880459.390883:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880459.390889:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780659072:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880459.390895:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880459.390895:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880459.390896:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880459.390902:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880459.390903:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:3.0:1788880459.390905:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880459.390909:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880459.390910:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880459.390913:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000010:3.0:1788880459.390915:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005bd099ee. 00000400:00000200:0.0:1788880459.390916:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880459.390920:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880459.390924:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ce91c67e. 00000400:00000010:3.0:1788880459.390928:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 00000100:00000200:3.0:1788880459.390930:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780659072, portal 10 00000100:00000001:3.0:1788880459.390933:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880459.390934:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [217:217:256:4294967295] 192.168.204.111@tcp LPNI seq info [217:217:8:4294967295] 00000100:00000001:3.0:1788880459.390934:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938436352 : -126823771115264 : ffff8ca78a2c7100) 00000100:00000040:3.0:1788880459.390943:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880459.390949:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880459.390954:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880459.390956:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000003539bb5a. 00000400:00000200:0.0:1788880459.390957:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:3.0:1788880459.390959:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780659072, offset 0 00000400:00000200:3.0:1788880459.390965:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880459.390969:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880459.390970:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880459.390975:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880459.390978:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000010:0.0:1788880459.390979:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000da26fd2f. 00000400:00000200:1.0:1788880459.390980:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880459.390984:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880459.390986:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [218:218:255:4294967295] 192.168.204.111@tcp LPNI seq info [218:218:7:4294967295] 00000800:00000200:0.0:1788880459.390991:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880459.390996:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:3.0:1788880459.391005:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880459.391006:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880459.391007:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880459.391012:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1788880459.391013:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000da26fd2f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880459.391017:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000010:1.0:1788880459.391019:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000400:00000200:3.0:1788880459.391021:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880459.391028:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880459.391029:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880459.391036:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [219:219:254:4294967295] 192.168.204.111@tcp LPNI seq info [219:219:6:4294967295] 00000800:00000200:1.0:1788880459.391041:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880459.391046:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880459.391048:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880459.391053:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880459.391056:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880459.391057:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 406022 usecs (407 jiffies) 00000100:00000001:1.0:1788880459.391059:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306095567 00000100:00000001:0.0:1788880459.391062:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306090903 00000100:00000010:1.0:1788880459.391066:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000100:00000001:1.0:1788880459.391074:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:3.0:1788880459.391077:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880459.391080:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306095567 00000100:00000001:0.0:1788880459.391081:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306095567 00000800:00000010:3.0:1788880459.391083:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000100:00000001:0.0:1788880459.391086:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306090903 00000100:00000010:1.0:1788880459.391087:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000800:00000200:3.0:1788880459.391092:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880459.391093:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880459.391100:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306095567 00000100:00000001:0.0:1788880459.391101:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306095567 00000800:00000200:3.0:1788880459.391101:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880459.391106:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306090903 00000800:00000200:3.0:1788880459.391108:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 00000100:00000010:1.0:1788880459.391109:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000800:00000200:2.0:1788880459.391113:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880459.391113:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880459.391115:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000010:2.0:1788880459.391121:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000da26fd2f. 00000100:00000040:3.0:1788880459.391121:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 25387 usecs (26 jiffies) 00000100:00000001:0.0:1788880459.391122:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306095567 00000100:00000001:1.0:1788880459.391122:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306095567 00000100:00000001:0.0:1788880459.391129:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306090903 00000100:00000010:1.0:1788880459.391129:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000100:00000001:3.0:1788880459.391129:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306090522 00000400:00000200:2.0:1788880459.391130:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880459.391136:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880459.391136:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306095567 00000400:00000200:2.0:1788880459.391142:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880459.391143:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880459.391143:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306090522 00000100:00000001:0.0:1788880459.391146:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306095567 00000100:00000001:3.0:1788880459.391147:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306095567 00000400:00000200:2.0:1788880459.391148:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000100:00000001:0.0:1788880459.391151:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880459.391152:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000001:1.0:1788880459.391155:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880459.391155:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306090522 00000100:00000001:0.0:1788880459.391158:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880459.391159:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880459.391161:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306095567 00000100:00000001:1.0:1788880459.391162:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880459.391165:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880459.391165:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880459.391168:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306090522 00000100:00000001:1.0:1788880459.391169:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880459.391172:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880459.391175:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306095567 00000100:00000001:0.0:1788880459.391178:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880459.391180:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880459.391181:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880459.391184:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880459.391186:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880459.391188:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880459.391189:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880459.391191:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880459.391196:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880459.391196:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880459.391218:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880459.391220:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880459.391258:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880459.391260:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:2.0:1788880459.391264:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880459.391267:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880459.391270:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000061cfcb81 00000400:00000010:2.0:1788880459.391272:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000061cfcb81. 00000100:00000001:2.0:1788880459.391274:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880459.391278:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880475 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880459.391285:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880459.391288:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880475 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880459.391292:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880459.391293:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880459.391326:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880459.391328:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000400:00000200:2.0:1788880459.391331:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880459.391333:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880459.391335:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000003539bb5a 00000400:00000010:2.0:1788880459.391336:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000003539bb5a. 00000100:00000001:2.0:1788880459.391337:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880459.391339:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880459.391343:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880459.391345:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880459.391348:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880459.391349:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880459.393187:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880459.393205:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880459.393219:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880459.393223:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880459.393258:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880459.393275:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b06880 00000400:00000200:2.0:1788880459.393284:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x7ad [1] + 224 00000800:00000001:2.0:1788880459.393292:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880459.393296:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880459.393299:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880459.393303:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880459.393308:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880459.393314:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880459.393334:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880459.393350:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880459.393357:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880459.393360:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880459.393361:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880459.393363:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880459.393367:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880459.393373:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06900 00000400:00000200:2.0:1788880459.393378:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x7b1 [1] + 224 00000800:00000001:2.0:1788880459.393382:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880459.393385:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880459.393387:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880459.393390:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880459.393391:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880459.393395:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880459.393396:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:2.0:1788880459.393400:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880459.393404:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880459.393407:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880459.393409:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:1.0:1788880459.393414:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000100:00000040:2.0:1788880459.393414:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880459.393417:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:1.0:1788880459.393422:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880459.393427:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880459.393450:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880459.393460:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880459.393461:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880459.393462:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880459.393467:0:201287:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788880459.393468:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:1.0:1788880459.393470:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788880459.393473:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880459.393475:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880459.393476:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880459.393479:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880459.393479:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880459.393486:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880459.393490:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880459.393493:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:1.0:1788880459.393496:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=137 v=5 (5 5 0 0) 00000400:00000200:2.0:1788880459.393497:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06980 00000100:00000001:0.0:1788880459.393498:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880459.393503:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00001000:1.0:1788880459.393505:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=137 v=5 (1 1 0 0) 00000400:00000001:0.0:1788880459.393506:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880459.393508:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8ada50c. 00000100:00000001:1.0:1788880459.393511:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:0.0:1788880459.393513:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000100:00000001:1.0:1788880459.393514:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880459.393516:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00010000:00000001:1.0:1788880459.393519:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880459.393520:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880459.393524:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x7bd [1] + 224 00010000:00000001:1.0:1788880459.393525:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880459.393527:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880459.393530:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880459.393532:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880459.393539:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880459.393543:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880459.393543:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880459.393543:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880459.393552:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880459.393558:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880459.393575:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880459.393578:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880459.393581:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880459.393588:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880459.393589:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880459.393592:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880459.393593:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:2.0:1788880459.393595:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880459.393597:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880459.393599:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880459.393603:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880459.393603:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880459.393607:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880459.393612:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:0.0:1788880459.393615:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=127 v=5 (5 5 0 0) 00000800:00000001:2.0:1788880459.393617:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880459.393621:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880459.393624:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=127 v=5 (1 1 0 0) 00000100:00100000:1.0:1788880459.393628:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780658816:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880459.393630:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880459.393634:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880459.393638:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880459.393638:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880459.393642:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880459.393643:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880459.393646:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880459.393649:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880459.393650:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cc13c26b x1875776780658816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880459.393651:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880459.393654:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:3.0:1788880459.393657:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880459.393659:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000001:3.0:1788880459.393661:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880459.393664:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880459.393665:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ce91c67e. 00000100:00000001:1.0:1788880459.393670:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000200:3.0:1788880459.393671:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 00000100:00000040:0.0:1788880459.393674:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880459.393675:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 02000000:00000001:1.0:1788880459.393676:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880459.393681:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000010c9710f. 00000100:00000001:3.0:1788880459.393681:0:201285:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:1.0:1788880459.393688:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:3.0:1788880459.393689:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880459.393691:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000020:00000001:1.0:1788880459.393692:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880459.393697:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880459.393698:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880459.393702:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880459.393704:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880459.393710:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 00000100:00000040:0.0:1788880459.393711:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880459.393714:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880459.393717:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880459.393720:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880459.393722:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cc13c26b. 00000100:00000001:1.0:1788880459.393727:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880459.393730:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:0.0:1788880459.393731:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780658944:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880459.393734:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880459.393744:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880459.393744:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880459.393746:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880459.393747:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880459.393750:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880459.393750:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880459.393751:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880459.393754:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880459.393755:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000910cb22c x1875776780658944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880459.393756:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880459.393760:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880459.393762:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880459.393766:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880459.393768:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880459.393770:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880459.393771:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880459.393774:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000115c4ce2. 00000100:00001000:3.0:1788880459.393778:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=137 v=5 (5 5 0 0) 02000000:00000001:0.0:1788880459.393779:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880459.393782:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00001000:3.0:1788880459.393784:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=111 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880459.393786:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880459.393788:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880459.393790:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00010000:00000001:3.0:1788880459.393792:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880459.393798:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880459.393803:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:3.0:1788880459.393808:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880459.393810:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c54a5c6f. 00000100:00000001:3.0:1788880459.393811:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880459.393813:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:0.0:1788880459.393816:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880459.393817:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000001:0.0:1788880459.393820:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880459.393820:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880459.393824:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000910cb22c. 00000100:00000040:3.0:1788880459.393827:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880459.393831:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880459.393834:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880459.393838:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880459.393840:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880459.393844:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880459.393845:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880459.393846:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880459.393850:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880459.393850:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880459.393854:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880459.393858:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880459.393864:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780659072:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880459.393871:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880459.393873:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880459.393880:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ea101a56 x1875776780659072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880475 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880459.393897:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880459.393899:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880459.393900:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005bd099ee. 02000000:00000001:3.0:1788880459.393903:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880459.393904:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880459.393906:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880459.393908:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880459.393909:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 02000000:00000001:3.0:1788880459.393911:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880459.393913:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880459.393914:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ea101a56. 00000100:00000001:3.0:1788880459.393916:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880459.393917:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880459.393919:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880459.393923:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880459.393924:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880459.393926:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880459.393927:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880459.454366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880459.454375:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880459.454384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880459.454385:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880459.454446:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880459.454450:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880459.454452:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880460.478386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880460.478395:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880460.478403:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880460.478404:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880460.478465:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880460.478470:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880460.478473:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880461.502352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880461.502359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880461.502365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880461.502382:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880461.502434:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880461.502437:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880461.502439:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880462.526334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880462.526340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880462.526345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880462.526347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880462.526399:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880462.526403:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880462.526437:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880463.550343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880463.550351:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880463.550353:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880463.550356:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880463.550361:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880463.550363:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880463.550364:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880463.550366:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880463.550370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880463.550372:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880463.550439:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880463.550444:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880463.550446:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880464.510361:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306095567, at 4306095616 00000100:00000040:1.0:1788880464.510361:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306095567, at 4306095616 00000100:00000040:3.0:1788880464.510361:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306095567, at 4306095616 00000100:00000040:2.0:1788880464.510379:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:0.0:1788880464.510381:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:3.0:1788880464.510382:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:1.0:1788880464.510385:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000001:2.0:1788880464.510397:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880464.510407:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000071512652. 00000020:00000040:2.0:1788880464.510427:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880464.510433:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880464.510436:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880464.510440:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880464.510448:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000100:00000001:2.0:1788880464.510455:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880464.510475:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880464.510486:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880464.510488:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880464.510495:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000071512652] to pc [ptlrpcd_00_02+2] req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880464.510520:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880464.510523:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880464.510526:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880464.510531:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ea101a56. 00000020:00000040:2.0:1788880464.510534:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880464.510537:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880464.510539:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880464.510542:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880464.510546:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 00000100:00000001:1.0:1788880464.510548:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880464.510551:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880464.510552:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880464.510554:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880464.510562:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880464.510563:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880464.510567:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:1.0:1788880464.510573:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880464.510632:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880464.510633:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880464.510636:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880464.510643:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880464.510646:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880464.510647:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880464.510654:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880464.510654:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ea101a56] to pc [ptlrpcd_00_03+3] req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880464.510668:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000071512652 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780659200:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880464.510677:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880464.510680:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880464.510683:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880464.510687:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1788880464.510688:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880464.510698:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880464.510698:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000910cb22c. 00000100:00000001:3.0:1788880464.510702:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880464.510711:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880464.510711:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880464.510713:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880464.510718:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880464.510722:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880464.510724:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880464.510726:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880464.510727:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000089aec2fa. 00000100:00000040:3.0:1788880464.510731:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880464.510734:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880464.510734:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006aca3ff7. 00000100:00000001:2.0:1788880464.510747:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880464.510753:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000040:2.0:1788880464.510758:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880464.510763:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880464.510770:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880464.510780:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880464.510782:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880464.510784:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 00000100:00000040:2.0:1788880464.510787:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000910cb22c] to pc [ptlrpcd_00_00+0] req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880464.510792:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:1.0:1788880464.510797:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780659200, portal 25 02000000:00000001:3.0:1788880464.510798:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880464.510809:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:3.0:1788880464.510810:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880464.510813:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430080 : -126823771121536 : ffff8ca78a2c5880) 00000100:00100000:3.0:1788880464.510817:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780659328:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880464.510822:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880464.510824:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880480 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880464.510830:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880464.510837:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880464.510841:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880464.510842:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:3.0:1788880464.510845:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880464.510852:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880464.510856:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11433) 00000100:00000001:1.0:1788880464.510857:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:3.0:1788880464.510859:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000659bd78d. 00000100:00000001:0.0:1788880464.510865:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880464.510865:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880464.510866:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880464.510868:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:1.0:1788880464.510871:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000100:00000001:0.0:1788880464.510872:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:1.0:1788880464.510875:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780659200, offset 0 00000400:00000010:3.0:1788880464.510880:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ce91c67e. 00000100:00000040:0.0:1788880464.510885:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880464.510885:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880464.510894:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880464.510896:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880464.510896:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306095567, at 4306095616 00000400:00000010:3.0:1788880464.510897:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 02000000:00000001:0.0:1788880464.510901:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:3.0:1788880464.510904:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780659328, portal 10 02000000:00000001:0.0:1788880464.510905:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880464.510909:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880464.510911:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:0.0:1788880464.510913:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880464.510914:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938436352 : -126823771115264 : ffff8ca78a2c7100) 00000100:00000010:2.0:1788880464.510918:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00100000:0.0:1788880464.510920:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780659456:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880464.510922:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880464.510928:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880464.510930:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788880464.510934:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880464.510935:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880464.510938:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880464.510938:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880464.510942:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880464.510944:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880464.510945:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:0.0:1788880464.510947:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000015664212. 00000400:00000010:3.0:1788880464.510948:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 02000000:00000001:0.0:1788880464.510953:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880464.510953:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:3.0:1788880464.510953:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780659328, offset 0 00000100:00000001:2.0:1788880464.510956:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788880464.510962:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8ada50c. 00000100:00000001:2.0:1788880464.510966:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880464.510968:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [220:220:256:4294967295] 192.168.204.111@tcp LPNI seq info [220:220:8:4294967295] 00000400:00000200:3.0:1788880464.510969:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880464.510980:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000100:00000200:0.0:1788880464.510985:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780659456, portal 10 00000400:00000200:1.0:1788880464.510990:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880464.510995:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880464.510997:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938437248 : -126823771114368 : ffff8ca78a2c7480) 00000800:00000200:1.0:1788880464.511001:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880464.511005:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880464.511005:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880464.511009:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000400:00000200:3.0:1788880464.511013:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880464.511020:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788880464.511020:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880464.511021:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:0.0:1788880464.511027:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880464.511028:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:0.0:1788880464.511030:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d6cef011. 00000100:00000200:0.0:1788880464.511034:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780659456, offset 0 00000800:00000200:1.0:1788880464.511036:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880464.511040:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880464.511044:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880464.511048:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880464.511055:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [221:221:255:4294967295] 192.168.204.111@tcp LPNI seq info [221:221:7:4294967295] 00000100:00000001:1.0:1788880464.511061:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880464.511068:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880464.511070:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306100687 00000800:00000200:3.0:1788880464.511075:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880464.511078:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000400:00000200:0.0:1788880464.511081:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:3.0:1788880464.511084:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000400:00000200:0.0:1788880464.511087:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880464.511088:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880464.511095:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880464.511096:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306100687 00000400:00000200:0.0:1788880464.511101:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:1.0:1788880464.511103:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000800:00000200:3.0:1788880464.511106:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880464.511108:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880464.511110:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880464.511117:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [222:222:254:4294967295] 192.168.204.111@tcp LPNI seq info [222:222:6:4294967295] 00000800:00000200:3.0:1788880464.511118:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880464.511123:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306100687 00000800:00000200:3.0:1788880464.511124:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 00000100:00000010:1.0:1788880464.511130:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000100:00000001:3.0:1788880464.511133:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880464.511137:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:0.0:1788880464.511139:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788880464.511140:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 399735 usecs (400 jiffies) 00000100:00000001:1.0:1788880464.511145:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306100687 00000100:00000001:3.0:1788880464.511147:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306096016 00000100:00000010:1.0:1788880464.511151:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000800:00000200:0.0:1788880464.511152:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880464.511158:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880464.511159:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306100687 00000800:00000010:0.0:1788880464.511160:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000004e0f6846. 00000100:00000001:1.0:1788880464.511165:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880464.511166:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306096016 00000800:00000200:0.0:1788880464.511172:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880464.511172:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306100687 00000100:00000001:1.0:1788880464.511177:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880464.511177:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306096016 00000800:00000200:0.0:1788880464.511183:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880464.511183:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880464.511184:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880464.511184:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306100687 00000800:00000010:2.0:1788880464.511194:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000100:00000001:1.0:1788880464.511195:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880464.511196:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000004e0f6846 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880464.511196:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306096016 00000400:00000200:2.0:1788880464.511205:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880464.511206:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306100687 00000100:00000001:0.0:1788880464.511207:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880464.511213:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880464.511215:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 25026 usecs (26 jiffies) 00000100:00000001:3.0:1788880464.511215:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880464.511219:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880464.511225:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880464.511226:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880464.511228:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306095642 00000100:00000001:1.0:1788880464.511230:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880464.511261:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306100687 00000100:00000001:0.0:1788880464.511264:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306095643 00000100:00000001:0.0:1788880464.511265:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306100687 00000100:00000001:0.0:1788880464.511269:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306095643 00000100:00000001:1.0:1788880464.511270:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880464.511272:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306100687 00000100:00000001:1.0:1788880464.511274:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880464.511276:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880464.511281:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880464.511282:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880464.511283:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306095643 00000400:00000200:2.0:1788880464.511285:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000100:00000001:3.0:1788880464.511288:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:2.0:1788880464.511290:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:0.0:1788880464.511293:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306100687 00000100:00000001:3.0:1788880464.511293:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880464.511301:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880464.511302:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880464.511303:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880464.511306:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880464.511310:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880464.511312:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880464.511313:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880480 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880464.511314:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880464.511322:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880464.511328:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880464.511335:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880464.511341:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880464.511344:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880464.511348:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880464.511352:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880480 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880464.511353:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880464.511359:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880464.511371:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880464.511372:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880464.511376:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880464.511390:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880464.511393:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000400:00000200:2.0:1788880464.511396:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880464.511401:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880464.511404:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000400:00000010:2.0:1788880464.511405:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:2.0:1788880464.511406:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880464.511408:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880464.511413:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880464.511414:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880464.511417:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880464.511418:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880464.511461:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880464.511477:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000004e0f6846. 00000400:00000200:2.0:1788880464.511481:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880464.511484:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880464.511485:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d6cef011 00000400:00000010:2.0:1788880464.511486:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d6cef011. 00000100:00000001:2.0:1788880464.511488:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880464.511490:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880464.511500:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880464.511501:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880464.511505:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880464.511506:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880464.513043:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880464.513060:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880464.513064:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880464.513068:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880464.513078:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880464.513092:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b06a00 00000400:00000200:2.0:1788880464.513100:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x7c5 [1] + 224 00000800:00000001:2.0:1788880464.513107:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880464.513110:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880464.513113:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880464.513117:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880464.513124:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880464.513129:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880464.513140:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880464.513156:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880464.513164:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880464.513167:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880464.513169:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880464.513171:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880464.513175:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880464.513181:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06a80 00000100:00000001:1.0:1788880464.513184:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880464.513187:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x7cd [1] + 224 00000100:00000001:1.0:1788880464.513188:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880464.513193:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880464.513196:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880464.513198:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880464.513200:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:1.0:1788880464.513205:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:1.0:1788880464.513207:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000800:00000001:2.0:1788880464.513209:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880464.513211:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880464.513214:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:1.0:1788880464.513217:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880464.513223:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880464.513230:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880464.513252:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880464.513260:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880464.513262:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880464.513265:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880464.513267:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880464.513271:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880464.513274:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:2.0:1788880464.513278:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880464.513279:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880464.513285:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880464.513296:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:1.0:1788880464.513301:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=142 v=5 (5 5 0 0) 00000800:00000001:2.0:1788880464.513302:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880464.513306:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880464.513307:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=142 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880464.513312:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880464.513314:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880464.513318:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880464.513321:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880464.513324:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880464.513327:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880464.513329:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880464.513330:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880464.513335:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880464.513337:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000001:3.0:1788880464.513337:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880464.513341:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ce91c67e. 00000100:00000001:1.0:1788880464.513342:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880464.513345:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880464.513345:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 00000400:00000010:3.0:1788880464.513347:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 00000100:00000040:1.0:1788880464.513349:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880464.513351:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880464.513356:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880464.513359:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000071512652 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780659200:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880464.513367:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880464.513370:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:3.0:1788880464.513370:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880464.513374:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000071512652 x1875776780659200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880464.513382:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880464.513384:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880464.513385:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880464.513393:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880464.513394:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880464.513395:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880464.513398:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880464.513400:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:3.0:1788880464.513400:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880464.513404:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000089aec2fa. 00000100:00001000:3.0:1788880464.513407:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=132 v=5 (5 5 0 0) 02000000:00000001:1.0:1788880464.513410:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:3.0:1788880464.513411:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=132 v=5 (1 1 0 0) 00000020:00000001:1.0:1788880464.513415:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880464.513415:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880464.513416:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880464.513419:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00010000:00000001:3.0:1788880464.513420:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:1.0:1788880464.513424:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:3.0:1788880464.513425:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880464.513427:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 00000100:00000001:3.0:1788880464.513430:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:1.0:1788880464.513431:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880464.513435:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:3.0:1788880464.513435:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000010:1.0:1788880464.513437:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000071512652. 00000100:00000001:3.0:1788880464.513440:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880464.513442:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880464.513444:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880464.513444:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880464.513447:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880464.513451:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880464.513459:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880464.513461:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880464.513463:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880464.513464:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880464.513557:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880464.513563:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880464.513564:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880464.513565:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880464.513569:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880464.513574:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06b00 00000400:00000200:2.0:1788880464.513578:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x7d5 [1] + 224 00000800:00000001:2.0:1788880464.513581:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880464.513584:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880464.513585:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880464.513587:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880464.513590:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880464.513592:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880464.513599:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880464.513611:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880464.513614:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880464.513616:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880464.513640:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880464.513643:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880464.513645:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880464.513645:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880464.513649:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880464.513650:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880464.513653:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880464.513657:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880464.513659:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8ada50c. 00000100:00100000:3.0:1788880464.513663:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780659328:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880464.513665:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000400:00000010:0.0:1788880464.513669:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00000100:00000001:3.0:1788880464.513671:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880464.513674:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880464.513675:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880464.513679:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ea101a56 x1875776780659328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880464.513687:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:0.0:1788880464.513690:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880464.513691:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880464.513694:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000659bd78d. 02000000:00000001:3.0:1788880464.513699:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880464.513701:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880464.513703:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000200:0.0:1788880464.513708:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880464.513715:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880464.513718:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880464.513719:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880464.513720:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880464.513722:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880464.513723:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880464.513724:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880464.513725:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880464.513729:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 00000100:00001000:0.0:1788880464.513731:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=142 v=5 (5 5 0 0) 02000000:00000001:3.0:1788880464.513734:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00001000:0.0:1788880464.513736:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=116 v=5 (1 1 0 0) 02000000:00000001:3.0:1788880464.513737:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880464.513739:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ea101a56. 00000100:00000001:0.0:1788880464.513741:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880464.513744:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880464.513745:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880464.513747:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880464.513748:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880464.513751:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880464.513754:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880464.513755:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880464.513756:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880464.513759:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:0.0:1788880464.513762:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880464.513770:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880464.513773:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880464.513775:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880464.513777:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880464.513778:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880464.513781:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880464.513787:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880464.513789:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880464.513790:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880464.513793:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880464.513800:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780659456:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880464.513805:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880464.513807:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880464.513809:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000910cb22c x1875776780659456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880480 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880464.513815:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880464.513817:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880464.513819:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000015664212. 02000000:00000001:0.0:1788880464.513822:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880464.513823:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880464.513824:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880464.513827:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880464.513828:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006aca3ff7. 02000000:00000001:0.0:1788880464.513830:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880464.513832:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880464.513833:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000910cb22c. 00000100:00000001:0.0:1788880464.513841:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880464.513842:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880464.513844:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880464.513848:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880464.513849:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880464.513851:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880464.513853:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880464.574373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880464.574382:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880464.574390:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880464.574392:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880464.574448:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880464.574487:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880464.574491:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880465.598355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880465.598362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880465.598367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880465.598368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880465.598426:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880465.598429:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880465.598432:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880466.622392:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880466.622402:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880466.622407:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880466.622411:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880466.622413:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880466.622415:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880466.622418:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880466.622420:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880466.622421:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880466.622422:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880466.622496:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880466.622501:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880466.622504:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880467.646363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880467.646369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880467.646376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880467.646378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880467.646448:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880467.646457:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880467.646461:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880468.670370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880468.670377:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880468.670383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880468.670383:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880468.670467:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880468.670472:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880468.670476:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880469.630339:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306100687, at 4306100736 00000100:00000040:1.0:1788880469.630339:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306100687, at 4306100736 00000100:00000040:2.0:1788880469.630340:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880469.630340:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306100687, at 4306100736 00000100:00000001:2.0:1788880469.630352:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880469.630356:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000010:2.0:1788880469.630361:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cc13c26b. 00000100:00000010:0.0:1788880469.630364:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000010:3.0:1788880469.630376:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000020:00000040:2.0:1788880469.630379:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880469.630386:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880469.630389:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880469.630394:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880469.630402:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f47bc5fc. 00000100:00000001:2.0:1788880469.630409:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880469.630418:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880469.630429:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880469.630431:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880469.630437:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cc13c26b] to pc [ptlrpcd_00_01+1] req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880469.630461:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880469.630464:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880469.630467:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880469.630470:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000005a8bb1de. 00000020:00000040:2.0:1788880469.630484:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880469.630485:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880469.630486:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880469.630487:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880469.630490:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880469.630490:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880469.630490:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880469.630492:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880469.630495:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:2.0:1788880469.630498:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003139f6ba. 00000100:00000040:0.0:1788880469.630503:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880469.630504:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880469.630513:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880469.630520:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880469.630524:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880469.630526:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880469.630530:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880469.630535:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880469.630537:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880469.630540:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880469.630541:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780659584:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880469.630547:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a8bb1de] to pc [ptlrpcd_00_02+2] req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880469.630548:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880469.630555:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880469.630558:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880469.630561:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880469.630566:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000235147aa. 00000100:00000001:2.0:1788880469.630568:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880469.630569:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880469.630573:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:0.0:1788880469.630577:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8ada50c. 00000100:00000001:2.0:1788880469.630578:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880469.630582:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000629edf55. 00000400:00000010:0.0:1788880469.630586:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000020:00000040:2.0:1788880469.630586:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880469.630589:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880469.630591:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880469.630591:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000200:0.0:1788880469.630594:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780659584, portal 25 02000000:00000001:2.0:1788880469.630595:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880469.630596:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880469.630597:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880469.630600:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880469.630601:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880469.630602:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880469.630603:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008eccda10. 00000100:00000001:0.0:1788880469.630604:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938433664 : -126823771117952 : ffff8ca78a2c6680) 00000100:00000040:1.0:1788880469.630608:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880469.630609:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880469.630614:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880485 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880469.630619:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880469.630639:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880469.630641:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880469.630643:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880469.630649:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000629edf55] to pc [ptlrpcd_00_03+3] req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880469.630654:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880469.630659:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a09430ba. 00000100:00000200:0.0:1788880469.630665:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780659584, offset 0 00000100:00000001:1.0:1788880469.630670:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880469.630674:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880469.630678:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880469.630679:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880469.630681:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880469.630683:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880469.630684:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880469.630687:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880469.630692:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780659712:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880469.630695:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880469.630698:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880469.630700:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880469.630701:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880469.630702:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880469.630705:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880469.630706:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880469.630706:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880469.630708:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11438) 02000000:00000001:1.0:1788880469.630710:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880469.630713:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880469.630715:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788880469.630715:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880469.630721:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000826ae8f4. 00000400:00000200:0.0:1788880469.630724:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880469.630724:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:1.0:1788880469.630725:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880469.630725:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306100687, at 4306100736 00000100:00000001:3.0:1788880469.630728:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880469.630732:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:1.0:1788880469.630732:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000010:2.0:1788880469.630733:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 02000000:00000001:3.0:1788880469.630733:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880469.630738:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880469.630740:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [223:223:256:4294967295] 192.168.204.111@tcp LPNI seq info [223:223:8:4294967295] 00000400:00000010:1.0:1788880469.630741:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 02000000:00000001:3.0:1788880469.630743:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880469.630747:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780659712, portal 10 00000100:00000001:1.0:1788880469.630752:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00100000:3.0:1788880469.630752:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780659840:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880469.630755:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938427392 : -126823771124224 : ffff8ca78a2c4e00) 00000100:00000001:3.0:1788880469.630762:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880469.630763:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:1.0:1788880469.630767:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880469.630768:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880469.630771:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880469.630776:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880469.630776:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880469.630782:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002b77e71a. 00000100:00000001:1.0:1788880469.630783:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000010:0.0:1788880469.630785:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c93f60a2. 02000000:00000001:3.0:1788880469.630787:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880469.630789:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880469.630794:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000062e8e6f4. 00000400:00000010:3.0:1788880469.630794:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ce91c67e. 00000800:00000200:0.0:1788880469.630796:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:1.0:1788880469.630800:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780659712, offset 0 00000400:00000010:3.0:1788880469.630803:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 00000800:00000200:0.0:1788880469.630810:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880469.630810:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880469.630816:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c93f60a2 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880469.630819:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880469.630824:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:3.0:1788880469.630825:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780659840, portal 10 00000400:00000200:1.0:1788880469.630828:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880469.630830:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880469.630832:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880469.630833:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880469.630834:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938432768 : -126823771118848 : ffff8ca78a2c6300) 00000100:00000001:0.0:1788880469.630839:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306105807 00000400:00000200:1.0:1788880469.630839:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:0.0:1788880469.630846:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006050354c. 00000100:00000040:3.0:1788880469.630847:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880469.630848:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [224:224:255:4294967295] 192.168.204.111@tcp LPNI seq info [224:224:7:4294967295] 00000100:00000001:0.0:1788880469.630854:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880469.630862:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306105807 00000100:00000001:3.0:1788880469.630865:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:0.0:1788880469.630867:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008921ad2e. 00000400:00000200:1.0:1788880469.630872:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880469.630873:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:3.0:1788880469.630873:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880469.630878:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306105807 00000400:00000010:3.0:1788880469.630878:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000100:00000200:3.0:1788880469.630884:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780659840, offset 0 00000100:00000010:0.0:1788880469.630886:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001a3d5793. 00000800:00000200:1.0:1788880469.630891:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880469.630893:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000010:1.0:1788880469.630896:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000400:00000200:3.0:1788880469.630898:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880469.630900:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306105807 00000800:00000200:1.0:1788880469.630904:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:0.0:1788880469.630907:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000975d5882. 00000400:00000200:3.0:1788880469.630908:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880469.630913:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:1.0:1788880469.630914:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880469.630918:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880469.630919:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880469.630920:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880469.630924:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880469.630925:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880469.630928:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880469.630931:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 393638 usecs (394 jiffies) 00000400:00000200:3.0:1788880469.630931:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880469.630933:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880469.630937:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306101130 00000100:00000001:0.0:1788880469.630938:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880469.630938:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880469.630944:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306105807 00000800:00000200:2.0:1788880469.630944:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:3.0:1788880469.630948:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [225:225:254:4294967295] 192.168.204.111@tcp LPNI seq info [225:225:6:4294967295] 00000100:00000001:0.0:1788880469.630949:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880469.630950:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306101130 00000800:00000010:2.0:1788880469.630950:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c93f60a2. 00000100:00000001:0.0:1788880469.630953:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880469.630956:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306105807 00000400:00000200:2.0:1788880469.630958:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880469.630959:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880469.630961:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306101130 00000100:00000001:0.0:1788880469.630963:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880469.630967:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880469.630967:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306105807 00000400:00000200:2.0:1788880469.630971:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880469.630973:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880469.630973:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880469.630974:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306101130 00000100:00000001:1.0:1788880469.630979:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306105807 00000100:00000001:1.0:1788880469.630983:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880469.630984:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a09430ba 00000800:00000200:3.0:1788880469.630986:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880469.630988:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880469.630989:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a09430ba. 00000100:00000001:1.0:1788880469.630993:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880469.630996:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880469.630997:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:3.0:1788880469.630998:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000100:00000001:1.0:1788880469.631003:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:2.0:1788880469.631003:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880485 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880469.631007:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:3.0:1788880469.631007:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880469.631013:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880469.631016:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:3.0:1788880469.631016:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880469.631019:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880469.631022:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880469.631023:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880469.631028:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880485 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880469.631036:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880469.631038:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880469.631052:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880469.631054:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:2.0:1788880469.631056:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880469.631059:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880469.631061:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000062e8e6f4 00000400:00000010:2.0:1788880469.631063:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000062e8e6f4. 00000100:00000001:2.0:1788880469.631065:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880469.631068:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880469.631074:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880469.631076:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880469.631083:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880469.631089:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880469.631120:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880469.631131:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880469.631134:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 24676 usecs (25 jiffies) 00000100:00000001:3.0:1788880469.631137:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306100761 00000100:00000001:3.0:1788880469.631139:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306105807 00000100:00000001:3.0:1788880469.631141:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306100761 00000100:00000001:3.0:1788880469.631142:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306105807 00000100:00000001:3.0:1788880469.631144:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306100761 00000100:00000001:3.0:1788880469.631151:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306105807 00000100:00000001:3.0:1788880469.631154:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306100761 00000100:00000001:3.0:1788880469.631156:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306105807 00000800:00000200:2.0:1788880469.631172:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880469.631174:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:2.0:1788880469.631178:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880469.631181:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880469.631184:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00000400:00000010:2.0:1788880469.631185:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000100:00000001:2.0:1788880469.631187:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880469.631190:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880469.631198:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880469.631201:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880469.631207:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880469.631209:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880469.631229:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880469.631231:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880469.631246:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880469.631248:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880469.631251:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880469.631251:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880469.631253:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880469.631254:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880469.631254:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880469.631256:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880469.632530:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880469.632541:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880469.632545:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880469.632548:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880469.632555:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880469.632566:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b06b80 00000400:00000200:2.0:1788880469.632573:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x7dd [1] + 224 00000800:00000001:2.0:1788880469.632579:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880469.632585:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880469.632588:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880469.632590:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880469.632596:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880469.632601:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880469.632610:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880469.632626:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880469.632640:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880469.632643:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880469.632932:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880469.632935:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880469.632940:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880469.632942:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880469.632944:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8ada50c. 00000400:00000200:0.0:1788880469.632955:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000400:00000010:0.0:1788880469.632957:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00000100:00000001:0.0:1788880469.632963:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880469.632967:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880469.632974:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880469.632979:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880469.632981:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880469.632983:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880469.632985:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880469.632988:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880469.632990:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880469.632993:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880469.633000:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=147 v=5 (5 5 0 0) 00000100:00001000:0.0:1788880469.633003:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=147 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880469.633005:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880469.633006:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880469.633009:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880469.633012:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880469.633014:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880469.633017:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880469.633035:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880469.633038:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880469.633039:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880469.633043:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880469.633049:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780659584:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880469.633054:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880469.633056:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880469.633058:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cc13c26b x1875776780659584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880469.633064:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880469.633066:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880469.633068:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000235147aa. 02000000:00000001:0.0:1788880469.633072:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880469.633074:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880469.633075:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880469.633077:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880469.633079:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f47bc5fc. 02000000:00000001:0.0:1788880469.633082:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880469.633084:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880469.633085:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cc13c26b. 00000100:00000001:0.0:1788880469.633088:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880469.633089:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880469.633091:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880469.633096:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880469.633097:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880469.633099:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880469.633101:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880469.633168:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880469.633174:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880469.633176:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880469.633178:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880469.633183:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880469.633190:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06c00 00000400:00000200:2.0:1788880469.633194:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x7e5 [1] + 224 00000800:00000001:2.0:1788880469.633197:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880469.633200:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880469.633201:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880469.633203:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880469.633205:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880469.633209:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880469.633217:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880469.633252:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880469.633258:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880469.633265:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880469.633268:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880469.633270:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880469.633271:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880469.633276:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880469.633276:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:1.0:1788880469.633281:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880469.633284:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788880469.633285:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06c80 00000400:00000010:1.0:1788880469.633287:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:1.0:1788880469.633293:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:1.0:1788880469.633301:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:1.0:1788880469.633304:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880469.633308:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x7ed [1] + 224 00000100:00000200:1.0:1788880469.633310:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880469.633316:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788880469.633320:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880469.633324:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880469.633325:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880469.633327:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880469.633328:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880469.633329:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880469.633332:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880469.633333:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880469.633336:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880469.633337:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:1.0:1788880469.633338:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880469.633343:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:1.0:1788880469.633344:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=137 v=5 (5 5 0 0) 00000100:00001000:1.0:1788880469.633348:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=137 v=5 (1 1 0 0) 00000100:00000200:2.0:1788880469.633350:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880469.633351:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880469.633352:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880469.633353:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880469.633356:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880469.633357:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880469.633359:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880469.633362:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880469.633364:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880469.633384:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880469.633388:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880469.633394:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880469.633399:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880469.633401:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880469.633406:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880469.633411:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780659712:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880469.633413:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880469.633414:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880469.633421:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:2.0:1788880469.633421:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788880469.633425:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000005a8bb1de x1875776780659712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880469.633427:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880469.633434:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880469.633435:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880469.633436:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880469.633439:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000826ae8f4. 00000100:00000001:3.0:1788880469.633439:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880469.633443:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000001:3.0:1788880469.633444:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000001:1.0:1788880469.633445:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880469.633447:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000001:3.0:1788880469.633447:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880469.633450:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ce91c67e. 00000020:00000001:1.0:1788880469.633451:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880469.633453:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003139f6ba. 00000400:00000200:3.0:1788880469.633455:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 02000000:00000001:1.0:1788880469.633456:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000010:3.0:1788880469.633457:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 02000000:00000001:1.0:1788880469.633459:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880469.633461:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000005a8bb1de. 00000100:00000001:3.0:1788880469.633461:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880469.633465:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000200:3.0:1788880469.633467:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880469.633468:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880469.633472:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880469.633477:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880469.633480:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880469.633480:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880469.633484:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880469.633486:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880469.633492:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880469.633493:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880469.633494:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880469.633496:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880469.633497:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880469.633497:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880469.633499:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880469.633502:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=147 v=5 (5 5 0 0) 00000100:00001000:3.0:1788880469.633504:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=121 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880469.633511:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880469.633512:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880469.633513:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880469.633515:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880469.633520:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880469.633521:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880469.633522:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880469.633523:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880469.633524:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880469.633525:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880469.633529:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880469.633530:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880469.633531:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880469.633532:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880469.633536:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780659840:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880469.633538:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880469.633539:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880469.633540:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000629edf55 x1875776780659840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880485 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880469.633543:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880469.633544:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880469.633546:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002b77e71a. 02000000:00000001:3.0:1788880469.633548:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880469.633548:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880469.633549:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880469.633550:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880469.633551:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008eccda10. 02000000:00000001:3.0:1788880469.633553:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880469.633556:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880469.633557:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000629edf55. 00000100:00000001:3.0:1788880469.633559:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880469.633560:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880469.633561:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880469.633564:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880469.633565:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880469.633566:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880469.633567:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880469.694383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880469.694389:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880469.694395:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880469.694396:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880469.694457:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880469.694461:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880469.694463:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880470.718310:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880470.718316:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880470.718322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880470.718323:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880470.718378:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880470.718382:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880470.718384:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880471.742313:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880471.742323:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880471.742341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880471.742343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880471.742441:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880471.742448:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880471.742451:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880472.766340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880472.766346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880472.766351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880472.766352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880472.766431:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880472.766435:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880472.766437:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880473.790365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880473.790375:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880473.790378:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880473.790382:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880473.790388:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880473.790390:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880473.790392:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880473.790394:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880473.790399:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880473.790400:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880473.790460:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880473.790464:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880473.790467:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880474.750383:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306105807, at 4306105856 00000100:00000040:3.0:1788880474.750383:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306105807, at 4306105856 00000100:00000040:1.0:1788880474.750387:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306105807, at 4306105856 00000100:00000010:3.0:1788880474.750407:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000975d5882. 00000100:00000010:2.0:1788880474.750410:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001a3d5793. 00000100:00000010:1.0:1788880474.750416:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008921ad2e. 00000100:00000040:0.0:1788880474.750442:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306105807, at 4306105856 00000100:00000040:3.0:1788880474.750449:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:0.0:1788880474.750452:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006050354c. 00000100:00000001:3.0:1788880474.750456:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880474.750463:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000020:00000040:3.0:1788880474.750469:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880474.750475:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880474.750479:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880474.750486:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880474.750562:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:3.0:1788880474.750572:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880474.750586:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880474.750598:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880474.750601:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880474.750608:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_00+0] req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880474.750645:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880474.750648:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880474.750652:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880474.750655:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000020:00000040:3.0:1788880474.750659:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880474.750661:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880474.750663:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880474.750666:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880474.750671:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880474.750676:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880474.750682:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880474.750687:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000055d7d92. 00000100:00000001:1.0:1788880474.750688:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880474.750691:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880474.750694:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880474.750697:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880474.750699:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880474.750721:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880474.750724:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880474.750729:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880474.750732:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880474.750736:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788880474.750737:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880474.750740:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880474.750744:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780659968:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880474.750748:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_01+1] req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880474.750763:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880474.750771:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880474.750772:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880474.750775:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880474.750777:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880474.750780:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880474.750782:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000010:2.0:1788880474.750787:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000039bd304c. 00000100:00000010:3.0:1788880474.750787:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 02000000:00000001:2.0:1788880474.750792:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880474.750793:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880474.750795:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880474.750798:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880474.750799:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880474.750802:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:2.0:1788880474.750802:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8697e0a. 02000000:00000001:3.0:1788880474.750802:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880474.750807:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880474.750811:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880474.750814:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000078d4a4ff. 02000000:00000010:3.0:1788880474.750814:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:3.0:1788880474.750821:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880474.750825:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780659968, portal 25 00000100:00000001:2.0:1788880474.750830:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880474.750831:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000100:00000001:1.0:1788880474.750834:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880474.750837:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880474.750838:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880490 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880474.750841:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880474.750845:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880474.750850:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880474.750857:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00100000:1.0:1788880474.750858:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780660096:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880474.750865:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880474.750866:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880474.750871:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:2.0:1788880474.750872:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f779aa21. 02000000:00000001:1.0:1788880474.750874:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880474.750878:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:2.0:1788880474.750878:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780659968, offset 0 02000000:00000010:1.0:1788880474.750887:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ed94d533. 00000400:00000200:2.0:1788880474.750889:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880474.750892:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880474.750898:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000400:00000010:1.0:1788880474.750904:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 00000100:00000200:1.0:1788880474.750907:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780660096, portal 10 00000400:00000200:2.0:1788880474.750909:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880474.750910:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788880474.750917:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880474.750921:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000400:00000200:2.0:1788880474.750925:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788880474.750930:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:2.0:1788880474.750932:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:1.0:1788880474.750940:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880474.750940:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880474.750949:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000400:00000200:2.0:1788880474.750950:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [226:226:256:4294967295] 192.168.204.111@tcp LPNI seq info [226:226:8:4294967295] 00000100:00000001:3.0:1788880474.750953:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880474.750962:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_02+2] req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880474.750965:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880474.750974:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880474.750980:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d71e0867. 00000400:00000200:2.0:1788880474.750980:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:1.0:1788880474.750986:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780660096, offset 0 00000800:00000200:2.0:1788880474.750991:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880474.750996:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880474.751000:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5d1e8ed. 00000400:00000200:1.0:1788880474.751004:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880474.751005:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880474.751010:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:1.0:1788880474.751011:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880474.751014:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:3.0:1788880474.751018:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:1.0:1788880474.751019:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880474.751026:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880474.751027:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11443) 00000800:00000200:2.0:1788880474.751030:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880474.751033:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788880474.751038:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5d1e8ed type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880474.751042:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [227:227:255:4294967295] 192.168.204.111@tcp LPNI seq info [227:227:7:4294967295] 00000100:00000001:3.0:1788880474.751047:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880474.751053:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:2.0:1788880474.751055:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880474.751056:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880474.751058:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:1.0:1788880474.751066:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880474.751068:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306110927 00000100:00000001:3.0:1788880474.751071:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:1.0:1788880474.751072:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000100:00000010:2.0:1788880474.751075:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008b80730b. 00000100:00000001:3.0:1788880474.751075:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:1.0:1788880474.751081:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880474.751081:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880474.751084:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880474.751085:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880474.751088:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:1.0:1788880474.751091:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880474.751094:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306110927 00000800:00000200:1.0:1788880474.751095:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000040:3.0:1788880474.751097:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788880474.751101:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fbf478f2. 00000100:00000001:1.0:1788880474.751102:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880474.751108:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:1.0:1788880474.751109:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 387726 usecs (388 jiffies) 00000100:00000001:3.0:1788880474.751113:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880474.751116:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306106244 00000100:00000001:3.0:1788880474.751116:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880474.751117:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306110927 02000000:00000001:3.0:1788880474.751123:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000010:2.0:1788880474.751124:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fa7e66e6. 02000000:00000001:3.0:1788880474.751126:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880474.751129:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306110927 02000000:00000001:3.0:1788880474.751130:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880474.751131:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880474.751139:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880474.751139:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780660224:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880474.751140:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306106244 00000100:00000001:2.0:1788880474.751140:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306110927 00000100:00000001:3.0:1788880474.751151:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880474.751152:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306110927 00000100:00000010:2.0:1788880474.751152:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045d7c07c. 00000100:00000001:0.0:1788880474.751153:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880474.751154:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880474.751156:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880474.751158:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306106244 02000000:00000001:3.0:1788880474.751159:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880474.751163:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306110927 02000000:00000010:3.0:1788880474.751164:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000056df9207. 00000100:00000001:1.0:1788880474.751169:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306106244 02000000:00000001:3.0:1788880474.751170:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880474.751173:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000010:3.0:1788880474.751175:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ce91c67e. 00000100:00000001:2.0:1788880474.751178:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788880474.751185:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 00000100:00000001:1.0:1788880474.751187:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306110927 00000100:00000001:2.0:1788880474.751188:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880474.751189:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780660224, portal 10 00000100:00000001:3.0:1788880474.751194:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880474.751195:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880474.751196:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880474.751196:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000100:00000001:1.0:1788880474.751200:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880474.751203:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880474.751205:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880474.751206:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880474.751210:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880474.751211:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880474.751216:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880474.751218:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880474.751220:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880474.751222:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880474.751223:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:3.0:1788880474.751224:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880474.751226:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:3.0:1788880474.751229:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b6d0d6b5. 00000100:00000001:1.0:1788880474.751230:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880474.751231:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880474.751254:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:3.0:1788880474.751254:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780660224, offset 0 00000100:00000001:1.0:1788880474.751257:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880474.751261:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880474.751262:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880474.751263:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880474.751286:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880474.751291:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880474.751295:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880474.751299:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788880474.751301:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:3.0:1788880474.751304:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000010:0.0:1788880474.751308:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5d1e8ed. 00000400:00000200:3.0:1788880474.751309:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [228:228:254:4294967295] 192.168.204.111@tcp LPNI seq info [228:228:6:4294967295] 00000400:00000200:0.0:1788880474.751322:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880474.751330:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:0.0:1788880474.751340:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000200:3.0:1788880474.751345:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880474.751351:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f779aa21 00000800:00000010:3.0:1788880474.751352:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000400:00000010:0.0:1788880474.751358:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f779aa21. 00000800:00000200:3.0:1788880474.751365:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880474.751369:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:3.0:1788880474.751373:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880474.751377:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000100:00000200:0.0:1788880474.751381:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880490 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880474.751385:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880474.751392:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 24335 usecs (25 jiffies) 00000100:00000001:3.0:1788880474.751403:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306105882 00000100:00000001:3.0:1788880474.751408:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306110927 00000100:00000001:0.0:1788880474.751409:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880474.751412:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306105882 00000100:00000001:3.0:1788880474.751418:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306110927 00000100:00000040:0.0:1788880474.751419:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880490 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880474.751422:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306105882 00000100:00000001:3.0:1788880474.751429:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306110927 00000100:00000001:3.0:1788880474.751434:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306105882 00000100:00000001:3.0:1788880474.751439:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306110927 00000100:00000001:3.0:1788880474.751447:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880474.751456:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880474.751458:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880474.751460:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880474.751470:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880474.751473:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880474.751487:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880474.751489:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:0.0:1788880474.751493:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880474.751496:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880474.751499:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d71e0867 00000400:00000010:0.0:1788880474.751501:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d71e0867. 00000100:00000001:0.0:1788880474.751503:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880474.751506:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880474.751515:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880474.751517:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880474.751522:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880474.751524:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880474.751580:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880474.751583:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000400:00000200:0.0:1788880474.751588:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880474.751591:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880474.751607:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b6d0d6b5 00000400:00000010:0.0:1788880474.751617:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b6d0d6b5. 00000100:00000001:0.0:1788880474.751621:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880474.751626:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880474.751633:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880474.751635:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880474.751640:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880474.751642:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880474.753760:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880474.753776:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880474.753781:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880474.753784:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880474.753795:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880474.753811:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06d80 00000400:00000200:2.0:1788880474.753820:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x7fd [1] + 224 00000800:00000001:2.0:1788880474.753828:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880474.753832:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880474.753835:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880474.753838:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880474.753856:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880474.753862:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880474.753871:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880474.753884:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880474.753892:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880474.753895:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880474.753896:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880474.753898:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880474.753907:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880474.753914:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b06d00 00000100:00000001:1.0:1788880474.753921:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880474.753921:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x7f5 [1] + 224 00000100:00000001:1.0:1788880474.753926:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880474.753932:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880474.753935:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880474.753938:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880474.753939:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788880474.753943:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:1.0:1788880474.753944:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:2.0:1788880474.753950:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880474.753951:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:1.0:1788880474.753955:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:1.0:1788880474.753958:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880474.753961:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880474.753961:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880474.753969:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880474.753974:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880474.753986:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880474.753987:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880474.753992:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880474.753995:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880474.753997:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880474.753999:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880474.754001:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880474.754001:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880474.754003:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880474.754007:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880474.754012:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880474.754013:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=142 v=5 (5 5 0 0) 00000800:00000001:2.0:1788880474.754014:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880474.754018:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880474.754019:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=142 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880474.754022:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880474.754026:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880474.754026:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00010000:00000001:1.0:1788880474.754031:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:2.0:1788880474.754034:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06e00 00000400:00000200:2.0:1788880474.754039:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x805 [1] + 224 00000800:00000001:2.0:1788880474.754044:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:1.0:1788880474.754046:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880474.754052:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880474.754053:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000800:00000001:2.0:1788880474.754056:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880474.754059:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000200:2.0:1788880474.754060:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880474.754061:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880474.754068:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000200:2.0:1788880474.754072:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880474.754074:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880474.754081:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:1.0:1788880474.754082:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880474.754094:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880474.754097:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880474.754099:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880474.754103:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:3.0:1788880474.754108:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00100000:1.0:1788880474.754115:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780660096:192.168.204.111@tcp:400: 00000400:00000010:3.0:1788880474.754115:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8697e0a. 00000400:00000200:3.0:1788880474.754126:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000078d4a4ff 00000100:00000001:1.0:1788880474.754128:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880474.754131:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000078d4a4ff. 00000100:00000001:1.0:1788880474.754134:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880474.754138:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780660096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880474.754139:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880474.754140:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880474.754148:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:2.0:1788880474.754155:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880474.754155:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880474.754156:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880474.754165:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ed94d533. 02000000:00000001:1.0:1788880474.754176:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880474.754182:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880474.754187:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880474.754200:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880474.754209:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000055d7d92. 02000000:00000001:1.0:1788880474.754222:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880474.754226:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880474.754229:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:1.0:1788880474.754287:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:2.0:1788880474.754288:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880474.754291:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880474.754297:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880474.754303:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880474.754311:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880474.754315:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880474.754316:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000001:2.0:1788880474.754321:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880474.754322:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880474.754322:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880474.754329:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880474.754329:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880474.754336:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880474.754340:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880474.754341:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880474.754346:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880474.754347:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880474.754348:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880474.754355:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880474.754357:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880474.754361:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880474.754365:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:3.0:1788880474.754366:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=152 v=5 (5 5 0 0) 00000400:00000010:0.0:1788880474.754367:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ce91c67e. 00000400:00000200:0.0:1788880474.754375:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 00000100:00001000:3.0:1788880474.754375:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=152 v=5 (1 1 0 0) 00000400:00000010:0.0:1788880474.754379:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 00000100:00000001:3.0:1788880474.754381:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880474.754384:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880474.754385:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:3.0:1788880474.754388:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880474.754391:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880474.754392:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880474.754395:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880474.754403:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880474.754410:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880474.754421:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880474.754426:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880474.754427:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880474.754429:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880474.754430:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880474.754434:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880474.754438:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:3.0:1788880474.754438:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880474.754441:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880474.754444:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880474.754450:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880474.754450:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780659968:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880474.754457:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880474.754459:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=152 v=5 (5 5 0 0) 00000100:00000001:3.0:1788880474.754459:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00001000:0.0:1788880474.754464:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=126 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880474.754469:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:3.0:1788880474.754471:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780659968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880474.754472:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880474.754475:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880474.754481:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880474.754485:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880474.754489:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880474.754492:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000039bd304c. 02000000:00000001:3.0:1788880474.754497:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880474.754501:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880474.754504:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00010000:00000001:0.0:1788880474.754510:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880474.754510:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880474.754514:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 00000100:00000001:0.0:1788880474.754516:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880474.754520:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:3.0:1788880474.754520:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880474.754522:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000001:3.0:1788880474.754524:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880474.754525:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880474.754527:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000040:0.0:1788880474.754532:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880474.754533:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880474.754536:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880474.754541:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880474.754546:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880474.754548:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880474.754549:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880474.754551:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880474.754554:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880474.754555:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880474.754559:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880474.754563:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880474.754569:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780660224:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880474.754574:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880474.754576:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880474.754578:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780660224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880490 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880474.754584:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880474.754586:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880474.754588:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000056df9207. 02000000:00000001:0.0:1788880474.754591:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880474.754592:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880474.754593:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880474.754595:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880474.754605:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 02000000:00000001:0.0:1788880474.754611:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880474.754612:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880474.754613:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:0.0:1788880474.754616:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880474.754616:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880474.754623:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880474.754628:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880474.754629:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880474.754630:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880474.754632:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880474.814389:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880474.814398:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880474.814406:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880474.814407:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880474.814464:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880474.814469:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880474.814471:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880475.838559:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880475.838568:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880475.838581:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880475.838582:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880475.838702:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880475.838706:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880475.838708:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880476.862362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880476.862371:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880476.862376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880476.862379:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880476.862381:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880476.862385:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880476.862387:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880476.862389:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880476.862391:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880476.862392:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880476.862456:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880476.862461:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880476.862464:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880477.886330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880477.886336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880477.886342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880477.886342:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880477.886410:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880477.886413:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880477.886414:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880478.910368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880478.910375:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880478.910382:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880478.910384:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880478.910450:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880478.910454:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880478.910457:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880479.870346:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306110927, at 4306110976 00000100:00000040:1.0:1788880479.870346:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306110927, at 4306110976 00000100:00000040:2.0:1788880479.870351:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306110927, at 4306110976 00000100:00000010:0.0:1788880479.870363:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008b80730b. 00000100:00000010:1.0:1788880479.870365:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fbf478f2. 00000100:00000010:2.0:1788880479.870365:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fa7e66e6. 00000100:00000040:3.0:1788880479.870421:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306110927, at 4306110976 00000100:00000040:1.0:1788880479.870422:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880479.870429:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880479.870430:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045d7c07c. 00000100:00000010:1.0:1788880479.870435:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002b387587. 00000020:00000040:1.0:1788880479.870440:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880479.870445:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880479.870448:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880479.870466:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880479.870473:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:1.0:1788880479.870480:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880479.870490:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880479.870497:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880479.870499:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880479.870504:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002b387587] to pc [ptlrpcd_00_03+3] req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880479.870533:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880479.870536:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880479.870540:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880479.870543:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e2be23b2. 00000020:00000040:1.0:1788880479.870545:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880479.870548:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880479.870549:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880479.870551:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880479.870552:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880479.870556:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:1.0:1788880479.870557:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:0.0:1788880479.870560:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880479.870562:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880479.870567:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880479.870570:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880479.870576:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880479.870579:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880479.870581:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880479.870582:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880479.870590:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e2be23b2] to pc [ptlrpcd_00_00+0] req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880479.870592:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880479.870596:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880479.870601:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880479.870604:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880479.870607:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880479.870610:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780660352:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880479.870615:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880479.870617:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880479.870622:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880479.870625:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:0.0:1788880479.870626:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:1.0:1788880479.870628:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 02000000:00000001:0.0:1788880479.870630:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880479.870633:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880479.870634:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880479.870641:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000036ec89fa. 00000100:00000001:1.0:1788880479.870642:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880479.870643:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880479.870645:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880479.870645:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880479.870647:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880479.870647:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:2.0:1788880479.870650:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880479.870652:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000400:00000010:0.0:1788880479.870654:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8ada50c. 00000100:00000001:1.0:1788880479.870656:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880479.870657:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:0.0:1788880479.870662:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000100:00000040:1.0:1788880479.870663:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880479.870663:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880479.870667:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880479.870668:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880479.870669:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:0.0:1788880479.870670:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780660352, portal 25 00000100:00000001:1.0:1788880479.870674:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:2.0:1788880479.870674:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880479.870675:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880479.870679:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_00_01+1] req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880479.870682:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880479.870684:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880479.870686:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938057728 : -126823771493888 : ffff8ca78a26aa00) 00000100:00000001:2.0:1788880479.870687:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880479.870690:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880479.870691:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:1.0:1788880479.870693:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880479.870693:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880479.870696:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880479.870697:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880495 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880479.870698:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1788880479.870700:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780660480:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880479.870704:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11448) 00000100:00000001:2.0:1788880479.870708:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880479.870712:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880479.870713:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:2.0:1788880479.870715:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880479.870720:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880479.870720:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880479.870721:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880479.870725:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000cda43252. 00000400:00000010:0.0:1788880479.870726:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00100000:3.0:1788880479.870726:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 02000000:00000001:2.0:1788880479.870729:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880479.870730:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880479.870735:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000100:00000001:3.0:1788880479.870736:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:0.0:1788880479.870737:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780660352, offset 0 00000100:00000001:3.0:1788880479.870740:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880479.870742:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000001:3.0:1788880479.870743:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880479.870746:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880479.870749:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880479.870749:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880479.870756:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880479.870757:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780660480, portal 10 00000100:00000001:2.0:1788880479.870760:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880479.870762:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055936 : -126823771495680 : ffff8ca78a26a300) 00000400:00000200:0.0:1788880479.870766:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:2.0:1788880479.870772:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880479.870776:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880479.870780:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880479.870783:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880479.870786:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880479.870788:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788880479.870792:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:2.0:1788880479.870792:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880479.870792:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880479.870796:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005071f3f0. 00000100:00000200:2.0:1788880479.870800:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780660480, offset 0 00000400:00000200:0.0:1788880479.870803:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880479.870805:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880479.870809:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880479.870812:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:3.0:1788880479.870812:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780660608:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880479.870818:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880479.870821:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [229:229:256:4294967295] 192.168.204.111@tcp LPNI seq info [229:229:8:4294967295] 02000000:00000001:3.0:1788880479.870822:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880479.870825:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880479.870828:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880479.870831:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005bd099ee. 02000000:00000001:3.0:1788880479.870834:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880479.870840:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:3.0:1788880479.870841:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000067d070dd. 00000400:00000010:3.0:1788880479.870848:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 00000800:00000200:0.0:1788880479.870851:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880479.870853:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:3.0:1788880479.870853:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780660608, portal 10 00000800:00000010:0.0:1788880479.870857:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000a371c7dc. 00000100:00000001:3.0:1788880479.870858:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788880479.870860:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880479.870862:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 00000400:00000200:2.0:1788880479.870866:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788880479.870867:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:3.0:1788880479.870871:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880479.870879:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788880479.870880:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880479.870886:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788880479.870887:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000a371c7dc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880479.870890:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:2.0:1788880479.870894:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [230:230:255:4294967295] 192.168.204.111@tcp LPNI seq info [230:230:7:4294967295] 00000100:00000040:3.0:1788880479.870897:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880479.870902:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b6d0d6b5. 00000100:00000001:0.0:1788880479.870905:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880479.870905:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780660608, offset 0 00000100:00000001:0.0:1788880479.870911:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306116047 00000400:00000200:3.0:1788880479.870917:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880479.870918:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c0bfe00. 00000400:00000200:2.0:1788880479.870923:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880479.870925:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880479.870932:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306116047 00000800:00000200:2.0:1788880479.870932:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880479.870935:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:0.0:1788880479.870937:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000007d3948ef. 00000800:00000010:2.0:1788880479.870939:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e5360691. 00000100:00000001:0.0:1788880479.870942:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:3.0:1788880479.870942:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880479.870948:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880479.870948:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880479.870949:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306116047 00000100:00000010:0.0:1788880479.870955:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000da3f50d1. 00000400:00000200:3.0:1788880479.870955:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1788880479.870958:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880479.870961:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:2.0:1788880479.870963:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e5360691 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880479.870963:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880479.870966:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306116047 00000400:00000200:3.0:1788880479.870971:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [231:231:254:4294967295] 192.168.204.111@tcp LPNI seq info [231:231:6:4294967295] 00000100:00000010:0.0:1788880479.870972:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006050354c. 00000100:00000001:2.0:1788880479.870972:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880479.870978:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:1.0:1788880479.870980:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:2.0:1788880479.870980:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 382000 usecs (382 jiffies) 00000100:00000001:0.0:1788880479.870982:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:1.0:1788880479.870985:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000a371c7dc. 00000100:00000001:2.0:1788880479.870986:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306111358 00000100:00000001:0.0:1788880479.870988:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880479.870992:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:1.0:1788880479.870993:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880479.870993:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306116047 00000100:00000001:0.0:1788880479.870998:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880479.871000:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306111358 00000400:00000200:1.0:1788880479.871002:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880479.871003:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880479.871006:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306116047 00000800:00000200:3.0:1788880479.871007:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880479.871010:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788880479.871011:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000100:00000001:2.0:1788880479.871012:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306111358 00000100:00000001:0.0:1788880479.871014:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:3.0:1788880479.871014:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000400:00000010:1.0:1788880479.871015:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:2.0:1788880479.871016:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306116047 00000100:00000001:0.0:1788880479.871020:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880479.871021:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880479.871021:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306111358 00000100:00000001:0.0:1788880479.871023:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:3.0:1788880479.871025:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880479.871026:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306116047 00000100:00000001:0.0:1788880479.871027:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880479.871028:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880495 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880479.871033:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880479.871033:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880479.871039:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880479.871039:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880479.871044:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880479.871046:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880479.871049:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880479.871051:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880479.871052:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880479.871053:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880495 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880479.871055:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880479.871057:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880479.871059:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880479.871062:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 24017 usecs (25 jiffies) 00000100:00000001:2.0:1788880479.871064:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880479.871069:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306111001 00000100:00000001:1.0:1788880479.871071:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880479.871073:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306116047 00000100:00000001:1.0:1788880479.871076:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880479.871077:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306111001 00000100:00000001:3.0:1788880479.871080:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306116047 00000100:00000001:3.0:1788880479.871083:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306111001 00000100:00000001:3.0:1788880479.871084:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306116047 00000100:00000001:3.0:1788880479.871087:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306111001 00000100:00000001:3.0:1788880479.871089:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306116047 00000100:00000001:3.0:1788880479.871091:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880479.871093:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880479.871094:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880479.871096:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880479.871098:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e5360691. 00000100:00000001:3.0:1788880479.871100:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880479.871123:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880479.871126:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880479.871129:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005071f3f0 00000400:00000010:1.0:1788880479.871131:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005071f3f0. 00000100:00000001:1.0:1788880479.871133:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880479.871136:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880479.871144:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880479.871146:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880479.871152:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880479.871153:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880479.871201:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880479.871203:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000400:00000200:1.0:1788880479.871217:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880479.871221:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880479.871224:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b6d0d6b5 00000400:00000010:1.0:1788880479.871225:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b6d0d6b5. 00000100:00000001:1.0:1788880479.871228:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880479.871230:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880479.871254:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880479.871257:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880479.871262:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880479.871264:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880479.873251:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880479.873263:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880479.873267:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880479.873269:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880479.873278:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880479.873291:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06f00 00000400:00000200:0.0:1788880479.873298:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x815 [1] + 224 00000800:00000001:0.0:1788880479.873303:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880479.873308:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880479.873311:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880479.873313:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880479.873317:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880479.873321:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880479.873328:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880479.873344:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880479.873353:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880479.873357:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880479.873359:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880479.873361:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880479.873365:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880479.873372:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b06e80 00000400:00000200:0.0:1788880479.873378:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x80d [1] + 224 00000100:00000001:2.0:1788880479.873381:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:0.0:1788880479.873383:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880479.873385:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880479.873396:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880479.873398:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:0.0:1788880479.873399:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:2.0:1788880479.873401:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:0.0:1788880479.873404:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:2.0:1788880479.873404:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:2.0:1788880479.873409:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880479.873410:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880479.873412:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880479.873415:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880479.873416:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880479.873423:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880479.873425:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880479.873449:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880479.873450:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880479.873452:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880479.873453:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880479.873457:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880479.873460:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880479.873463:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880479.873465:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880479.873469:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:2.0:1788880479.873472:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=147 v=5 (5 5 0 0) 00000800:00000001:0.0:1788880479.873475:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:2.0:1788880479.873477:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=147 v=5 (1 1 0 0) 00000800:00000001:0.0:1788880479.873480:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880479.873481:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880479.873483:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880479.873484:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:2.0:1788880479.873486:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880479.873486:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:0.0:1788880479.873487:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880479.873490:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:2.0:1788880479.873491:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880479.873495:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000001:3.0:1788880479.873496:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:2.0:1788880479.873497:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:3.0:1788880479.873500:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880479.873501:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000010:3.0:1788880479.873504:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8ada50c. 00000400:00000200:0.0:1788880479.873505:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880479.873505:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880479.873512:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880479.873512:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000400:00000010:3.0:1788880479.873517:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00000400:00000200:0.0:1788880479.873520:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b06f80 00000100:00000001:3.0:1788880479.873523:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880479.873529:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880479.873531:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x81d [1] + 224 00000100:00000200:3.0:1788880479.873532:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880479.873535:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880479.873538:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880479.873541:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:2.0:1788880479.873547:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880479.873549:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880479.873554:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:3.0:1788880479.873557:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880479.873563:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880479.873570:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880479.873575:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880479.873576:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880479.873580:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880479.873586:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:0.0:1788880479.873594:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880479.873597:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00100000:2.0:1788880479.873599:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780660480:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880479.873601:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880479.873605:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880479.873608:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880479.873611:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880479.873612:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880479.873616:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e2be23b2 x1875776780660480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880479.873625:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=7 v=5 (5 5 5 0) 00000100:00000001:2.0:1788880479.873627:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880479.873628:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:2.0:1788880479.873630:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00001000:3.0:1788880479.873632:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=7 v=5 (1 1 1 0) 02000000:00000010:2.0:1788880479.873634:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000cda43252. 00000800:00000001:0.0:1788880479.873635:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880479.873639:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:0.0:1788880479.873641:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880479.873642:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880479.873642:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880479.873646:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00010000:00000001:3.0:1788880479.873648:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000040:2.0:1788880479.873649:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00010000:00000001:3.0:1788880479.873652:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880479.873654:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880479.873656:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880479.873658:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 02000000:00000001:2.0:1788880479.873664:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:3.0:1788880479.873664:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880479.873665:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880479.873666:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880479.873668:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:2.0:1788880479.873668:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e2be23b2. 00000400:00000001:1.0:1788880479.873673:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880479.873673:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000400:00000001:1.0:1788880479.873675:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880479.873678:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000067d070dd. 00000100:00000001:3.0:1788880479.873679:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880479.873680:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880479.873684:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880479.873686:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:1.0:1788880479.873687:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 00000100:00000001:3.0:1788880479.873689:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880479.873690:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 00000100:00000001:2.0:1788880479.873690:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880479.873694:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880479.873695:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1788880479.873697:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880479.873698:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880479.873701:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880479.873705:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880479.873722:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780660352:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880479.873726:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880479.873730:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880479.873735:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880479.873740:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880479.873740:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002b387587 x1875776780660352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880479.873743:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880479.873750:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880479.873751:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880479.873752:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880479.873755:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880479.873756:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880479.873756:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:1.0:1788880479.873760:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880479.873760:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000036ec89fa. 00000100:00001000:1.0:1788880479.873764:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=7 v=5 (5 5 5 0) 02000000:00000001:3.0:1788880479.873765:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:1.0:1788880479.873769:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=131 v=5 (1 1 0 0) 00000020:00000001:3.0:1788880479.873769:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880479.873772:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880479.873773:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880479.873774:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880479.873777:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:3.0:1788880479.873779:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880479.873781:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 00010000:00080000:1.0:1788880479.873782:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880479.873785:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880479.873788:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880479.873790:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002b387587. 00010000:00000001:1.0:1788880479.873792:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880479.873795:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880479.873797:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880479.873799:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880479.873800:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880479.873801:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880479.873804:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880479.873805:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880479.873812:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880479.873812:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880479.873815:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880479.873818:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880479.873820:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880479.873828:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880479.873837:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880479.873838:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880479.873842:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880479.873849:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780660608:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880479.873860:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880479.873862:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880479.873864:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780660608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880495 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880479.873870:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880479.873871:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880479.873872:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005bd099ee. 02000000:00000001:1.0:1788880479.873875:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880479.873876:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880479.873877:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880479.873879:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880479.873880:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:1.0:1788880479.873882:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880479.873883:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880479.873884:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000001:1.0:1788880479.873887:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880479.873888:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880479.873889:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880479.873893:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880479.873894:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880479.873895:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880479.873897:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880479.936887:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880479.936894:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880479.936901:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880479.936902:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880479.936968:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880479.936971:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880479.936975:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880480.958359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880480.958366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880480.958373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880480.958374:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880480.958544:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880480.958551:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880480.958556:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880481.982339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880481.982344:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880481.982349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880481.982350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880481.982395:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880481.982398:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880481.982400:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880483.006435:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880483.006447:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880483.006471:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880483.006473:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880483.006536:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880483.006546:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880483.006549:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880484.030324:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880484.030336:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880484.030339:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880484.030343:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880484.030349:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880484.030352:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880484.030354:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880484.030356:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880484.030361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880484.030363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880484.030437:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880484.030441:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880484.030444:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880484.990312:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306116047, at 4306116096 00000100:00000040:1.0:1788880484.990323:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306116047, at 4306116096 00000100:00000040:3.0:1788880484.990324:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306116047, at 4306116096 00000100:00000010:2.0:1788880484.990326:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000da3f50d1. 00000100:00000010:3.0:1788880484.990332:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006050354c. 00000100:00000010:1.0:1788880484.990335:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000007d3948ef. 00000100:00000040:0.0:1788880484.990347:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306116047, at 4306116096 00000100:00000010:0.0:1788880484.990355:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c0bfe00. 00000100:00000040:2.0:1788880484.990358:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880484.990362:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880484.990369:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000020:00000040:2.0:1788880484.990373:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880484.990377:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880484.990379:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880484.990384:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880484.990391:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f9812869. 00000100:00000001:2.0:1788880484.990398:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880484.990408:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880484.990418:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880484.990421:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880484.990427:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_02+2] req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880484.990457:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880484.990460:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880484.990464:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880484.990468:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000884e74b8. 00000020:00000040:2.0:1788880484.990470:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880484.990473:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880484.990475:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880484.990475:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880484.990477:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880484.990482:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000001:3.0:1788880484.990482:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880484.990487:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880484.990487:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:3.0:1788880484.990491:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880484.990493:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880484.990497:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:2.0:1788880484.990498:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880484.990501:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880484.990505:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880484.990508:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880484.990509:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880484.990511:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880484.990511:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880484.990524:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000884e74b8] to pc [ptlrpcd_00_03+3] req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880484.990526:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880484.990548:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880484.990551:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880484.990556:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880484.990556:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880484.990560:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880484.990560:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880484.990565:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880484.990566:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880484.990571:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 00000100:00100000:3.0:1788880484.990572:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780660736:192.168.204.111@tcp:400: 00000020:00000040:2.0:1788880484.990575:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880484.990576:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880484.990579:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880484.990581:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00100000:0.0:1788880484.990582:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 02000000:00000001:2.0:1788880484.990585:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880484.990586:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880484.990586:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880484.990590:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880484.990594:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880484.990595:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880484.990597:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880484.990600:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1788880484.990601:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005bd099ee. 02000000:00000010:2.0:1788880484.990602:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5d1e8ed. 00000100:00000001:0.0:1788880484.990603:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880484.990606:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880484.990607:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880484.990609:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880484.990609:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880484.990614:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009c9e5278. 00000100:00000040:0.0:1788880484.990618:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880484.990618:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:3.0:1788880484.990621:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000100:00000200:3.0:1788880484.990630:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780660736, portal 25 00000100:00000001:3.0:1788880484.990636:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880484.990647:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 00000100:00000001:0.0:1788880484.990648:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880484.990648:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880484.990651:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880484.990651:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880484.990657:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880484.990660:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_00+0] req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880484.990661:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880484.990667:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880484.990670:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880500 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880484.990674:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780660864:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880484.990682:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880484.990686:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880484.990687:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880484.990689:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880484.990692:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880484.990693:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880484.990693:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:0.0:1788880484.990697:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e10984e3. 00000100:00000040:2.0:1788880484.990698:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880484.990699:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 02000000:00000001:0.0:1788880484.990702:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880484.990705:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780660736, offset 0 00000100:00000040:2.0:1788880484.990706:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:0.0:1788880484.990709:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c1b65ba9. 00000100:00000040:2.0:1788880484.990712:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11453) 00000400:00000010:0.0:1788880484.990716:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000400:00000200:3.0:1788880484.990717:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:0.0:1788880484.990724:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780660864, portal 10 00000100:00000001:2.0:1788880484.990724:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880484.990725:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880484.990728:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00100000:2.0:1788880484.990728:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788880484.990732:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938423808 : -126823771127808 : ffff8ca78a2c4000) 00000400:00000200:3.0:1788880484.990732:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880484.990733:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880484.990734:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880484.990739:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:0.0:1788880484.990742:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880484.990744:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880484.990749:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880484.990754:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880484.990755:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880484.990759:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880484.990760:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788880484.990760:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [232:232:256:4294967295] 192.168.204.111@tcp LPNI seq info [232:232:8:4294967295] 00000100:00000001:2.0:1788880484.990763:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880484.990766:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880484.990767:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880484.990768:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880484.990774:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000115221d8. 00000400:00000200:3.0:1788880484.990774:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:2.0:1788880484.990777:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880484.990779:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780660864, offset 0 00000100:00000001:2.0:1788880484.990785:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000200:3.0:1788880484.990787:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880484.990788:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880484.990793:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880484.990795:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000010:3.0:1788880484.990796:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 02000000:00000001:2.0:1788880484.990798:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880484.990802:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780660992:192.168.204.111@tcp:400: 00000800:00000200:3.0:1788880484.990807:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880484.990808:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880484.990815:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880484.990818:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:2.0:1788880484.990819:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000800:00000200:3.0:1788880484.990821:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:2.0:1788880484.990823:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880484.990827:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:3.0:1788880484.990827:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880484.990828:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880484.990834:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000010:2.0:1788880484.990834:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000039bd304c. 02000000:00000001:2.0:1788880484.990838:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880484.990839:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:2.0:1788880484.990843:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000400:00000200:0.0:1788880484.990844:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880484.990846:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880484.990850:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000400:00000200:0.0:1788880484.990852:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [233:233:255:4294967295] 192.168.204.111@tcp LPNI seq info [233:233:7:4294967295] 00000100:00000001:3.0:1788880484.990852:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306121167 00000100:00000200:2.0:1788880484.990858:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780660992, portal 10 00000100:00000010:3.0:1788880484.990860:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b0b8cfdf. 00000100:00000001:2.0:1788880484.990862:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880484.990866:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000100:00000001:3.0:1788880484.990867:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880484.990871:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880484.990874:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306121167 00000100:00000040:2.0:1788880484.990877:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880484.990880:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788880484.990881:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a5ad6403. 00000800:00000010:0.0:1788880484.990887:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000100:00000001:3.0:1788880484.990887:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:2.0:1788880484.990890:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880484.990895:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306121167 00000800:00000200:0.0:1788880484.990897:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:2.0:1788880484.990899:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788880484.990901:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000067a34a59. 00000400:00000010:2.0:1788880484.990904:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a3b2cf4a. 00000100:00000001:3.0:1788880484.990907:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:0.0:1788880484.990908:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:2.0:1788880484.990908:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780660992, offset 0 00000800:00000200:0.0:1788880484.990914:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880484.990918:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880484.990919:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306121167 00000100:00000001:0.0:1788880484.990921:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880484.990925:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:3.0:1788880484.990927:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000dba5f48d. 00000100:00000040:0.0:1788880484.990928:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 376435 usecs (377 jiffies) 00000400:00000200:2.0:1788880484.990928:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880484.990933:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880484.990934:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306116473 00000100:00000001:0.0:1788880484.990939:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306121167 00000400:00000200:2.0:1788880484.990939:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880484.990939:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880484.990945:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306116473 00000400:00000200:2.0:1788880484.990945:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880484.990945:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880484.990951:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306121167 00000400:00000200:2.0:1788880484.990951:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880484.990953:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880484.990957:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306116473 00000800:00000200:1.0:1788880484.990957:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:2.0:1788880484.990957:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [234:234:254:4294967295] 192.168.204.111@tcp LPNI seq info [234:234:6:4294967295] 00000100:00000001:3.0:1788880484.990959:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880484.990963:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306121167 00000800:00000010:1.0:1788880484.990963:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000100:00000001:3.0:1788880484.990964:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880484.990967:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880484.990969:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306116473 00000100:00000001:3.0:1788880484.990972:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880484.990973:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880484.990974:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306121167 00000400:00000200:2.0:1788880484.990977:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880484.990978:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880484.990979:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880484.990984:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880484.990986:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880484.990988:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880484.990992:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880484.990994:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880484.990995:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000800:00000010:2.0:1788880484.990997:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008eccda10. 00000400:00000010:1.0:1788880484.990998:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000800:00000200:2.0:1788880484.991006:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880484.991009:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:2.0:1788880484.991013:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:1.0:1788880484.991017:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880500 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880484.991018:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008eccda10 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880484.991025:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880484.991031:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 23701 usecs (24 jiffies) 00000100:00000001:1.0:1788880484.991034:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880484.991042:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880500 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880484.991045:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306116120 00000100:00000001:2.0:1788880484.991051:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306121167 00000100:00000001:2.0:1788880484.991055:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306116120 00000100:00000001:1.0:1788880484.991056:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880484.991058:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306121167 00000100:00000001:1.0:1788880484.991061:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880484.991062:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306116120 00000100:00000001:2.0:1788880484.991066:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306121167 00000100:00000001:2.0:1788880484.991068:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306116120 00000100:00000001:2.0:1788880484.991070:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306121167 00000100:00000001:2.0:1788880484.991072:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880484.991087:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880484.991090:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880484.991090:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880484.991093:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880484.991094:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000400:00000200:1.0:1788880484.991100:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880484.991103:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880484.991106:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000115221d8 00000400:00000010:1.0:1788880484.991113:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000115221d8. 00000100:00000001:1.0:1788880484.991117:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880484.991120:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880484.991129:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880484.991131:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880484.991137:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880484.991138:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880484.991158:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880484.991160:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008eccda10. 00000400:00000200:1.0:1788880484.991162:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880484.991164:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880484.991166:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a3b2cf4a 00000400:00000010:1.0:1788880484.991166:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a3b2cf4a. 00000100:00000001:1.0:1788880484.991168:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880484.991170:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880484.991174:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880484.991175:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880484.991179:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880484.991180:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880484.993331:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880484.993344:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880484.993347:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880484.993350:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880484.993359:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880484.993373:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b07000 00000400:00000200:1.0:1788880484.993380:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x825 [1] + 224 00000800:00000001:1.0:1788880484.993385:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880484.993387:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880484.993400:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880484.993404:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880484.993410:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880484.993417:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880484.993427:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880484.993448:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880484.993456:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880484.993460:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880484.993462:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880484.993465:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880484.993470:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880484.993474:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880484.993476:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07080 00000100:00000001:3.0:1788880484.993479:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880484.993485:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x82d [1] + 224 00000800:00000001:1.0:1788880484.993490:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1788880484.993490:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880484.993493:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:1.0:1788880484.993494:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880484.993495:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009c9e5278. 00000400:00000200:1.0:1788880484.993498:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880484.993499:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00000400:00000010:3.0:1788880484.993502:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000400:00000200:1.0:1788880484.993505:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880484.993506:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880484.993511:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880484.993512:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880484.993517:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880484.993529:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880484.993535:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880484.993538:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880484.993540:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880484.993545:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880484.993549:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880484.993552:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880484.993555:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880484.993557:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880484.993560:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:3.0:1788880484.993564:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=12 v=5 (5 5 5 0) 00000800:00000001:1.0:1788880484.993566:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:3.0:1788880484.993570:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=12 v=5 (1 1 1 0) 00000800:00000001:1.0:1788880484.993572:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880484.993574:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:1.0:1788880484.993576:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880484.993577:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880484.993578:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880484.993582:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:1.0:1788880484.993583:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00010000:00000001:3.0:1788880484.993586:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880484.993589:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07100 00000100:00000001:3.0:1788880484.993598:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880484.993602:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880484.993605:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x835 [1] + 224 00000100:00000040:3.0:1788880484.993605:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880484.993614:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:1.0:1788880484.993616:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880484.993618:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880484.993622:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880484.993622:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880484.993626:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c1b65ba9. 00000100:00000001:3.0:1788880484.993627:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880484.993630:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880484.993632:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000100:00000040:3.0:1788880484.993633:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880484.993636:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880484.993637:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000400:00000200:1.0:1788880484.993642:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880484.993644:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:3.0:1788880484.993646:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780660736:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880484.993651:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:0.0:1788880484.993652:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880484.993652:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880484.993656:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880484.993660:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1788880484.993663:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780660736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880484.993667:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880484.993675:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880484.993690:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880484.993694:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:1.0:1788880484.993695:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880484.993697:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005bd099ee. 00000100:00000001:0.0:1788880484.993700:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:3.0:1788880484.993702:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880484.993704:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880484.993706:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880484.993708:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880484.993713:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880484.993714:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f9812869. 02000000:00000001:3.0:1788880484.993719:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880484.993720:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880484.993721:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000100:00000001:0.0:1788880484.993722:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880484.993725:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880484.993725:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880484.993727:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880484.993727:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880484.993728:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880484.993731:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880484.993732:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880484.993735:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:0.0:1788880484.993736:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880484.993737:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880484.993740:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880484.993741:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880484.993742:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:0.0:1788880484.993744:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=152 v=5 (5 5 0 0) 00000100:00000001:3.0:1788880484.993746:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880484.993751:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=152 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880484.993755:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880484.993757:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880484.993759:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880484.993762:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880484.993765:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880484.993768:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880484.993770:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880484.993771:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880484.993772:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880484.993776:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880484.993779:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788880484.993783:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880484.993786:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880484.993789:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000100:00000001:0.0:1788880484.993791:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880484.993793:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880484.993795:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880484.993798:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880484.993803:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880484.993807:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880484.993811:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880484.993816:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880484.993833:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880484.993839:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880484.993840:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880484.993842:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880484.993843:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880484.993845:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880484.993846:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880484.993847:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880484.993852:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=12 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880484.993854:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=136 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880484.993857:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880484.993858:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880484.993859:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880484.993861:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880484.993868:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880484.993870:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880484.993871:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880484.993872:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880484.993873:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880484.993876:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880484.993882:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880484.993884:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880484.993885:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880484.993888:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880484.993906:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780660864:192.168.204.111@tcp:400: 00000100:00100000:2.0:1788880484.993912:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780660992:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880484.993915:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880484.993919:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880484.993920:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880484.993923:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880484.993925:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000884e74b8 x1875776780660864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880484.993928:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780660992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880500 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880484.993948:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880484.993949:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880484.993951:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:2.0:1788880484.993952:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880484.993955:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e10984e3. 02000000:00000010:2.0:1788880484.993956:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000039bd304c. 02000000:00000001:0.0:1788880484.993961:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:2.0:1788880484.993963:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880484.993964:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000001:2.0:1788880484.993966:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880484.993967:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000040:2.0:1788880484.993970:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880484.993971:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880484.993974:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 00000020:00000001:2.0:1788880484.993976:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:0.0:1788880484.993978:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000010:2.0:1788880484.993979:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5d1e8ed. 02000000:00000001:0.0:1788880484.993981:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880484.993983:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000884e74b8. 02000000:00000001:2.0:1788880484.993986:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880484.993988:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:2.0:1788880484.993989:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880484.993990:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:2.0:1788880484.993993:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 00000100:00000001:0.0:1788880484.993994:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880484.994000:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880484.994000:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880484.994003:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880484.994004:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880484.994008:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880484.994009:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880484.994011:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880484.994015:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880484.994018:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880484.994020:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880484.994022:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880485.055306:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880485.055313:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880485.055319:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880485.055320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880485.055387:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880485.055391:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880485.055394:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880486.078358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880486.078370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880486.078377:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880486.078378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880486.078440:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880486.078445:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880486.078447:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880487.102744:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880487.102752:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880487.102758:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880487.102761:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880487.102764:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880487.102767:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880487.102770:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880487.102772:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880487.102774:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880487.102776:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880487.102842:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880487.102846:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880487.102849:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880488.126921:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880488.126927:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880488.126932:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880488.126941:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880488.126960:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880488.126963:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880488.126966:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880489.150339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880489.150344:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880489.150350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880489.150352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880489.150518:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880489.150522:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880489.150526:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880490.110348:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306121167, at 4306121216 00000100:00000040:3.0:1788880490.110348:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306121167, at 4306121216 00000100:00000040:2.0:1788880490.110350:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306121167, at 4306121216 00000100:00000010:3.0:1788880490.110367:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000dba5f48d. 00000100:00000010:0.0:1788880490.110369:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b0b8cfdf. 00000100:00000010:2.0:1788880490.110378:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000067a34a59. 00000100:00000040:3.0:1788880490.110404:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880490.110408:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880490.110413:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000020:00000040:3.0:1788880490.110418:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000040:1.0:1788880490.110419:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306121167, at 4306121216 00000100:00000001:3.0:1788880490.110425:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880490.110428:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000010:1.0:1788880490.110437:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a5ad6403. 02000000:00000001:3.0:1788880490.110440:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880490.110449:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:3.0:1788880490.110455:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880490.110465:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880490.110473:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880490.110475:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880490.110479:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_01+1] req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880490.110499:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880490.110500:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880490.110503:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880490.110505:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880490.110507:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880490.110508:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880490.110509:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880490.110510:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880490.110512:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000252f4482. 00000100:00000001:3.0:1788880490.110516:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880490.110519:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880490.110519:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880490.110523:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880490.110526:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880490.110527:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880490.110530:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880490.110534:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880490.110536:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880490.110537:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_02+2] req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880490.110544:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880490.110557:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880490.110560:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880490.110563:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880490.110565:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880490.110569:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880490.110574:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780661120:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880490.110580:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880490.110585:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880490.110589:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880490.110591:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880490.110595:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000094e066a1. 02000000:00000001:2.0:1788880490.110597:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880490.110602:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880490.110603:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000100:00000040:3.0:1788880490.110607:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:2.0:1788880490.110611:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000001:3.0:1788880490.110612:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880490.110618:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000100:00000200:2.0:1788880490.110621:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780661120, portal 25 00000020:00000040:3.0:1788880490.110623:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880490.110625:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880490.110628:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000100:00000001:3.0:1788880490.110629:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880490.110634:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880490.110635:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880490.110636:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880490.110640:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880506 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880490.110641:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880490.110645:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880490.110645:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880490.110649:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:0.0:1788880490.110652:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880490.110658:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880490.110659:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880490.110664:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880490.110666:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880490.110668:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:2.0:1788880490.110671:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ac9b6c42. 00000100:00000200:2.0:1788880490.110677:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780661120, offset 0 00000400:00000200:2.0:1788880490.110689:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880490.110690:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880490.110694:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880490.110702:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788880490.110703:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_03+3] req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880490.110705:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880490.110708:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880490.110710:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880490.110714:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880490.110718:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788880490.110720:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880490.110726:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00100000:0.0:1788880490.110728:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780661248:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880490.110733:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880490.110738:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880490.110739:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880490.110740:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1788880490.110741:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880490.110743:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880490.110746:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880490.110747:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000010:0.0:1788880490.110752:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e10984e3. 00000100:00000040:3.0:1788880490.110752:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:2.0:1788880490.110757:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [235:235:256:4294967295] 192.168.204.111@tcp LPNI seq info [235:235:8:4294967295] 02000000:00000001:0.0:1788880490.110758:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880490.110762:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11458) 00000100:00000001:1.0:1788880490.110763:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788880490.110765:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c1b65ba9. 00000100:00000001:1.0:1788880490.110770:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:0.0:1788880490.110773:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000001:1.0:1788880490.110776:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:0.0:1788880490.110779:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780661248, portal 10 00000100:00000001:0.0:1788880490.110782:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880490.110784:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000400:00000200:2.0:1788880490.110787:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:1.0:1788880490.110792:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880490.110801:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880490.110822:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880490.110836:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5d1e8ed. 00000800:00000200:2.0:1788880490.110859:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880490.110864:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880490.110869:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880490.110870:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880490.110879:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880490.110879:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880490.110887:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5d1e8ed type 1, nob 320 niov 1 nkiov 1 02000000:00000001:1.0:1788880490.110890:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:0.0:1788880490.110894:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 02000000:00000001:1.0:1788880490.110896:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880490.110899:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780661248, offset 0 02000000:00000001:1.0:1788880490.110904:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880490.110911:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880490.110913:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788880490.110913:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780661376:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880490.110919:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306126287 00000400:00000200:0.0:1788880490.110932:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880490.110932:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000010:2.0:1788880490.110932:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fa7e66e6. 00000400:00000200:0.0:1788880490.110941:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880490.110941:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880490.110942:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 02000000:00000001:1.0:1788880490.110950:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880490.110953:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306126287 00000400:00000200:0.0:1788880490.110954:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788880490.110956:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:2.0:1788880490.110960:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ad5555dd. 00000400:00000200:0.0:1788880490.110962:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000010:1.0:1788880490.110965:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000010c9710f. 00000100:00000001:2.0:1788880490.110967:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 02000000:00000001:1.0:1788880490.110972:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880490.110975:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880490.110993:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306126287 00000400:00000200:0.0:1788880490.110994:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [236:236:255:4294967295] 192.168.204.111@tcp LPNI seq info [236:236:7:4294967295] 00000400:00000010:1.0:1788880490.110995:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000010:2.0:1788880490.111005:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045d7c07c. 00000400:00000010:1.0:1788880490.111007:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 00000400:00000200:0.0:1788880490.111016:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:1.0:1788880490.111017:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780661376, portal 10 00000100:00000001:2.0:1788880490.111020:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880490.111023:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880490.111027:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306126287 00000800:00000200:0.0:1788880490.111033:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880490.111034:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fbf478f2. 00000100:00000001:1.0:1788880490.111040:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000800:00000010:0.0:1788880490.111041:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000131c1a55. 00000100:00000001:2.0:1788880490.111041:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:0.0:1788880490.111053:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880490.111053:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880490.111057:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880490.111061:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880490.111070:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880490.111072:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880490.111080:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880490.111085:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000131c1a55 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880490.111089:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880490.111093:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880490.111096:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880490.111097:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788880490.111100:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880490.111103:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 371036 usecs (372 jiffies) 00000100:00000001:2.0:1788880490.111103:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:3.0:1788880490.111105:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5d1e8ed. 00000400:00000010:1.0:1788880490.111108:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000100:00000001:2.0:1788880490.111110:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880490.111111:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306121588 00000400:00000200:3.0:1788880490.111113:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880490.111116:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:1.0:1788880490.111117:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780661376, offset 0 00000100:00000001:0.0:1788880490.111119:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306126287 00000100:00000001:2.0:1788880490.111126:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880490.111127:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306121588 00000400:00000200:3.0:1788880490.111127:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880490.111137:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880490.111139:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306126287 00000100:00000001:2.0:1788880490.111139:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880490.111148:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306121588 00000400:00000200:3.0:1788880490.111149:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ac9b6c42 00000100:00000001:0.0:1788880490.111152:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306126287 00000400:00000010:3.0:1788880490.111153:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ac9b6c42. 00000100:00000001:0.0:1788880490.111159:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306121588 00000400:00000200:1.0:1788880490.111159:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880490.111162:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880490.111172:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306126287 00000100:00000200:3.0:1788880490.111175:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880506 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880490.111178:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880490.111179:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880490.111190:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880490.111193:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880490.111198:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880490.111201:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880490.111206:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880490.111209:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880490.111216:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788880490.111219:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [237:237:254:4294967295] 192.168.204.111@tcp LPNI seq info [237:237:6:4294967295] 00000100:00000001:0.0:1788880490.111221:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880490.111228:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880490.111261:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880490.111267:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880490.111275:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880490.111275:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880490.111297:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880490.111306:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000800:00000200:1.0:1788880490.111315:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880490.111323:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880490.111327:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880490.111331:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880490.111335:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 23397 usecs (24 jiffies) 00000100:00000001:1.0:1788880490.111338:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306121241 00000100:00000001:1.0:1788880490.111341:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306126287 00000100:00000001:1.0:1788880490.111343:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306121241 00000100:00000001:1.0:1788880490.111345:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306126287 00000100:00000001:1.0:1788880490.111347:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306121241 00000100:00000001:1.0:1788880490.111348:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306126287 00000100:00000001:1.0:1788880490.111349:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306121241 00000100:00000001:1.0:1788880490.111358:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306126287 00000100:00000001:1.0:1788880490.111360:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880490.111363:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880490.111365:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880490.111372:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880490.111382:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880490.111386:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880490.111390:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880490.111391:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880490.111394:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880490.111396:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880490.111402:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880506 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880490.111404:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880490.111419:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880490.111422:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880490.111443:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880490.111445:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000131c1a55. 00000400:00000200:3.0:1788880490.111449:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880490.111453:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880490.111456:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000400:00000010:3.0:1788880490.111457:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:3.0:1788880490.111460:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880490.111462:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880490.111470:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880490.111472:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880490.111478:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880490.111479:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880490.111510:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880490.111512:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:3.0:1788880490.111515:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880490.111518:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880490.111521:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000400:00000010:3.0:1788880490.111522:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:3.0:1788880490.111524:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880490.111527:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880490.111534:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880490.111536:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880490.111555:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880490.111556:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880490.113368:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880490.113384:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880490.113388:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880490.113391:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880490.113401:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880490.113416:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b07180 00000400:00000200:1.0:1788880490.113498:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x83d [1] + 224 00000800:00000001:1.0:1788880490.113505:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880490.113509:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880490.113512:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880490.113515:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880490.113522:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880490.113528:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880490.113538:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880490.113564:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880490.113571:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880490.113574:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880490.113576:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880490.113578:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880490.113581:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880490.113597:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07200 00000400:00000200:1.0:1788880490.113602:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x845 [1] + 224 00000800:00000001:1.0:1788880490.113606:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880490.113609:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880490.113610:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880490.113613:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880490.113616:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880490.113619:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880490.113627:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880490.113644:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880490.113648:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880490.113651:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880490.113653:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880490.113655:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880490.113659:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880490.113664:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07280 00000400:00000200:1.0:1788880490.113669:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x84d [1] + 224 00000800:00000001:1.0:1788880490.113673:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880490.113680:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880490.113682:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880490.113686:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880490.113689:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880490.113689:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880490.113694:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:1.0:1788880490.113696:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880490.113702:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880490.113705:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:1.0:1788880490.113707:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880490.113708:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c1b65ba9. 00000400:00000200:0.0:1788880490.113715:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:0.0:1788880490.113724:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788880490.113728:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880490.113733:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880490.113734:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880490.113741:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880490.113744:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880490.113753:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880490.113754:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880490.113757:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880490.113761:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880490.113764:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880490.113767:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880490.113769:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880490.113773:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880490.113782:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=8 v=5 (5 5 5 0) 00000100:00001000:0.0:1788880490.113786:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=8 v=5 (1 1 1 0) 00000100:00000001:0.0:1788880490.113789:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880490.113790:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880490.113792:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880490.113795:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880490.113798:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880490.113800:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880490.113802:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880490.113803:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880490.113807:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880490.113814:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880490.113817:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880490.113819:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880490.113823:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880490.113830:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780661248:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880490.113837:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880490.113838:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880490.113843:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880490.113847:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000001:2.0:1788880490.113849:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:0.0:1788880490.113853:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780661248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788880490.113854:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880490.113857:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:2.0:1788880490.113864:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000100:00000001:0.0:1788880490.113866:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000010:2.0:1788880490.113868:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 02000000:00000001:0.0:1788880490.113870:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880490.113873:0:201286:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000010:0.0:1788880490.113874:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e10984e3. 02000000:00000001:0.0:1788880490.113879:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:2.0:1788880490.113880:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880490.113882:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880490.113885:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880490.113891:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880490.113894:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000252f4482. 00000100:00000200:2.0:1788880490.113899:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880490.113901:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880490.113904:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880490.113906:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:2.0:1788880490.113910:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880490.113912:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880490.113914:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880490.113915:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880490.113918:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788880490.113920:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880490.113922:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880490.113925:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880490.113926:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880490.113928:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788880490.113929:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880490.113931:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880490.113934:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=18 v=5 (5 5 5 0) 00000100:00000001:0.0:1788880490.113935:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880490.113938:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880490.113940:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=18 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880490.113944:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880490.113946:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880490.113947:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880490.113948:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880490.113950:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880490.113952:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880490.113957:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880490.113958:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880490.113959:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880490.113961:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880490.113974:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780661120:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880490.113988:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880490.113990:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880490.113992:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780661120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880490.113995:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880490.113996:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880490.113998:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000094e066a1. 02000000:00000001:2.0:1788880490.113999:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880490.114000:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880490.114001:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880490.114002:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880490.114003:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 02000000:00000001:2.0:1788880490.114005:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880490.114006:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880490.114006:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000001:2.0:1788880490.114008:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880490.114009:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880490.114016:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880490.114019:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880490.114020:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880490.114021:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880490.114022:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880490.114062:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880490.114064:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880490.114067:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880490.114068:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880490.114069:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:3.0:1788880490.114071:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:3.0:1788880490.114072:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:3.0:1788880490.114074:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880490.114076:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880490.114081:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880490.114085:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880490.114085:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880490.114086:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880490.114087:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880490.114088:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880490.114089:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880490.114090:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880490.114093:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=18 v=5 (5 5 5 0) 00000100:00001000:3.0:1788880490.114095:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=142 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880490.114097:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880490.114097:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880490.114098:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880490.114100:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880490.114104:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880490.114105:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880490.114106:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880490.114107:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880490.114107:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880490.114113:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880490.114117:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880490.114119:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880490.114124:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880490.114126:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880490.114130:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780661376:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880490.114132:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880490.114134:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880490.114135:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780661376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880506 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880490.114138:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880490.114139:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880490.114140:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000010c9710f. 02000000:00000001:3.0:1788880490.114142:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880490.114142:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880490.114143:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880490.114144:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880490.114145:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 02000000:00000001:3.0:1788880490.114147:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880490.114147:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880490.114148:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:3.0:1788880490.114150:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880490.114150:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880490.114151:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880490.114154:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880490.114155:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880490.114156:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880490.114157:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880490.174359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880490.174365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880490.174371:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880490.174372:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880490.174396:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880490.174398:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880490.174400:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880491.198363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880491.198370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880491.198376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880491.198377:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880491.198465:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880491.198469:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880491.198472:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880492.222303:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880492.222309:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880492.222315:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880492.222316:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880492.222337:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880492.222339:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880492.222342:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880493.246306:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880493.246318:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880493.246321:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880493.246325:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880493.246331:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880493.246333:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880493.246335:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880493.246337:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880493.246341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880493.246343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880493.246624:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880493.246627:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880493.246630:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880494.270352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880494.270359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880494.270366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880494.270368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880494.270387:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880494.270390:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880494.270392:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880495.230322:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306126287, at 4306126336 00000100:00000040:1.0:1788880495.230322:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306126287, at 4306126336 00000100:00000040:2.0:1788880495.230328:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306126287, at 4306126336 00000100:00000040:3.0:1788880495.230330:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880495.230339:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880495.230340:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fa7e66e6. 00000100:00000010:2.0:1788880495.230342:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045d7c07c. 00000100:00000010:1.0:1788880495.230344:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ad5555dd. 00000100:00000010:3.0:1788880495.230361:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000020:00000040:3.0:1788880495.230367:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880495.230371:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880495.230374:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880495.230379:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880495.230387:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:3.0:1788880495.230394:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880495.230405:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880495.230429:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880495.230431:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880495.230437:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_00+0] req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880495.230458:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880495.230461:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880495.230465:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880495.230469:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880495.230472:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880495.230475:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880495.230475:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880495.230477:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880495.230480:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880495.230480:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880495.230485:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:3.0:1788880495.230488:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000040:2.0:1788880495.230492:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880495.230494:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880495.230494:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880495.230498:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880495.230502:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880495.230512:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880495.230513:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880495.230516:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788880495.230518:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_01+1] req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880495.230520:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880495.230527:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880495.230534:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880495.230537:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880495.230538:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880495.230542:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880495.230547:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00100000:2.0:1788880495.230550:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780661504:192.168.204.111@tcp:400: 00000100:00000010:3.0:1788880495.230551:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880495.230554:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880495.230555:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880495.230557:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880495.230558:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880495.230560:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880495.230561:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880495.230563:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880495.230563:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880495.230565:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880495.230569:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880495.230569:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000040:1.0:1788880495.230571:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880495.230573:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880495.230575:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880495.230579:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000010:2.0:1788880495.230581:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000094e066a1. 02000000:00000001:2.0:1788880495.230586:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880495.230592:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880495.230593:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880495.230595:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 02000000:00000001:1.0:1788880495.230596:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880495.230600:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880495.230602:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880495.230604:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00100000:1.0:1788880495.230606:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780661632:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880495.230611:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000200:2.0:1788880495.230612:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780661504, portal 25 02000000:00000001:1.0:1788880495.230614:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880495.230617:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880495.230617:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880495.230620:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 02000000:00000001:1.0:1788880495.230621:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880495.230625:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000072dc05f6. 02000000:00000001:1.0:1788880495.230627:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880495.230630:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880511 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880495.230631:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005fdfe59c. 00000400:00000010:1.0:1788880495.230638:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e249bdf9. 00000100:00000200:1.0:1788880495.230642:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780661632, portal 10 00000100:00000001:1.0:1788880495.230646:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880495.230648:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000100:00000001:3.0:1788880495.230650:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880495.230653:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880495.230654:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880495.230658:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880495.230659:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_02+2] req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880495.230662:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880495.230668:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000006bff322d. 00000100:00000200:2.0:1788880495.230674:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780661504, offset 0 00000100:00000001:1.0:1788880495.230681:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:2.0:1788880495.230685:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880495.230686:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880495.230691:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000066e8f89d. 00000100:00000200:1.0:1788880495.230694:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780661632, offset 0 00000100:00000001:3.0:1788880495.230696:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880495.230698:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880495.230699:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:2.0:1788880495.230702:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:3.0:1788880495.230706:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:2.0:1788880495.230710:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880495.230710:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11464) 00000400:00000200:2.0:1788880495.230715:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880495.230720:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880495.230720:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880495.230721:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880495.230724:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:0.0:1788880495.230726:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880495.230726:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880495.230727:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880495.230732:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [238:238:256:4294967295] 192.168.204.111@tcp LPNI seq info [238:238:8:4294967295] 00000100:00000001:3.0:1788880495.230733:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880495.230736:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880495.230740:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880495.230743:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880495.230745:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880495.230749:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788880495.230750:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880495.230760:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000200:2.0:1788880495.230762:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880495.230763:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880495.230763:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880495.230769:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880495.230770:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:2.0:1788880495.230771:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003139f6ba. 02000000:00000001:3.0:1788880495.230778:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880495.230781:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880495.230782:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:3.0:1788880495.230785:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880495.230786:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880495.230792:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00100000:3.0:1788880495.230793:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780661760:192.168.204.111@tcp:400: 00000800:00000200:2.0:1788880495.230795:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880495.230798:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [239:239:255:4294967295] 192.168.204.111@tcp LPNI seq info [239:239:7:4294967295] 00000100:00000001:3.0:1788880495.230800:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880495.230803:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880495.230806:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880495.230808:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003139f6ba type 1, nob 320 niov 1 nkiov 1 02000000:00000001:3.0:1788880495.230811:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880495.230816:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000039f224f6. 00000400:00000200:1.0:1788880495.230817:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:3.0:1788880495.230821:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880495.230825:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880495.230826:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880495.230828:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009c9e5278. 00000800:00000010:1.0:1788880495.230829:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:2.0:1788880495.230832:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306131407 00000400:00000010:3.0:1788880495.230834:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000800:00000200:1.0:1788880495.230835:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:2.0:1788880495.230838:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045d7c07c. 00000100:00000200:3.0:1788880495.230841:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780661760, portal 10 00000800:00000200:1.0:1788880495.230843:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880495.230844:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880495.230846:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:1.0:1788880495.230847:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880495.230849:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000001:1.0:1788880495.230852:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880495.230852:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306131407 00000100:00000040:1.0:1788880495.230857:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 365792 usecs (366 jiffies) 00000100:00000010:2.0:1788880495.230858:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008b80730b. 00000100:00000001:1.0:1788880495.230862:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306126702 00000100:00000001:2.0:1788880495.230864:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880495.230866:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306131407 00000100:00000001:1.0:1788880495.230869:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306126702 00000100:00000001:2.0:1788880495.230870:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306131407 00000100:00000001:1.0:1788880495.230871:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306131407 00000800:00000200:0.0:1788880495.230874:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880495.230875:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306126702 00000100:00000010:2.0:1788880495.230875:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005fb7b615. 00000800:00000010:0.0:1788880495.230890:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003139f6ba. 00000100:00000001:2.0:1788880495.230892:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:0.0:1788880495.230897:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880495.230898:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306131407 00000100:00000001:2.0:1788880495.230898:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306131407 00000100:00000001:1.0:1788880495.230904:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306126702 00000100:00000001:2.0:1788880495.230905:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306126287 00000400:00000200:0.0:1788880495.230910:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880495.230911:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306131407 00000100:00000001:2.0:1788880495.230911:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880495.230916:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880495.230917:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880495.230918:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000006bff322d 00000100:00000001:1.0:1788880495.230919:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880495.230922:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880495.230922:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880495.230927:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880495.230927:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880495.230928:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000006bff322d. 00000100:00000001:1.0:1788880495.230935:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880495.230936:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880495.230937:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880495.230938:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880495.230939:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880495.230942:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880495.230943:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:0.0:1788880495.230944:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880511 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880495.230945:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880495.230947:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880495.230947:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880495.230950:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880495.230956:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880495.230957:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880495.230965:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880495.230967:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880495.230969:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880511 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880495.230985:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880495.230987:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880495.230994:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880495.230999:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880495.231002:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000003539bb5a. 00000100:00000200:3.0:1788880495.231006:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780661760, offset 0 00000800:00000200:0.0:1788880495.231028:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880495.231030:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:0.0:1788880495.231034:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880495.231039:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880495.231042:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000066e8f89d 00000400:00000010:0.0:1788880495.231044:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000066e8f89d. 00000100:00000001:0.0:1788880495.231048:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880495.231052:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880495.231060:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880495.231066:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:3.0:1788880495.231068:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:0.0:1788880495.231072:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880495.231074:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880495.231078:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880495.231084:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880495.231088:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880495.231091:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:3.0:1788880495.231125:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880495.231129:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [240:240:255:4294967295] 192.168.204.111@tcp LPNI seq info [240:240:7:4294967295] 00000400:00000200:3.0:1788880495.231136:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880495.231140:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880495.231143:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000800:00000200:3.0:1788880495.231148:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880495.231152:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880495.231160:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880495.231171:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880495.231174:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 23107 usecs (24 jiffies) 00000100:00000001:3.0:1788880495.231177:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306126360 00000100:00000001:3.0:1788880495.231179:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306131407 00000100:00000001:3.0:1788880495.231181:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306126360 00000100:00000001:3.0:1788880495.231183:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306131407 00000100:00000001:3.0:1788880495.231186:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306126360 00000100:00000001:3.0:1788880495.231188:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306131407 00000100:00000001:3.0:1788880495.231189:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306126360 00000100:00000001:3.0:1788880495.231191:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306131407 00000100:00000001:3.0:1788880495.231193:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880495.231196:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880495.231198:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880495.231200:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880495.231220:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880495.231223:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000400:00000200:0.0:1788880495.231228:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880495.231254:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880495.231257:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000003539bb5a 00000400:00000010:0.0:1788880495.231259:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000003539bb5a. 00000100:00000001:0.0:1788880495.231262:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880495.231265:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880495.231272:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880495.231281:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880495.231288:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880495.231291:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880495.232721:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880495.232728:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880495.232730:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880495.232732:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880495.232737:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880495.232745:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b07300 00000400:00000200:1.0:1788880495.232751:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x855 [1] + 224 00000800:00000001:1.0:1788880495.232756:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880495.232761:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880495.232763:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880495.232766:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880495.232771:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880495.232775:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880495.232784:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880495.232798:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880495.232803:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880495.232805:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880495.232820:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880495.232823:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880495.232827:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880495.232829:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880495.232830:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:2.0:1788880495.232834:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880495.232836:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880495.232839:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880495.232842:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880495.232863:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880495.232867:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880495.232868:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880495.232870:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880495.232872:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880495.232874:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880495.232876:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880495.232887:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880495.232893:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=23 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880495.232897:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=23 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880495.232899:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880495.232900:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880495.232903:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880495.232905:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880495.232907:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880495.232911:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880495.232917:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880495.232920:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880495.232922:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880495.232924:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880495.232931:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780661504:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880495.232935:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880495.232937:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880495.232939:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780661504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880495.232946:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880495.232949:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880495.232951:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000094e066a1. 02000000:00000001:2.0:1788880495.232959:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880495.232961:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880495.232962:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880495.232965:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880495.232967:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 02000000:00000001:2.0:1788880495.232971:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880495.232973:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880495.232974:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:2.0:1788880495.232977:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880495.232978:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880495.232980:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880495.232983:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880495.232985:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880495.232987:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880495.232988:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880495.233113:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880495.233120:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880495.233123:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880495.233125:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880495.233130:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880495.233138:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07400 00000400:00000200:0.0:1788880495.233144:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x865 [1] + 224 00000800:00000001:0.0:1788880495.233148:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880495.233151:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880495.233153:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880495.233155:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880495.233159:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880495.233162:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880495.233169:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880495.233183:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880495.233189:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880495.233193:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880495.233195:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880495.233197:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880495.233200:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880495.233202:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880495.233204:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880495.233208:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880495.233210:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880495.233211:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009c9e5278. 00000400:00000200:3.0:1788880495.233216:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00000400:00000010:3.0:1788880495.233219:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000400:00000200:0.0:1788880495.233220:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07380 00000100:00000001:3.0:1788880495.233224:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:0.0:1788880495.233229:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x859 [1] + 224 00000100:00000200:3.0:1788880495.233231:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880495.233260:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880495.233262:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880495.233269:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880495.233270:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880495.233272:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880495.233275:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880495.233276:0:201288:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:0.0:1788880495.233282:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:3.0:1788880495.233285:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880495.233287:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880495.233289:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880495.233294:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880495.233296:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:3.0:1788880495.233298:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=23 v=5 (5 5 5 0) 00000100:00000200:0.0:1788880495.233302:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880495.233304:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=147 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880495.233310:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880495.233312:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880495.233316:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880495.233324:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880495.233325:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880495.233343:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880495.233346:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788880495.233347:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880495.233348:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880495.233352:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880495.233354:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880495.233356:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:3.0:1788880495.233359:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880495.233361:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880495.233370:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880495.233372:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880495.233373:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880495.233375:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880495.233376:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880495.233380:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:3.0:1788880495.233382:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880495.233383:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880495.233385:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005fdfe59c. 00000400:00000200:1.0:1788880495.233390:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e249bdf9 00000400:00000010:1.0:1788880495.233392:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e249bdf9. 00000100:00000001:1.0:1788880495.233396:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:3.0:1788880495.233398:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780661760:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880495.233402:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880495.233406:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880495.233411:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:1.0:1788880495.233414:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880495.233420:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880495.233423:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780661760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880495.233427:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880495.233431:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880495.233432:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880495.233435:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880495.233439:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880495.233439:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880495.233442:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:1.0:1788880495.233443:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880495.233445:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000039f224f6. 00000100:00001000:1.0:1788880495.233449:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=13 v=5 (5 5 5 0) 02000000:00000001:3.0:1788880495.233450:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:1.0:1788880495.233455:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=13 v=5 (1 1 1 0) 00000020:00000001:3.0:1788880495.233458:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880495.233460:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000020:00000040:3.0:1788880495.233461:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880495.233463:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880495.233465:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:1.0:1788880495.233467:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000010:3.0:1788880495.233468:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 00010000:00000001:1.0:1788880495.233470:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880495.233472:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880495.233473:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:3.0:1788880495.233475:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:1.0:1788880495.233476:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880495.233479:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000010:3.0:1788880495.233479:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:1.0:1788880495.233481:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880495.233484:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:1.0:1788880495.233487:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880495.233487:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880495.233492:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880495.233498:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880495.233499:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880495.233500:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880495.233501:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880495.233502:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880495.233505:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880495.233507:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880495.233508:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880495.233517:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780661632:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880495.233521:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880495.233523:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880495.233526:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780661632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880511 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880495.233532:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880495.233533:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880495.233535:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000072dc05f6. 02000000:00000001:1.0:1788880495.233538:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880495.233539:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880495.233545:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880495.233547:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880495.233548:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 02000000:00000001:1.0:1788880495.233551:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880495.233552:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880495.233553:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:1.0:1788880495.233556:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880495.233558:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880495.233560:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880495.233564:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880495.233565:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880495.233567:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880495.233568:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880495.294310:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880495.294316:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880495.294321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880495.294322:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880495.294412:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880495.294416:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880495.294419:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880496.318371:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880496.318378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880496.318384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880496.318387:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880496.318390:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880496.318393:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880496.318395:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880496.318397:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880496.318398:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880496.318399:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880496.318426:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880496.318428:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880496.318429:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880497.342388:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880497.342396:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880497.342403:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880497.342404:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880497.342465:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880497.342469:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880497.342471:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880498.366329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880498.366336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880498.366341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880498.366342:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880498.366379:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880498.366382:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880498.366384:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880499.390382:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880499.390391:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880499.390400:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880499.390401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880499.390464:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880499.390471:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880499.390475:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880500.350352:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306131407, at 4306131456 00000100:00000040:1.0:1788880500.350353:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306131407, at 4306131456 00000100:00000040:3.0:1788880500.350357:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306131407, at 4306131456 00000100:00000010:2.0:1788880500.350370:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005fb7b615. 00000100:00000010:1.0:1788880500.350379:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008b80730b. 00000100:00000010:3.0:1788880500.350381:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fbf478f2. 00000100:00000040:0.0:1788880500.350440:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880500.350447:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880500.350455:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ff0be673. 00000020:00000040:0.0:1788880500.350460:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880500.350464:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880500.350467:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880500.350471:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880500.350478:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000ac392bfe. 00000100:00000001:0.0:1788880500.350485:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880500.350494:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880500.350504:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880500.350506:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880500.350512:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ff0be673] to pc [ptlrpcd_00_03+3] req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880500.350550:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880500.350552:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880500.350554:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880500.350556:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006b32fd51. 00000020:00000040:0.0:1788880500.350558:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880500.350559:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880500.350561:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880500.350562:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880500.350565:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000131c1a55. 00000100:00000001:0.0:1788880500.350568:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880500.350571:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880500.350575:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880500.350576:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880500.350578:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006b32fd51] to pc [ptlrpcd_00_00+0] req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880500.350579:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880500.350583:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880500.350584:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880500.350587:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880500.350588:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880500.350592:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880500.350600:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880500.350603:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880500.350603:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880500.350607:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880500.350607:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880500.350609:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:0.0:1788880500.350611:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000009ee4c6bf. 00000100:00100000:2.0:1788880500.350613:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000020:00000040:0.0:1788880500.350614:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:3.0:1788880500.350614:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880500.350615:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880500.350619:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880500.350620:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880500.350622:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880500.350623:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880500.350624:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:3.0:1788880500.350625:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880500.350626:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000030024c2c. 00000100:00000001:2.0:1788880500.350627:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880500.350631:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880500.350632:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880500.350632:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780661888:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880500.350636:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880500.350638:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880500.350639:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880500.350641:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880500.350645:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880500.350649:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880500.350653:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880500.350658:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880500.350659:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880500.350662:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880500.350664:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880500.350666:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880500.350666:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f8a9d8a8. 02000000:00000001:2.0:1788880500.350667:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880500.350669:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880500.350672:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880500.350673:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880500.350676:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000009ee4c6bf] to pc [ptlrpcd_00_01+1] req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880500.350680:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780662016:192.168.204.111@tcp:400: 00000400:00000010:3.0:1788880500.350682:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009c9e5278. 00000100:00000001:2.0:1788880500.350688:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880500.350691:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880500.350692:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880500.350692:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000100:00000040:0.0:1788880500.350695:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:2.0:1788880500.350698:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880500.350699:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11469) 02000000:00000001:2.0:1788880500.350701:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880500.350704:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880500.350707:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000016c67947. 00000100:00000200:3.0:1788880500.350708:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780661888, portal 25 02000000:00000001:2.0:1788880500.350710:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880500.350713:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880500.350716:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306131407, at 4306131456 00000400:00000010:2.0:1788880500.350716:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000100:00000001:3.0:1788880500.350716:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000400:00000010:2.0:1788880500.350723:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000010:0.0:1788880500.350725:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045d7c07c. 00000100:00000040:3.0:1788880500.350727:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880516 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880500.350729:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780662016, portal 10 00000100:00000001:2.0:1788880500.350734:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880500.350737:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849362304 : -126822860189312 : ffff8ca7c0781180) 00000100:00000001:3.0:1788880500.350744:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880500.350747:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880500.350750:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880500.350755:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e78d7264. 00000100:00000001:2.0:1788880500.350760:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:3.0:1788880500.350761:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780661888, offset 0 00000100:00000040:2.0:1788880500.350766:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880500.350769:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000400:00000200:3.0:1788880500.350772:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788880500.350780:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780662016, offset 0 00000400:00000200:3.0:1788880500.350798:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880500.350803:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880500.350805:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880500.350811:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880500.350815:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880500.350816:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880500.350820:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00100000:1.0:1788880500.350821:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:1.0:1788880500.350823:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880500.350827:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [241:241:256:4294967295] 192.168.204.111@tcp LPNI seq info [241:241:8:4294967295] 00000100:00000001:1.0:1788880500.350829:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880500.350832:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880500.350835:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880500.350836:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880500.350838:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880500.350844:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880500.350847:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880500.350859:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000200:3.0:1788880500.350860:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880500.350862:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880500.350862:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:3.0:1788880500.350866:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 02000000:00000001:1.0:1788880500.350868:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:2.0:1788880500.350871:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880500.350872:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1788880500.350876:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:1.0:1788880500.350877:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880500.350879:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00100000:1.0:1788880500.350884:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780662144:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880500.350885:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788880500.350886:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880500.350892:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880500.350893:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880500.350893:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:1.0:1788880500.350897:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880500.350901:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880500.350901:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [242:242:255:4294967295] 192.168.204.111@tcp LPNI seq info [242:242:7:4294967295] 02000000:00000001:1.0:1788880500.350904:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880500.350907:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880500.350911:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f26cffa0. 00000100:00000001:3.0:1788880500.350912:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306136527 02000000:00000001:1.0:1788880500.350916:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880500.350918:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:3.0:1788880500.350918:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002c2d3615. 00000400:00000010:1.0:1788880500.350922:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005fdfe59c. 00000100:00000001:3.0:1788880500.350924:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:2.0:1788880500.350928:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880500.350929:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e249bdf9. 00000100:00000001:3.0:1788880500.350930:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306136527 00000800:00000010:2.0:1788880500.350935:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000100:00000200:1.0:1788880500.350936:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780662144, portal 10 00000100:00000010:3.0:1788880500.350936:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002ad0ec0e. 00000100:00000001:1.0:1788880500.350941:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880500.350942:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:2.0:1788880500.350944:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880500.350945:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306136527 00000100:00000001:1.0:1788880500.350951:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849358720 : -126822860192896 : ffff8ca7c0780380) 00000800:00000200:2.0:1788880500.350951:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880500.350957:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000010:3.0:1788880500.350957:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000025d9193b. 00000100:00000040:1.0:1788880500.350963:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880500.350963:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880500.350964:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880500.350969:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306136527 00000100:00000040:2.0:1788880500.350970:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 360691 usecs (361 jiffies) 00000100:00000010:3.0:1788880500.350976:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000dba5f48d. 00000100:00000001:2.0:1788880500.350977:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306131817 00000100:00000001:1.0:1788880500.350981:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880500.350982:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000040:1.0:1788880500.350988:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880500.350988:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880500.350989:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306136527 00000400:00000010:1.0:1788880500.350994:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 00000100:00000001:3.0:1788880500.350994:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880500.350995:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306131817 00000100:00000001:2.0:1788880500.351000:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306136527 00000100:00000200:1.0:1788880500.351001:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780662144, offset 0 00000100:00000001:3.0:1788880500.351001:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880500.351004:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306131817 00000100:00000001:3.0:1788880500.351006:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880500.351010:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306136527 00000400:00000200:1.0:1788880500.351012:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880500.351014:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880500.351016:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306131817 00000100:00000001:3.0:1788880500.351017:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880500.351019:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306136527 00000400:00000200:1.0:1788880500.351022:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880500.351023:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880500.351023:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880500.351027:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880500.351029:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880500.351030:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880500.351031:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880500.351033:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880500.351036:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880500.351037:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880500.351037:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880500.351042:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880500.351044:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880500.351045:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880500.351049:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880500.351050:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880500.351052:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880500.351057:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [243:243:254:4294967295] 192.168.204.111@tcp LPNI seq info [243:243:6:4294967295] 00000400:00000200:1.0:1788880500.351067:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880500.351071:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880500.351075:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000800:00000200:1.0:1788880500.351080:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880500.351084:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880500.351087:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880500.351090:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880500.351092:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 22811 usecs (23 jiffies) 00000100:00000001:1.0:1788880500.351094:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306131479 00000100:00000001:1.0:1788880500.351096:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306136527 00000100:00000001:1.0:1788880500.351098:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306131479 00000100:00000001:1.0:1788880500.351099:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306136527 00000100:00000001:1.0:1788880500.351101:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306131479 00000100:00000001:1.0:1788880500.351103:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306136527 00000100:00000001:1.0:1788880500.351104:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306131479 00000100:00000001:1.0:1788880500.351112:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306136527 00000100:00000001:1.0:1788880500.351114:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880500.351116:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880500.351117:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880500.351119:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880500.351189:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880500.351192:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:0.0:1788880500.351196:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880500.351199:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880500.351203:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e78d7264 00000400:00000010:0.0:1788880500.351204:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e78d7264. 00000100:00000001:0.0:1788880500.351208:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880500.351210:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880516 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880500.351216:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880500.351218:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880516 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880500.351222:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880500.351223:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880500.351254:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880500.351256:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000400:00000200:0.0:1788880500.351259:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880500.351263:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880500.351265:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:0.0:1788880500.351266:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:0.0:1788880500.351269:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880500.351271:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880500.351286:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880500.351288:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880500.351292:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880500.351292:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880500.351348:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880500.351350:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:0.0:1788880500.351353:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880500.351355:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880500.351357:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000400:00000010:0.0:1788880500.351358:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000001:0.0:1788880500.351359:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880500.351361:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880500.351366:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880500.351367:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880500.351370:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880500.351371:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880500.353376:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880500.353388:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880500.353392:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880500.353395:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880500.353402:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880500.353414:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b07480 00000400:00000200:1.0:1788880500.353421:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x86d [1] + 224 00000800:00000001:1.0:1788880500.353426:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880500.353430:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880500.353432:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880500.353436:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880500.353440:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880500.353444:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880500.353453:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880500.353468:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880500.353474:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880500.353478:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880500.353479:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880500.353481:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880500.353485:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880500.353491:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07500 00000400:00000200:1.0:1788880500.353497:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x871 [1] + 224 00000800:00000001:1.0:1788880500.353501:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880500.353504:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880500.353506:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880500.353510:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880500.353515:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:3.0:1788880500.353515:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880500.353519:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:1.0:1788880500.353522:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:3.0:1788880500.353523:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009c9e5278. 00000400:00000200:3.0:1788880500.353528:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00000400:00000010:3.0:1788880500.353530:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000100:00000001:3.0:1788880500.353532:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880500.353534:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880500.353537:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880500.353539:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880500.353562:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880500.353565:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880500.353580:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880500.353583:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880500.353586:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880500.353588:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880500.353590:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880500.353591:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880500.353593:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880500.353596:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880500.353597:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880500.353602:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788880500.353602:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=28 v=5 (5 5 5 0) 00000800:00000001:1.0:1788880500.353604:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880500.353605:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880500.353610:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:3.0:1788880500.353615:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=28 v=5 (1 1 1 0) 00000400:00000200:1.0:1788880500.353619:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07580 00000100:00000001:3.0:1788880500.353621:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880500.353625:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880500.353625:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880500.353626:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x87d [1] + 224 00000100:00000001:2.0:1788880500.353629:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:3.0:1788880500.353631:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:1.0:1788880500.353632:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:2.0:1788880500.353634:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00010000:00000001:3.0:1788880500.353636:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788880500.353637:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:1.0:1788880500.353638:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880500.353640:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880500.353641:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:2.0:1788880500.353642:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:2.0:1788880500.353649:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000200:1.0:1788880500.353650:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:3.0:1788880500.353650:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880500.353654:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880500.353660:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880500.353663:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880500.353664:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788880500.353668:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880500.353669:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880500.353678:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880500.353681:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880500.353690:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880500.353695:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880500.353714:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880500.353722:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780661888:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880500.353727:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880500.353733:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880500.353734:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880500.353735:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880500.353739:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880500.353739:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:1.0:1788880500.353740:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880500.353744:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ff0be673 x1875776780661888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880500.353745:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880500.353749:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880500.353753:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880500.353756:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880500.353758:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880500.353761:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f8a9d8a8. 00000100:00000001:2.0:1788880500.353762:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880500.353764:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880500.353766:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880500.353767:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880500.353767:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880500.353773:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880500.353775:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:2.0:1788880500.353775:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=18 v=5 (5 5 5 0) 00000400:00000010:0.0:1788880500.353777:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005fdfe59c. 00000020:00000001:3.0:1788880500.353780:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880500.353792:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000200:0.0:1788880500.353794:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e249bdf9 00000100:00001000:2.0:1788880500.353796:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=18 v=5 (1 1 1 0) 00000020:00000001:3.0:1788880500.353796:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000010:0.0:1788880500.353799:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e249bdf9. 02000000:00000010:3.0:1788880500.353800:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000ac392bfe. 00000100:00000001:2.0:1788880500.353802:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880500.353805:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880500.353806:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880500.353807:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00010000:00000001:2.0:1788880500.353810:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:3.0:1788880500.353811:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000200:0.0:1788880500.353812:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880500.353815:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880500.353815:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ff0be673. 00000100:00000001:3.0:1788880500.353822:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880500.353823:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880500.353825:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880500.353827:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880500.353829:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880500.353830:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880500.353832:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880500.353836:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880500.353837:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880500.353843:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:2.0:1788880500.353845:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880500.353845:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880500.353850:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880500.353853:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880500.353859:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880500.353861:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880500.353862:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880500.353864:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880500.353868:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880500.353868:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880500.353872:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880500.353872:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880500.353878:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880500.353881:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880500.353882:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=28 v=5 (5 5 5 0) 00000100:00001000:0.0:1788880500.353887:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=152 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880500.353893:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880500.353895:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880500.353899:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00100000:2.0:1788880500.353903:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780662016:192.168.204.111@tcp:400: 00010000:00080000:0.0:1788880500.353904:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880500.353912:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880500.353913:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880500.353917:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880500.353917:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880500.353920:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880500.353922:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880500.353922:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880500.353925:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006b32fd51 x1875776780662016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880500.353927:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880500.353953:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880500.353953:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880500.353954:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880500.353957:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880500.353957:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:0.0:1788880500.353960:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880500.353961:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000016c67947. 02000000:00000001:2.0:1788880500.353968:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880500.353971:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00100000:0.0:1788880500.353972:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780662144:192.168.204.111@tcp:400: 00000020:00000040:2.0:1788880500.353972:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880500.353979:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880500.353980:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:2.0:1788880500.353982:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000131c1a55. 00000100:00000001:0.0:1788880500.353983:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880500.353988:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000009ee4c6bf x1875776780662144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880516 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880500.353988:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880500.353997:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880500.353998:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000010:2.0:1788880500.354000:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006b32fd51. 02000000:00000001:0.0:1788880500.354001:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880500.354005:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880500.354006:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880500.354008:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788880500.354011:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f26cffa0. 00000100:00000001:2.0:1788880500.354014:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1788880500.354016:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880500.354017:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880500.354020:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880500.354021:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:0.0:1788880500.354023:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880500.354024:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880500.354027:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880500.354029:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000030024c2c. 02000000:00000001:0.0:1788880500.354031:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880500.354032:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880500.354033:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000009ee4c6bf. 00000100:00000001:0.0:1788880500.354035:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880500.354036:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880500.354038:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880500.354041:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880500.354042:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880500.354043:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880500.354044:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880500.414631:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880500.414639:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880500.414645:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880500.414646:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880500.414682:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880500.414685:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880500.414687:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880501.438400:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880501.438407:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880501.438413:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880501.438414:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880501.438473:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880501.438480:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880501.438484:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880502.462405:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880502.462412:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880502.462420:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880502.462421:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880502.462450:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880502.462453:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880502.462456:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880503.486404:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880503.486413:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880503.486415:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880503.486418:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880503.486423:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880503.486426:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880503.486428:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880503.486435:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880503.486439:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880503.486441:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880503.486525:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880503.486529:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880503.486532:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880504.510406:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880504.510414:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880504.510422:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880504.510424:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880504.510456:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880504.510459:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880504.510461:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880505.470365:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306136527, at 4306136576 00000100:00000010:2.0:1788880505.470382:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000025d9193b. 00000100:00000040:3.0:1788880505.470478:0:184745:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306136527, at 4306136576 00000100:00000010:3.0:1788880505.470485:0:184745:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000dba5f48d. 00000100:00000040:1.0:1788880505.471861:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306136527, at 4306136577 00000100:00000010:1.0:1788880505.471880:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002ad0ec0e. 00000100:00000040:0.0:1788880505.472035:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306136527, at 4306136577 00000100:00000010:0.0:1788880505.472043:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002c2d3615. 00000100:00000040:3.0:1788880505.472070:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880505.472077:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880505.472090:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880505.472096:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880505.472101:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880505.472105:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880505.472113:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880505.472125:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:3.0:1788880505.472135:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880505.472150:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880505.472161:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880505.472165:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880505.472172:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_02+2] req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880505.472208:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880505.472211:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880505.472215:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880505.472218:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788880505.472221:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880505.472223:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880505.472225:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880505.472227:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880505.472231:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:2.0:1788880505.472257:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880505.472262:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880505.472264:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880505.472266:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880505.472267:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880505.472268:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880505.472272:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:2.0:1788880505.472278:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880505.472292:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880505.472294:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880505.472299:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880505.472302:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880505.472306:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880505.472310:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780662272:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880505.472317:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880505.472326:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788880505.472327:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880505.472329:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880505.472332:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880505.472336:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_03+3] req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880505.472338:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880505.472345:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000016c67947. 02000000:00000001:2.0:1788880505.472348:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880505.472353:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880505.472356:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000100:00000040:3.0:1788880505.472357:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880505.472363:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880505.472367:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000400:00000010:2.0:1788880505.472368:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000020:00000040:3.0:1788880505.472371:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880505.472375:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880505.472377:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000200:2.0:1788880505.472379:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780662272, portal 25 02000000:00000001:3.0:1788880505.472379:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880505.472381:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880505.472385:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880505.472386:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:3.0:1788880505.472387:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:2.0:1788880505.472388:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000001:0.0:1788880505.472390:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880505.472392:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880505.472398:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880505.472399:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880521 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880505.472400:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880505.472429:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880505.472431:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880505.472432:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880505.472435:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880505.472436:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880505.472439:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880505.472442:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880505.472447:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_00+0] req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880505.472455:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780662400:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880505.472462:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880505.472462:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880505.472466:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880505.472469:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880505.472470:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880505.472472:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:2.0:1788880505.472476:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000023cb0718. 02000000:00000010:0.0:1788880505.472477:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d653e4f6. 02000000:00000001:0.0:1788880505.472480:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880505.472482:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780662272, offset 0 00000400:00000010:0.0:1788880505.472484:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c1b65ba9. 00000100:00000001:3.0:1788880505.472488:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880505.472490:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000040:3.0:1788880505.472492:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:2.0:1788880505.472496:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880505.472499:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000200:0.0:1788880505.472504:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780662400, portal 10 00000100:00000040:3.0:1788880505.472506:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11474) 00000100:00000001:0.0:1788880505.472507:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880505.472508:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880505.472510:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000001:1.0:1788880505.472513:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880505.472517:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880505.472517:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:0.0:1788880505.472518:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880505.472526:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880505.472528:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880505.472529:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880505.472534:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880505.472537:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880505.472538:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:0.0:1788880505.472539:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000fbbca8a. 00000100:00000001:3.0:1788880505.472543:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880505.472544:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780662400, offset 0 00000400:00000200:2.0:1788880505.472547:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880505.472553:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:0.0:1788880505.472554:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880505.472556:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880505.472557:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880505.472562:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:2.0:1788880505.472564:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [244:244:256:4294967295] 192.168.204.111@tcp LPNI seq info [244:244:8:4294967295] 02000000:00000001:1.0:1788880505.472565:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880505.472570:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880505.472575:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780662528:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880505.472581:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880505.472585:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1788880505.472585:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:1.0:1788880505.472588:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880505.472592:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:2.0:1788880505.472595:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000010:1.0:1788880505.472597:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000cc126c4. 00000400:00000200:0.0:1788880505.472599:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788880505.472602:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880505.472604:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006aca3ff7. 00000400:00000200:0.0:1788880505.472605:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:1.0:1788880505.472608:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000066432675. 00000400:00000200:0.0:1788880505.472611:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:1.0:1788880505.472615:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000065ad0e62. 00000800:00000200:2.0:1788880505.472616:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880505.472617:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:1.0:1788880505.472621:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780662528, portal 10 00000400:00000200:0.0:1788880505.472624:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880505.472626:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880505.472629:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000400:00000200:0.0:1788880505.472632:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [245:245:255:4294967295] 192.168.204.111@tcp LPNI seq info [245:245:7:4294967295] 00000800:00000200:2.0:1788880505.472633:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:1.0:1788880505.472639:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880505.472641:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006aca3ff7 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880505.472651:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880505.472657:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880505.472664:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880505.472664:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880505.472665:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880505.472669:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000800:00000010:0.0:1788880505.472672:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000100:00000200:1.0:1788880505.472674:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780662528, offset 0 00000100:00000001:2.0:1788880505.472674:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306141649 00000800:00000200:0.0:1788880505.472680:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:2.0:1788880505.472683:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005fb7b615. 00000400:00000200:1.0:1788880505.472694:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880505.472700:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880505.472701:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:0.0:1788880505.472705:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880505.472705:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880505.472709:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306141649 00000400:00000200:1.0:1788880505.472710:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880505.472715:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880505.472717:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880505.472717:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003dccfd54. 00000400:00000200:1.0:1788880505.472720:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:0.0:1788880505.472724:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 355737 usecs (356 jiffies) 00000100:00000001:2.0:1788880505.472724:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:1.0:1788880505.472725:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880505.472730:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306136934 00000100:00000001:2.0:1788880505.472730:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306141649 00000400:00000200:1.0:1788880505.472733:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [246:246:254:4294967295] 192.168.204.111@tcp LPNI seq info [246:246:6:4294967295] 00000100:00000001:0.0:1788880505.472735:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306141649 00000100:00000010:2.0:1788880505.472736:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ccadc63b. 00000100:00000001:0.0:1788880505.472742:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306136934 00000100:00000001:2.0:1788880505.472742:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:3.0:1788880505.472747:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880505.472748:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306141649 00000100:00000001:2.0:1788880505.472749:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306141649 00000400:00000200:1.0:1788880505.472753:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000010:3.0:1788880505.472754:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006aca3ff7. 00000100:00000001:0.0:1788880505.472755:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306136934 00000100:00000010:2.0:1788880505.472755:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e91f0731. 00000100:00000001:0.0:1788880505.472760:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306141649 00000100:00000001:0.0:1788880505.472766:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306136934 00000400:00000200:3.0:1788880505.472766:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880505.472768:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:1.0:1788880505.472773:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880505.472774:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880505.472775:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306141649 00000100:00000001:2.0:1788880505.472780:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880505.472781:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:1.0:1788880505.472781:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000400:00000200:3.0:1788880505.472784:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880505.472787:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880505.472788:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880505.472792:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880505.472792:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000023cb0718 00000100:00000001:0.0:1788880505.472793:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880505.472794:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880505.472798:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880505.472798:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000023cb0718. 00000800:00000200:1.0:1788880505.472803:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880505.472807:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880505.472807:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880505.472807:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:1.0:1788880505.472808:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880505.472811:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880505.472811:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880505.472814:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880505.472815:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:3.0:1788880505.472816:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880521 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880505.472817:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880505.472818:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880505.472820:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 22533 usecs (23 jiffies) 00000100:00000001:0.0:1788880505.472821:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880505.472821:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880505.472825:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880505.472827:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.472827:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306136601 00000100:00000001:2.0:1788880505.472832:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.472834:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306141649 00000100:00000001:1.0:1788880505.472840:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306136601 00000100:00000001:1.0:1788880505.472844:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306141649 00000100:00000001:1.0:1788880505.472845:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306136601 00000100:00000001:1.0:1788880505.472847:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306141649 00000100:00000001:3.0:1788880505.472849:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880505.472850:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306136601 00000100:00000001:1.0:1788880505.472854:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306141649 00000100:00000040:3.0:1788880505.472856:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880521 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880505.472857:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880505.472863:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.472868:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.472872:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880505.472873:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880505.472877:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880505.472878:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880505.472881:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880505.472883:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880505.472885:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880505.472887:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.472889:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880505.472894:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880505.472897:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000400:00000200:3.0:1788880505.472902:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880505.472906:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880505.472909:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000fbbca8a 00000400:00000010:3.0:1788880505.472910:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000fbbca8a. 00000100:00000001:3.0:1788880505.472913:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880505.472915:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880505.472922:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880505.472924:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880505.472931:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880505.472932:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880505.473008:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880505.473011:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:3.0:1788880505.473014:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880505.473018:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880505.473020:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000400:00000010:3.0:1788880505.473022:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:3.0:1788880505.473024:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880505.473027:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880505.473039:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880505.473042:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880505.473048:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880505.473049:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880505.474567:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880505.474581:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880505.474586:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880505.474588:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880505.474597:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880505.474611:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b07600 00000400:00000200:0.0:1788880505.474620:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x885 [1] + 224 00000800:00000001:0.0:1788880505.474627:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880505.474633:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880505.474636:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880505.474639:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880505.474654:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880505.474658:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880505.474668:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880505.474683:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880505.474702:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880505.474706:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.474730:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880505.474734:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880505.474741:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880505.474742:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880505.474745:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:2.0:1788880505.474749:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880505.474750:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880505.474754:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880505.474759:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880505.474768:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880505.474775:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880505.474777:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.474780:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880505.474783:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880505.474786:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880505.474788:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880505.474791:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880505.474798:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=33 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880505.474802:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=33 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880505.474805:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880505.474807:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880505.474809:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880505.474812:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880505.474823:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880505.474828:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880505.474836:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880505.474840:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880505.474841:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880505.474846:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880505.474854:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780662272:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880505.474859:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.474861:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880505.474864:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780662272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880505.474871:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880505.474874:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880505.474877:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000016c67947. 02000000:00000001:2.0:1788880505.474883:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880505.474885:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880505.474888:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880505.474891:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880505.474893:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 02000000:00000001:2.0:1788880505.474898:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880505.474900:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880505.474902:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:2.0:1788880505.474906:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880505.474908:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.474911:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.474924:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880505.474926:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880505.474929:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880505.474932:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880505.474945:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880505.474963:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880505.474967:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880505.474969:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880505.474980:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880505.475014:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07680 00000400:00000200:1.0:1788880505.475022:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x88d [1] + 224 00000800:00000001:1.0:1788880505.475029:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880505.475037:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880505.475039:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880505.475044:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880505.475051:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880505.475057:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880505.475068:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880505.475084:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880505.475093:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880505.475095:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880505.475118:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880505.475129:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880505.475133:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880505.475135:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880505.475137:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c1b65ba9. 00000400:00000200:0.0:1788880505.475141:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:0.0:1788880505.475143:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788880505.475146:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880505.475151:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880505.475160:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880505.475167:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880505.475168:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880505.475170:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880505.475172:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880505.475173:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880505.475175:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880505.475177:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880505.475184:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=23 v=5 (5 5 5 0) 00000100:00001000:0.0:1788880505.475187:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=23 v=5 (1 1 1 0) 00000100:00000001:0.0:1788880505.475190:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880505.475191:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880505.475193:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880505.475196:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880505.475200:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880505.475209:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880505.475211:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880505.475213:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880505.475217:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880505.475222:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880505.475224:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880505.475226:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880505.475227:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880505.475267:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:0.0:1788880505.475273:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780662400:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880505.475280:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880505.475282:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880505.475285:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780662400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880505.475291:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880505.475293:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880505.475295:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d653e4f6. 02000000:00000001:0.0:1788880505.475298:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880505.475299:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880505.475300:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880505.475302:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880505.475304:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 02000000:00000001:0.0:1788880505.475308:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880505.475311:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880505.475312:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:0.0:1788880505.475316:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000800:00000001:3.0:1788880505.475316:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880505.475318:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880505.475322:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880505.475323:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880505.475328:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880505.475332:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880505.475336:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880505.475340:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880505.475344:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880505.475345:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880505.475359:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07700 00000400:00000200:3.0:1788880505.475369:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x895 [1] + 224 00000800:00000001:3.0:1788880505.475375:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880505.475382:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880505.475385:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880505.475388:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880505.475392:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880505.475397:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880505.475406:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880505.475418:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880505.475423:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880505.475424:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880505.475484:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880505.475488:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880505.475495:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880505.475496:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880505.475499:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000066432675. 00000400:00000200:1.0:1788880505.475503:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000065ad0e62 00000400:00000010:1.0:1788880505.475505:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000065ad0e62. 00000100:00000001:1.0:1788880505.475508:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880505.475516:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880505.475524:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880505.475529:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880505.475531:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880505.475533:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880505.475536:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880505.475538:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880505.475540:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880505.475543:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880505.475558:0:201285:0:(import.c:2149:obd_at_measure()) add 0 to 0000000028cb2d55 time=33 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880505.475560:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=7 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880505.475562:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880505.475564:0:201285:0:(client.c:1493:ptlrpc_check_status()) Process leaving (rc=18446744073709551509 : -107 : ffffffffffffff95) 00000100:00000001:1.0:1788880505.475567:0:201285:0:(recover.c:202:ptlrpc_request_handle_notconn()) Process entered 00000100:00080000:1.0:1788880505.475570:0:201285:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0001-mdc-ffff8ca787527000 of lustre-MDT0001_UUID@192.168.204.111@tcp abruptly disconnected: reconnecting 00000100:00080000:1.0:1788880505.475575:0:201285:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1788880505.475578:0:201285:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0001-mdc-ffff8ca787527000: Connection to lustre-MDT0001 (at 192.168.204.111@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00000001:1.0:1788880505.479805:0:201285:0:(obd_class.h:1223:obd_import_event()) Process entered 00000002:00000001:1.0:1788880505.479813:0:201285:0:(mdc_request.c:2879:mdc_import_event()) Process entered 00000002:00000001:1.0:1788880505.479815:0:201285:0:(mdc_request.c:2953:mdc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.479818:0:201285:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00000001:1.0:1788880505.479820:0:201285:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:1.0:1788880505.479823:0:201285:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:1.0:1788880505.479827:0:201285:0:(import.c:485:import_select_connection()) Process entered 00000100:00080000:1.0:1788880505.479833:0:201285:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 11162 00000400:00000200:1.0:1788880505.479843:0:201285:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000100:00000040:1.0:1788880505.479847:0:201285:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880505.479850:0:201285:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:1.0:1788880505.479852:0:201285:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880505.479855:0:201285:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000001:1.0:1788880505.479858:0:201285:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:1.0:1788880505.479860:0:201285:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:1.0:1788880505.479862:0:201285:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880505.479865:0:201285:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 00000020:00000001:1.0:1788880505.479867:0:201285:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000020:00000001:1.0:1788880505.479869:0:201285:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000100:00000040:1.0:1788880505.479871:0:201285:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880505.479873:0:201285:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:1.0:1788880505.479875:0:201285:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880505.479890:0:201285:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000040:1.0:1788880505.479893:0:201285:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 00000100:00080000:1.0:1788880505.479896:0:201285:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: import 00000000b98b8a4e using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000100:00000001:1.0:1788880505.479898:0:201285:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880505.479900:0:201285:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:1.0:1788880505.479908:0:201285:0:(sec.c:1628:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880505.479910:0:201285:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.479912:0:201285:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:1.0:1788880505.479915:0:201285:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 2097152, lost: 0. 00000008:00000001:1.0:1788880505.479918:0:201285:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.479919:0:201285:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880505.479925:0:201285:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 00000020:00000040:1.0:1788880505.479928:0:201285:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=5 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880505.479932:0:201285:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 02000000:00000001:1.0:1788880505.479933:0:201285:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.479936:0:201285:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880505.479938:0:201285:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880505.479942:0:201285:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880505.479947:0:201285:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 00000000fb6b8aba. 00000100:00000001:1.0:1788880505.479953:0:201285:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880505.479960:0:201285:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880505.479970:0:201285:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880505.479972:0:201285:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880505.479975:0:201285:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_rcv+-1] req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880505.479997:0:201285:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.480000:0:201285:0:(recover.c:221:ptlrpc_request_handle_notconn()) Process leaving 00000100:00000001:1.0:1788880505.480001:0:201285:0:(client.c:1701:after_reply()) Process leaving (rc=18446744073709551509 : -107 : ffffffffffffff95) 00000100:00000040:1.0:1788880505.480007:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Rpc:RNQU/200/0 rc -107/-107 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880505.480025:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551509 : -107 : 0xffffffffffffff95) 00000100:00000001:1.0:1788880505.480029:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880505.480030:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880505.480034:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Interpret:RNQU/200/0 rc -107/-107 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880505.480042:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780662528:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880505.480047:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880505.480048:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880505.480053:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880505.480053:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880505.480058:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780662528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880521 ref 1 fl Complete:RNQU/200/0 rc -107/-107 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880505.480058:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880505.480065:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880505.480069:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880505.480071:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000cc126c4. 00000100:00000040:2.0:1788880505.480073:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880505.480077:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880505.480080:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880505.480082:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880505.480087:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880505.480088:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000010:1.0:1788880505.480090:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 00000100:00000001:2.0:1788880505.480093:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880505.480096:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880505.480098:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880505.480099:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:2.0:1788880505.480101:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:1.0:1788880505.480102:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 02000000:00000001:2.0:1788880505.480105:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.480108:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880505.480111:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:2.0:1788880505.480111:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780662656:192.168.204.111@tcp:38: 00000100:00000001:1.0:1788880505.480115:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.480118:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880505.480123:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880505.480124:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788880505.480126:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.480127:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880505.480132:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880505.480132:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880505.480134:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880505.480139:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000de3147fa. 02000000:00000001:2.0:1788880505.480142:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880505.480148:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000400:00000010:2.0:1788880505.480154:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000200:2.0:1788880505.480160:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780662656, portal 10 00000100:00000001:2.0:1788880505.480162:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880505.480164:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 00000100:00000040:2.0:1788880505.480170:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880515 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880505.480177:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880505.480179:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880505.480181:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cc594c. 00000100:00000200:2.0:1788880505.480183:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780662656, offset 0 00000400:00000200:2.0:1788880505.480188:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880505.480198:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880505.480209:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880505.480213:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880505.480216:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880505.480219:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880505.480222:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [247:247:256:4294967295] 192.168.204.111@tcp LPNI seq info [247:247:8:4294967295] 00000400:00000200:2.0:1788880505.480271:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880505.480278:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880505.480283:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000007cb951ea. 00000800:00000200:2.0:1788880505.480290:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880505.480300:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880505.480303:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000007cb951ea type 1, nob 616 niov 1 nkiov 1 00000100:00000001:2.0:1788880505.480316:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880505.480320:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 22262 usecs (23 jiffies) 00000100:00000001:2.0:1788880505.480323:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306136609 00000100:00000001:2.0:1788880505.480325:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306141649 00000100:00000001:2.0:1788880505.480326:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306136609 00000100:00000001:2.0:1788880505.480328:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306141649 00000100:00000001:2.0:1788880505.480329:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306136609 00000100:00000001:2.0:1788880505.480331:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306141649 00000100:00000001:2.0:1788880505.480331:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306136609 00000100:00000001:2.0:1788880505.480332:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306141649 00000100:00000001:2.0:1788880505.480333:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880505.480335:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880505.480337:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880505.480338:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.480341:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880505.480342:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:2.0:1788880505.480344:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880505.480345:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880505.480345:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880505.480346:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880505.480432:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880505.480438:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000007cb951ea. 00000400:00000200:3.0:1788880505.480442:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880505.480447:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880505.480452:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cc594c 00000400:00000010:3.0:1788880505.480454:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cc594c. 00000100:00000001:3.0:1788880505.480458:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880505.480467:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880515 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880505.480476:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880505.480478:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880515 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880505.480484:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880505.480486:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880505.485040:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880505.485057:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880505.485062:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880505.485064:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880505.485090:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880505.485107:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07780 00000400:00000200:1.0:1788880505.485116:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x89d [1] + 0 00000800:00000001:1.0:1788880505.485123:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880505.485130:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880505.485133:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880505.485137:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880505.485145:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880505.485152:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880515 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880505.485163:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880515 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880505.485183:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880505.485191:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880505.485194:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.485227:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880505.485232:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880505.485263:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880505.485265:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880505.485274:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:2.0:1788880505.485277:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880505.485279:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880505.485283:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880505.485286:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880515 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880505.485294:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880515 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880505.485299:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880505.485301:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.485303:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880505.485306:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880505.485308:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880505.485310:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880505.485313:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880505.485322:0:201284:0:(import.c:2149:obd_at_measure()) add 0 to 0000000028cb2d55 time=33 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880505.485325:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=7 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880505.485328:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880505.485330:0:201284:0:(client.c:1493:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:2.0:1788880505.485332:0:201284:0:(client.c:1698:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:2.0:1788880505.485335:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880515 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880505.485340:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:2.0:1788880505.485343:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880505.485344:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.485347:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:2.0:1788880505.485349:0:201284:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:2.0:1788880505.485351:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788880505.485355:0:201284:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.111@tcp failed (-19) 00000100:00000001:2.0:1788880505.485357:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:2.0:1788880505.485360:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880515 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880505.485375:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780662656:192.168.204.111@tcp:38: 00000100:00000001:2.0:1788880505.485379:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.485382:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880505.485384:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780662656/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880515 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880505.485390:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880505.485392:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880505.485395:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000de3147fa. 02000000:00000001:2.0:1788880505.485398:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880505.485400:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880505.485401:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880505.485404:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880505.485406:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 00000000fb6b8aba. 02000000:00000001:2.0:1788880505.485410:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880505.485411:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880505.485412:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000001:2.0:1788880505.485416:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880505.485417:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.485419:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880505.485426:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880505.485427:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880505.485429:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880505.485431:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880505.534372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880505.534382:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880505.534388:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880505.534389:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880505.534473:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880505.534481:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880505.534487:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880506.558356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880506.558364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880506.558371:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880506.558374:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880506.558377:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880506.558380:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880506.558383:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880506.558386:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880506.558387:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880506.558389:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880506.558409:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880506.558412:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880506.558415:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880507.582458:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880507.582468:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880507.582502:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880507.582503:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880507.582617:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880507.582623:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880507.582626:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880508.606880:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880508.606888:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880508.606895:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880508.606896:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880508.606956:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880508.606963:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880508.606968:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880509.630338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880509.630344:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880509.630349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880509.630350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880509.630398:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880509.630403:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880509.630406:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880510.590348:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306141649, at 4306141696 00000100:00000010:2.0:1788880510.590366:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ccadc63b. 00000100:00000040:1.0:1788880510.590379:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306141649, at 4306141696 00000100:00000010:1.0:1788880510.590388:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003dccfd54. 00000100:00000040:0.0:1788880510.590887:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306141649, at 4306141696 00000100:00000010:0.0:1788880510.590894:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005fb7b615. 00000100:00000040:3.0:1788880510.590913:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306141649, at 4306141696 00000100:00000010:3.0:1788880510.590919:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e91f0731. 00000100:00000040:2.0:1788880510.590983:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880510.590988:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880510.590996:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 00000020:00000040:2.0:1788880510.591000:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880510.591003:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880510.591006:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880510.591009:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880510.591027:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000100:00000001:2.0:1788880510.591039:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880510.591050:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880510.591061:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880510.591063:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880510.591069:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_01+1] req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.591114:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880510.591118:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880510.591120:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880510.591125:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.591132:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880510.591134:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880510.591138:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880510.591140:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880510.591143:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880510.591146:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780662784:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880510.591150:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880510.591155:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880510.591168:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880510.591173:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880510.591178:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000065b4c69f. 02000000:00000001:3.0:1788880510.591180:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880510.591185:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009c9e5278. 00000400:00000010:3.0:1788880510.591191:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000100:00000200:3.0:1788880510.591197:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780662784, portal 25 00000100:00000001:3.0:1788880510.591199:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880510.591201:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000100:00000040:3.0:1788880510.591205:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.591224:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880510.591226:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880510.591229:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000200:3.0:1788880510.591249:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780662784, offset 0 00000400:00000200:3.0:1788880510.591256:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880510.591269:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880510.591275:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880510.591278:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880510.591282:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880510.591284:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880510.591288:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [248:248:256:4294967295] 192.168.204.111@tcp LPNI seq info [248:248:8:4294967295] 00000400:00000200:3.0:1788880510.591299:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880510.591304:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880510.591309:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000088d6bc1. 00000800:00000200:3.0:1788880510.591315:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880510.591324:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880510.591328:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000088d6bc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880510.591342:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880510.591346:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306146768 00000100:00000010:3.0:1788880510.591350:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000dba5f48d. 00000100:00000001:3.0:1788880510.591354:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880510.591358:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306146768 00000100:00000010:3.0:1788880510.591360:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000007dfb4281. 00000100:00000001:3.0:1788880510.591363:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880510.591366:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306146768 00000100:00000010:3.0:1788880510.591369:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d5dcfb55. 00000100:00000001:3.0:1788880510.591371:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880510.591373:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306146768 00000100:00000010:3.0:1788880510.591376:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000837757ae. 00000100:00000001:3.0:1788880510.591384:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880510.591387:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880510.591390:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880510.591393:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880510.591395:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880510.591437:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880510.591440:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880510.591444:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880510.591447:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000020:00000040:2.0:1788880510.591450:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880510.591453:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880510.591454:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000800:00000200:0.0:1788880510.591457:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 02000000:00000001:2.0:1788880510.591457:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880510.591462:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 00000800:00000010:0.0:1788880510.591463:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000088d6bc1. 00000100:00000001:1.0:1788880510.591467:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880510.591467:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880510.591470:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880510.591471:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880510.591475:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:0.0:1788880510.591483:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880510.591489:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880510.591491:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880510.591492:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000400:00000010:0.0:1788880510.591496:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000040:2.0:1788880510.591497:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_02+2] req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880510.591502:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880510.591509:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880510.591519:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880510.591523:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880510.591530:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.591531:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880510.591542:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:2.0:1788880510.591546:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788880510.591550:0:53635:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00000001:2.0:1788880510.591552:0:53635:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:2.0:1788880510.591554:0:53635:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:0.0:1788880510.591559:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880510.591559:0:53635:0:(import.c:485:import_select_connection()) Process entered 00000100:00100000:0.0:1788880510.591562:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:0.0:1788880510.591565:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00080000:2.0:1788880510.591570:0:53635:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 11469 00000100:00000001:0.0:1788880510.591574:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880510.591577:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880510.591578:0:53635:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000100:00000001:0.0:1788880510.591580:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880510.591583:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:2.0:1788880510.591584:0:53635:0:(import.c:2149:obd_at_measure()) add 10 to 0000000040f48a4f time=12 v=5 (1 1 1 0) 00000100:00001000:2.0:1788880510.591602:0:53635:0:(import.c:2215:obd_at_measure()) AT 0000000040f48a4f change: old=5 new=10 delta=5 (val=10) hist 10 1 1 0 00000100:00000001:0.0:1788880510.591606:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00080000:2.0:1788880510.591607:0:53635:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: tried all connections, increasing latency to 5s 00000100:00000040:0.0:1788880510.591614:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880510.591614:0:53635:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880510.591619:0:53635:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880510.591622:0:53635:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880510.591627:0:53635:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000001:0.0:1788880510.591633:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000020:00000001:2.0:1788880510.591633:0:53635:0:(genops.c:813:class_conn2export()) Process entered 00000100:00000001:0.0:1788880510.591637:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880510.591637:0:53635:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:2.0:1788880510.591642:0:53635:0:(lustre_handles.c:129:class_handle2object()) Process entered 02000000:00000001:0.0:1788880510.591643:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880510.591647:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000020:00000040:2.0:1788880510.591647:0:53635:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 00000100:00000001:1.0:1788880510.591648:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880510.591652:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880510.591653:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880510.591653:0:53635:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000100:00100000:0.0:1788880510.591659:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780662912:192.168.204.111@tcp:400: 00000020:00000001:2.0:1788880510.591659:0:53635:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000100:00000040:2.0:1788880510.591665:0:53635:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:0.0:1788880510.591667:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880510.591669:0:53635:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 02000000:00000001:0.0:1788880510.591672:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880510.591674:0:53635:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 02000000:00000001:0.0:1788880510.591675:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880510.591677:0:53635:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 02000000:00000001:0.0:1788880510.591679:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000020:00000040:2.0:1788880510.591681:0:53635:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 02000000:00000010:0.0:1788880510.591685:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000063883087. 00000100:00080000:2.0:1788880510.591686:0:53635:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: import 00000000b98b8a4e using connection 192.168.204.111@tcp/192.168.204.111@tcp 02000000:00000001:0.0:1788880510.591689:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880510.591691:0:53635:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880510.591694:0:53635:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 00000400:00000010:0.0:1788880510.591696:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c1b65ba9. 02000000:00000001:2.0:1788880510.591700:0:53635:0:(sec.c:1628:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880510.591701:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 02000000:00000001:2.0:1788880510.591706:0:53635:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880510.591708:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780662912, portal 10 00000100:00000001:2.0:1788880510.591710:0:53635:0:(obd_class.h:889:obd_reconnect()) Process entered 00000100:00000001:0.0:1788880510.591711:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880510.591714:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 00000008:00100000:2.0:1788880510.591716:0:53635:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 2097152, lost: 0. 00000008:00000001:2.0:1788880510.591720:0:53635:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880510.591722:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880510.591723:0:53635:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880510.591730:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002084a2b4. 00000020:00000040:2.0:1788880510.591734:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880510.591735:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880510.591738:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880510.591741:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e73139ca. 02000000:00000001:2.0:1788880510.591743:0:53635:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 00000100:00000200:0.0:1788880510.591745:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780662912, offset 0 02000000:00000001:2.0:1788880510.591746:0:53635:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880510.591751:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000200:0.0:1788880510.591754:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880510.591754:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880510.591758:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880510.591761:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000010:2.0:1788880510.591763:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 000000008d40d8dd. 00000400:00000200:0.0:1788880510.591765:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880510.591770:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880510.591770:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880510.591774:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880510.591780:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00100000:2.0:1788880510.591780:0:53635:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:0.0:1788880510.591787:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [249:249:256:4294967295] 192.168.204.111@tcp LPNI seq info [249:249:8:4294967295] 00000100:00000001:2.0:1788880510.591792:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880510.591793:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880510.591798:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002084a2b4] to pc [ptlrpcd_rcv+-1] req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880510.591813:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880510.591819:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880510.591823:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000a371c7dc. 00000100:00000001:2.0:1788880510.591823:0:53635:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880510.591827:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000200:0.0:1788880510.591829:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:2.0:1788880510.591834:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000800:00000200:0.0:1788880510.591839:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:2.0:1788880510.591842:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11479) 00000800:00000200:0.0:1788880510.591843:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000a371c7dc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880510.591848:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880510.591852:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880510.591854:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880510.591857:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880510.591861:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880510.591864:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 350912 usecs (351 jiffies) 00000100:00000001:0.0:1788880510.591869:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306142048 00000100:00000001:0.0:1788880510.591874:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306146768 00000100:00000001:1.0:1788880510.591877:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880510.591878:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306142048 00000100:00000001:1.0:1788880510.591879:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.591882:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306146768 02000000:00000001:1.0:1788880510.591883:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880510.591886:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880510.591888:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306142048 02000000:00000001:1.0:1788880510.591890:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.591892:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306146768 00000100:00000001:0.0:1788880510.591896:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306142048 00000100:00100000:1.0:1788880510.591896:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780663040:192.168.204.111@tcp:38: 00000100:00000001:0.0:1788880510.591901:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306146768 00000100:00000001:1.0:1788880510.591902:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880510.591905:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880510.591906:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880510.591909:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.591911:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.591913:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880510.591915:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880510.591918:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880510.591923:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000845fe3a7. 02000000:00000001:1.0:1788880510.591925:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880510.591929:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000066432675. 00000400:00000010:1.0:1788880510.591933:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000065ad0e62. 00000100:00000200:1.0:1788880510.591941:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780663040, portal 10 00000100:00000001:3.0:1788880510.591942:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880510.591945:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880510.591946:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880510.591950:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880510.591952:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880510.591953:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880510.591955:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938429184 : -126823771122432 : ffff8ca78a2c5500) 00000100:00000001:3.0:1788880510.591955:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880510.591968:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880525 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880510.591978:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880510.591981:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880510.591984:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000200:1.0:1788880510.591987:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780663040, offset 0 00000400:00000200:1.0:1788880510.591992:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880510.591998:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788880510.592001:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:1.0:1788880510.592002:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:3.0:1788880510.592006:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000a371c7dc. 00000400:00000200:1.0:1788880510.592007:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880510.592012:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880510.592013:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880510.592016:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880510.592023:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880510.592025:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [250:250:255:4294967295] 192.168.204.111@tcp LPNI seq info [250:250:7:4294967295] 00000400:00000200:1.0:1788880510.592034:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880510.592039:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880510.592040:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e73139ca 00000400:00000010:3.0:1788880510.592043:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e73139ca. 00000800:00000010:1.0:1788880510.592044:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000100:00000001:3.0:1788880510.592048:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:1.0:1788880510.592051:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:3.0:1788880510.592055:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880510.592061:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880510.592067:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 616 niov 1 nkiov 1 00000100:00000001:1.0:1788880510.592072:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880510.592074:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880510.592076:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 22040 usecs (23 jiffies) 00000100:00000040:3.0:1788880510.592079:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880510.592081:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306141720 00000100:00000001:1.0:1788880510.592086:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306146768 00000100:00000001:1.0:1788880510.592091:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306141720 00000100:00000001:1.0:1788880510.592095:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306146768 00000100:00000001:3.0:1788880510.592097:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880510.592099:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306141720 00000100:00000001:3.0:1788880510.592101:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880510.592108:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306146768 00000100:00000001:1.0:1788880510.592111:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306141720 00000100:00000001:1.0:1788880510.592113:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306146768 00000100:00000001:1.0:1788880510.592115:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880510.592117:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880510.592120:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880510.592121:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.592125:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880510.592127:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:1.0:1788880510.592129:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:3.0:1788880510.592130:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880510.592131:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880510.592134:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880510.592134:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000100:00000001:1.0:1788880510.592138:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880510.592270:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880510.592274:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880510.592277:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:3.0:1788880510.592279:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:3.0:1788880510.592282:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880510.592285:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880525 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.592303:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880510.592305:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880525 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.592311:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880510.592313:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880510.592982:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880510.592992:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880510.592995:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880510.592997:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880510.593003:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880510.593012:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b07800 00000400:00000200:1.0:1788880510.593019:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x8a5 [1] + 224 00000800:00000001:1.0:1788880510.593024:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880510.593029:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880510.593031:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880510.593034:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880510.593038:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880510.593041:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880510.593051:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.593132:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880510.593135:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880510.593156:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880510.593162:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880510.593165:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880510.593317:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880510.593319:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880510.593321:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009c9e5278. 00000400:00000200:3.0:1788880510.593324:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00000400:00000010:3.0:1788880510.593325:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000100:00000001:3.0:1788880510.593328:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880510.593331:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880510.593339:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.593344:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880510.593346:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880510.593348:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880510.593350:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880510.593352:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880510.593354:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880510.593356:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880510.593360:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=38 v=5 (5 5 5 0) 00000100:00001000:3.0:1788880510.593363:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=38 v=5 (1 1 1 0) 00000100:00000001:3.0:1788880510.593366:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880510.593367:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880510.593369:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880510.593372:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880510.593374:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880510.593380:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.593398:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880510.593401:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880510.593402:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880510.593405:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880510.593412:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780662784:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880510.593417:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880510.593418:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880510.593420:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780662784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.593426:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880510.593429:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880510.593431:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000065b4c69f. 02000000:00000001:3.0:1788880510.593433:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880510.593434:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880510.593436:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880510.593438:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880510.593440:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 02000000:00000001:3.0:1788880510.593443:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880510.593444:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880510.593446:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 00000100:00000001:3.0:1788880510.593449:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880510.593450:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880510.593452:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880510.593456:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880510.593457:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880510.593459:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880510.593460:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880510.595496:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880510.595506:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880510.595509:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880510.595511:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880510.595520:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880510.595533:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07880 00000400:00000200:0.0:1788880510.595541:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x8ad [1] + 224 00000800:00000001:0.0:1788880510.595546:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880510.595563:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880510.595565:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880510.595583:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880510.595599:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880510.595604:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880510.595614:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880510.595641:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880510.595649:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880510.595652:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880510.595744:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880510.595749:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880510.595755:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880510.595757:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880510.595759:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c1b65ba9. 00000400:00000200:2.0:1788880510.595763:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:2.0:1788880510.595766:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:2.0:1788880510.595769:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880510.595773:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880510.595783:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880510.595790:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880510.595792:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880510.595794:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880510.595803:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880510.595805:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880510.595807:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880510.595809:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880510.595816:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=28 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880510.595820:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=28 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880510.595823:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880510.595824:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880510.595826:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880510.595830:0:201288:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880510.595833:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880510.595835:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880510.595837:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880510.595838:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880510.595842:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880510.595850:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880510.595853:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880510.595854:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880510.595857:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880510.595864:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780662912:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880510.595868:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880510.595871:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880510.595873:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780662912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880526 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880510.595879:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880510.595880:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880510.595883:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000063883087. 02000000:00000001:2.0:1788880510.595886:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880510.595888:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880510.595889:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880510.595892:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880510.595900:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 02000000:00000001:2.0:1788880510.595903:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880510.595904:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880510.595906:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000100:00000001:2.0:1788880510.595909:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880510.595910:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880510.595912:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880510.595917:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880510.595918:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880510.595920:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880510.595922:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880510.600394:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880510.600417:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880510.600422:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880510.600425:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880510.600436:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880510.600453:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07900 00000400:00000200:0.0:1788880510.600462:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x8b5 [1] + 0 00000800:00000001:0.0:1788880510.600468:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880510.600475:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880510.600478:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880510.600482:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880510.600489:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880510.600495:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880525 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880510.600507:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880525 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880510.600537:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880510.600547:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880510.600550:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.600575:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880510.600580:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880510.600705:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880510.600707:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880510.600710:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000066432675. 00000400:00000200:1.0:1788880510.600714:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000065ad0e62 00000400:00000010:1.0:1788880510.600716:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000065ad0e62. 00000100:00000001:1.0:1788880510.600719:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880510.600723:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880525 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880510.600733:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880525 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880510.600739:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880510.600741:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.600743:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880510.600745:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880510.600747:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880510.600750:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880510.600752:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880510.600761:0:201284:0:(import.c:2149:obd_at_measure()) add 0 to 0000000028cb2d55 time=38 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880510.600764:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=12 v=10 (10 1 1 0) 00000100:00000001:1.0:1788880510.600767:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880510.600769:0:201284:0:(client.c:1493:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:1.0:1788880510.600771:0:201284:0:(client.c:1698:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:1.0:1788880510.600774:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880525 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880510.600781:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:1.0:1788880510.600784:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880510.600786:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.600787:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:1.0:1788880510.600789:0:201284:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:1.0:1788880510.600792:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788880510.600796:0:201284:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.111@tcp failed (-19) 00000100:00000001:1.0:1788880510.600810:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:1.0:1788880510.600813:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880525 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880510.600821:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780663040:192.168.204.111@tcp:38: 00000100:00000001:1.0:1788880510.600825:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.600828:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880510.600830:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002084a2b4 x1875776780663040/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880525 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880510.600836:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880510.600838:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880510.600840:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000845fe3a7. 02000000:00000001:1.0:1788880510.600843:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880510.600844:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880510.600846:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880510.600848:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880510.600850:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 000000008d40d8dd. 02000000:00000001:1.0:1788880510.600853:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880510.600855:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880510.600856:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002084a2b4. 00000100:00000001:1.0:1788880510.600859:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880510.600860:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.600862:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.600868:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880510.600870:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880510.600872:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880510.600873:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880510.654359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880510.654364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880510.654370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880510.654371:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880510.654413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880510.654431:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880510.654433:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000001:1.0:1788880510.963377:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880510.963395:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880510.963399:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880510.963402:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880510.963423:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880510.963437:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 328 into portal 15 MB=0x6aa025121e780 00000400:00000200:1.0:1788880510.963444:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index f from 12345-192.168.204.111@tcp of length 328/328 into md 0xd [2] + 984 00000800:00000001:1.0:1788880510.963450:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880510.963457:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880510.963459:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880510.963463:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880510.963470:0:201279:0:(events.c:286:request_in_callback()) Process entered 00000100:00000200:1.0:1788880510.963472:0:201279:0:(events.c:296:request_in_callback()) event type 2, status 0, service ldlm_cbd 00000100:00000010:1.0:1788880510.963478:0:201279:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b5bd1bf6. 00000100:00000040:1.0:1788880510.963482:0:201279:0:(events.c:337:request_in_callback()) incoming req@00000000b5bd1bf6 x1875776788096896 msgsize 328 00000100:00100000:1.0:1788880510.963486:0:201279:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.111@tcp (source: 12345-192.168.204.111@tcp) 00000100:00000001:1.0:1788880510.963502:0:201279:0:(events.c:373:request_in_callback()) Process leaving 00000800:00000001:1.0:1788880510.963510:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880510.963513:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880510.963559:0:203393:0:(service.c:2282:ptlrpc_server_handle_req_in()) Process entered 00000100:00100000:0.0:1788880510.963563:0:203393:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875776788096896 02000000:00000001:0.0:1788880510.963566:0:203393:0:(sec.c:2497:sptlrpc_svc_unwrap_request()) Process entered 00000100:00000001:0.0:1788880510.963576:0:203393:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880510.963578:0:203393:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880510.963592:0:203393:0:(sec.c:2546:sptlrpc_svc_unwrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880510.963595:0:203393:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875776788096896 00000020:00000001:0.0:1788880510.963598:0:203393:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:0.0:1788880510.963600:0:203393:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000001:0.0:1788880510.963602:0:203393:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880510.963605:0:203393:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=3 00000020:00000001:0.0:1788880510.963608:0:203393:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000020:00000001:0.0:1788880510.963610:0:203393:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000100:00000001:0.0:1788880510.963614:0:203393:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:0.0:1788880510.963619:0:203393:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 000000007e98b8cb. 00000100:00000001:0.0:1788880510.963621:0:203393:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:0.0:1788880510.963623:0:203393:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 000000007e98b8cb. 00000020:00000010:0.0:1788880510.963629:0:203393:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000fdf771c7. 00000020:00000010:0.0:1788880510.963634:0:203393:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 0000000038bba271. 00000020:00000010:0.0:1788880510.963638:0:203393:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 00000000d01614d5. 00000100:00000040:0.0:1788880510.963647:0:203393:0:(service.c:1432:ptlrpc_at_set_timer()) armed ldlm_cbd at +6s 00000100:00000001:0.0:1788880510.963650:0:203393:0:(service.c:2003:ptlrpc_server_request_add()) Process entered 00000100:00000001:0.0:1788880510.963651:0:203393:0:(service.c:1919:ptlrpc_server_hpreq_init()) Process entered 00000100:00000001:0.0:1788880510.963653:0:203393:0:(service.c:1949:ptlrpc_server_hpreq_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.963661:0:203393:0:(service.c:2101:ptlrpc_server_request_add()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.963674:0:203393:0:(service.c:2445:ptlrpc_server_handle_req_in()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880510.963684:0:203393:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 00000100:00000001:0.0:1788880510.963686:0:203393:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 00000100:00100000:0.0:1788880510.963691:0:203393:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.111@tcp, seq: 3 00000100:00000040:0.0:1788880510.963694:0:203393:0:(service.c:2255:ptlrpc_server_request_get()) RPC GETting export 00000000ea058c72 : new rpc_count 1 00000100:00000001:0.0:1788880510.963696:0:203393:0:(service.c:2257:ptlrpc_server_request_get()) Process leaving (rc=18446617249938059520 : -126823771492096 : ffff8ca78a26b100) 00000100:00000040:0.0:1788880510.963703:0:203393:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Interpret req@00000000b5bd1bf6 x1875776788096896/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:131/0 lens 328/0 e 0 to 0 dl 1788880521 ref 1 fl New:/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880510.963720:0:203393:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:0.0:1788880510.963723:0:203393:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 000000007e98b8cb. 00000100:00000001:0.0:1788880510.963725:0:203393:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:0.0:1788880510.963726:0:203393:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 000000007e98b8cb. 00000100:00100000:0.0:1788880510.963730:0:203393:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@00000000b5bd1bf6 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358207:x1875776788096896:12345-192.168.204.111@tcp:104: 00000100:00000200:0.0:1788880510.963734:0:203393:0:(service.c:2563:ptlrpc_server_handle_request()) got req 1875776788096896 00010000:00000001:0.0:1788880510.963736:0:203393:0:(ldlm_lockd.c:2360:ldlm_callback_handler()) Process entered 00010000:00000001:0.0:1788880510.963743:0:203393:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880510.963745:0:203393:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880510.963747:0:203393:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000d9c80304 refcount=2 00000020:00000001:0.0:1788880510.963749:0:203393:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000001:0.0:1788880510.963751:0:203393:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000001:0.0:1788880510.963755:0:203393:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:0.0:1788880510.963758:0:203393:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880510.963761:0:203393:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:0.0:1788880510.963770:0:203393:0:(pack_generic.c:296:lustre_pack_reply_v2()) Process entered 02000000:00000001:0.0:1788880510.963773:0:203393:0:(sec.c:2568:sptlrpc_svc_alloc_rs()) Process entered 02000000:00000010:0.0:1788880510.963777:0:203393:0:(sec_null.c:305:null_alloc_rs()) kmalloced '(rs)': 520 at 00000000c2821725. 02000000:00000001:0.0:1788880510.963779:0:203393:0:(sec.c:2605:sptlrpc_svc_alloc_rs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.963781:0:203393:0:(pack_generic.c:331:lustre_pack_reply_v2()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880510.963785:0:203393:0:(ldlm_lib.c:3317:target_pack_pool_reply()) Process entered 00010000:00000001:0.0:1788880510.963787:0:203393:0:(ldlm_lib.c:3327:target_pack_pool_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880510.963790:0:203393:0:(import.c:2149:obd_at_measure()) add 1 to 0000000013afe31a time=315 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880510.963794:0:203393:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:0.0:1788880510.963797:0:203393:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 13 to 192.168.204.111@tcp 00000100:00000001:0.0:1788880510.963799:0:203393:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 02000000:00000001:0.0:1788880510.963801:0:203393:0:(sec.c:2619:sptlrpc_svc_wrap_reply()) Process entered 02000000:00000001:0.0:1788880510.963802:0:203393:0:(sec.c:2630:sptlrpc_svc_wrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.963805:0:203393:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880510.963814:0:203393:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880510.963818:0:203393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002aa6ba25. 00000100:00000200:0.0:1788880510.963823:0:203393:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 16, xid 1875776788096896, offset 224 00000400:00000200:0.0:1788880510.963829:0:203393:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880510.963845:0:203393:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880510.963852:0:203393:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [251:251:256:4294967295] 192.168.204.111@tcp LPNI seq info [251:251:8:4294967295] 00000400:00000200:0.0:1788880510.963862:0:203393:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880510.963867:0:203393:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880510.963872:0:203393:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c93f60a2. 00000800:00000200:0.0:1788880510.963879:0:203393:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880510.963886:0:203393:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880510.963890:0:203393:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c93f60a2 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880510.963912:0:203393:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880510.963916:0:203393:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00010000:00000001:0.0:1788880510.963920:0:203393:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:0.0:1788880510.963923:0:203393:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at 000000004c515e6c. 00010000:00000001:0.0:1788880510.963927:0:203393:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:0.0:1788880510.963939:0:203393:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00010000:00000001:0.0:1788880510.963941:0:203393:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880510.963943:0:203393:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880510.963945:0:203393:0:(ldlm_lockd.c:2533:ldlm_callback_handler()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880510.963949:0:203393:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b5bd1bf6 x1875776788096896/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:131/0 lens 328/224 e 0 to 0 dl 1788880521 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788880510.963958:0:203393:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@00000000b5bd1bf6 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358207:x1875776788096896:12345-192.168.204.111@tcp:104: Request processed in 237us (474us total) trans 0 rc 0/0 00000100:00100000:0.0:1788880510.963967:0:203393:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.111@tcp, seq: 3 00000100:00000040:0.0:1788880510.963970:0:203393:0:(service.c:1113:ptlrpc_server_finish_active_request()) RPC PUTting export 00000000ea058c72 : new rpc_count 0 00000100:00000001:0.0:1788880510.963972:0:203393:0:(service.c:1955:ptlrpc_server_hpreq_fini()) Process entered 00000100:00000001:0.0:1788880510.963974:0:203393:0:(service.c:1966:ptlrpc_server_hpreq_fini()) Process leaving 00000020:00000010:0.0:1788880510.963978:0:203393:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 0000000038bba271. 00000020:00000001:2.0:1788880510.963981:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000800:00000200:3.0:1788880510.963983:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000020:00000001:2.0:1788880510.963986:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:0.0:1788880510.963989:0:203393:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 00000000d01614d5. 00000800:00000010:3.0:1788880510.963989:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c93f60a2. 00010000:00010000:2.0:1788880510.963990:0:203399:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000020:00000010:0.0:1788880510.963996:0:203393:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000fdf771c7. 00010000:00000001:2.0:1788880510.963996:0:203399:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00000400:00000200:3.0:1788880510.963996:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00010000:00000001:2.0:1788880510.963998:0:203399:0:(ldlm_lockd.c:1905:ldlm_handle_bl_callback()) Process entered 00010000:00010000:2.0:1788880510.964002:0:203399:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00000040:0.0:1788880510.964003:0:203393:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 2 00000400:00000200:3.0:1788880510.964007:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880510.964009:0:203393:0:(service.c:2635:ptlrpc_server_handle_request()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880510.964016:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002aa6ba25 00010000:00010000:2.0:1788880510.964017:0:203399:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock 00000000d9c80304 already unused, calling callback (00000000387296d6) 10000000:00000001:2.0:1788880510.964021:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:2.0:1788880510.964023:0:203399:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880510.964029:0:203399:0:(ldlm_request.c:1770:ldlm_cli_cancel()) Process entered 00010000:00000001:2.0:1788880510.964031:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:2.0:1788880510.964033:0:203399:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880510.964035:0:203399:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000d9c80304 refcount=3 00000020:00000001:2.0:1788880510.964037:0:203399:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000001:2.0:1788880510.964040:0:203399:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344576 : -126823429207040 : ffff8ca79e8d8c00) 00010000:00000001:2.0:1788880510.964043:0:203399:0:(ldlm_request.c:1386:ldlm_cli_cancel_local()) Process entered 00010000:00010000:2.0:1788880510.964044:0:203399:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00000001:2.0:1788880510.964050:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:2.0:1788880510.964058:0:203399:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788880510.964063:0:203399:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:00000001:2.0:1788880510.964066:0:203399:0:(mgc_request.c:719:mgc_requeue_add()) Process entered 10000000:00000040:2.0:1788880510.964067:0:203399:0:(mgc_request.c:723:mgc_requeue_add()) log lustre-sptlrpc: requeue (r=2 sp=0 st=1) 10000000:00000040:2.0:1788880510.964069:0:203399:0:(mgc_request.c:726:mgc_requeue_add()) lockh 0x0 00010000:00000001:2.0:1788880510.964071:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00010000:00000001:2.0:1788880510.964073:0:203399:0:(ldlm_lock.c:617:__ldlm_handle2lock()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880510.964077:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002aa6ba25. 00000100:00000001:3.0:1788880510.964082:0:201278:0:(events.c:386:reply_out_callback()) Process entered 00000100:00000001:3.0:1788880510.964084:0:201278:0:(events.c:398:reply_out_callback()) Process leaving 10000000:00000001:2.0:1788880510.964092:0:203399:0:(mgc_request.c:742:mgc_requeue_add()) Process leaving 10000000:00000001:2.0:1788880510.964094:0:203399:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:2.0:1788880510.964096:0:203399:0:(mgc_request.c:123:config_log_put()) log lustre-sptlrpc(000000007b8fd88a) refs 2 10000000:00000001:2.0:1788880510.964099:0:203399:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:2.0:1788880510.964101:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880510.964104:0:203399:0:(ldlm_lock.c:2502:ldlm_lock_cancel()) Process entered 00010000:00000001:2.0:1788880510.964107:0:203399:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:2.0:1788880510.964109:0:203399:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:2.0:1788880510.964112:0:203399:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:2.0:1788880510.964114:0:203399:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880510.964116:0:203399:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object 00000000d9c80304 with handle 0x805d548a36a886d6 from hash 00010000:00000001:2.0:1788880510.964119:0:203399:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:2.0:1788880510.964121:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880510.964123:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788880510.964125:0:203399:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:2.0:1788880510.964127:0:203399:0:(ldlm_lock.c:2538:ldlm_lock_cancel()) Process leaving 00010000:00000001:2.0:1788880510.964128:0:203399:0:(ldlm_request.c:1422:ldlm_cli_cancel_local()) Process leaving (rc=70368744177664 : 70368744177664 : 400000000000) 00010000:00000001:2.0:1788880510.964132:0:203399:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:2.0:1788880510.964135:0:203399:0:(ldlm_request.c:1527:ldlm_cli_cancel_req()) Process entered 00000100:00000010:2.0:1788880510.964140:0:203399:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000020:00000040:2.0:1788880510.964143:0:203399:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880510.964147:0:203399:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880510.964149:0:203399:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880510.964153:0:203399:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880510.964158:0:203399:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000b936126e. 00000100:00000001:2.0:1788880510.964163:0:203399:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880510.964165:0:203399:0:(ldlm_request.c:1461:ldlm_cancel_pack()) Process entered 00010000:00010000:2.0:1788880510.964173:0:203399:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788880510.964179:0:203399:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00000001:2.0:1788880510.964181:0:203399:0:(ldlm_request.c:1500:ldlm_cancel_pack()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880510.964184:0:203399:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880510.964186:0:203399:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880510.964192:0:203399:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_03+3] req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880510.964216:0:203399:0:(ldlm_request.c:1595:ldlm_cli_cancel_req()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:2.0:1788880510.964219:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880510.964220:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788880510.964221:0:203399:0:(ldlm_request.c:2504:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880510.964224:0:203399:0:(ldlm_request.c:1834:ldlm_cli_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880510.964225:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880510.964227:0:203399:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880510.964250:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788880510.964255:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:1.0:1788880510.964257:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.964261:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880510.964265:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788880510.964268:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00000100:00000001:1.0:1788880510.964269:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00010000:2.0:1788880510.964271:0:203399:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.111@tcp lock: 00000000d9c80304/0x805d548a36a886d6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7ac9 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880510.964272:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880510.964275:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880510.964283:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880510.964285:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:2.0:1788880510.964287:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788880510.964290:0:203399:0:(ldlm_lockd.c:1935:ldlm_handle_bl_callback()) Process leaving 00000800:00000001:3.0:1788880510.964292:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000010:2.0:1788880510.964294:0:203399:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at 000000004c515e6c. 00000800:00000001:3.0:1788880510.964295:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880510.964298:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000001:3.0:1788880510.964300:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.964301:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880510.964301:0:203399:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880510.964307:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 02000000:00000001:1.0:1788880510.964308:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:3.0:1788880510.964308:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000020:00000001:2.0:1788880510.964311:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880510.964312:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00010000:00010000:2.0:1788880510.964314:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 02000000:00000001:1.0:1788880510.964316:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.964318:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:2.0:1788880510.964318:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000400:00000200:3.0:1788880510.964318:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 328 into portal 15 MB=0x6aa025121e800 00000020:00000001:2.0:1788880510.964321:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.964324:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880510.964324:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780663168:192.168.204.111@tcp:103: 00010000:00010000:2.0:1788880510.964326:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000200:3.0:1788880510.964328:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index f from 12345-192.168.204.111@tcp of length 328/328 into md 0xd [2] + 1312 00000020:00000001:2.0:1788880510.964331:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880510.964334:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880510.964336:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000001:3.0:1788880510.964336:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00010000:2.0:1788880510.964338:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 02000000:00000001:1.0:1788880510.964343:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000800:00000001:3.0:1788880510.964343:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880510.964346:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880510.964348:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880510.964353:0:201280:0:(events.c:286:request_in_callback()) Process entered 02000000:00000001:1.0:1788880510.964354:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880510.964356:0:201280:0:(events.c:296:request_in_callback()) event type 2, status 0, service ldlm_cbd 02000000:00000001:1.0:1788880510.964359:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:3.0:1788880510.964361:0:201280:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 02000000:00000010:1.0:1788880510.964364:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000046cece22. 02000000:00000001:1.0:1788880510.964366:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880510.964371:0:201280:0:(events.c:337:request_in_callback()) incoming req@000000002a42fbe1 x1875776788097024 msgsize 328 00000400:00000010:1.0:1788880510.964374:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000066432675. 00000100:00100000:3.0:1788880510.964379:0:201280:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.111@tcp (source: 12345-192.168.204.111@tcp) 00000400:00000010:1.0:1788880510.964382:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000065ad0e62. 00000100:00000200:1.0:1788880510.964389:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780663168, portal 18 00000100:00000001:1.0:1788880510.964391:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880510.964392:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 00000100:00000001:3.0:1788880510.964394:0:201280:0:(events.c:373:request_in_callback()) Process leaving 00000100:00000040:1.0:1788880510.964399:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880510.964401:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880510.964405:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.964410:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880510.964413:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880510.964416:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000100:00000200:1.0:1788880510.964419:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 17, xid 1875776780663168, offset 0 00000400:00000200:1.0:1788880510.964424:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880510.964431:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880510.964435:0:203394:0:(service.c:2282:ptlrpc_server_handle_req_in()) Process entered 00000400:00000200:1.0:1788880510.964436:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:0.0:1788880510.964440:0:203394:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875776788097024 00000400:00000200:1.0:1788880510.964443:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880510.964444:0:203394:0:(sec.c:2497:sptlrpc_svc_unwrap_request()) Process entered 00000100:00000001:0.0:1788880510.964447:0:203394:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880510.964450:0:203394:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880510.964450:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1788880510.964456:0:203394:0:(sec.c:2546:sptlrpc_svc_unwrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880510.964456:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00100000:0.0:1788880510.964460:0:203394:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875776788097024 00000020:00000001:0.0:1788880510.964464:0:203394:0:(genops.c:813:class_conn2export()) Process entered 00000400:00000200:1.0:1788880510.964464:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [252:252:256:4294967295] 192.168.204.111@tcp LPNI seq info [252:252:8:4294967295] 00000020:00000040:0.0:1788880510.964466:0:203394:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000001:0.0:1788880510.964469:0:203394:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880510.964472:0:203394:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=3 00000020:00000001:0.0:1788880510.964476:0:203394:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000020:00000001:0.0:1788880510.964481:0:203394:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000400:00000200:1.0:1788880510.964481:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880510.964486:0:203394:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000800:00000200:1.0:1788880510.964490:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000020:00000010:0.0:1788880510.964491:0:203394:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 000000007e98b8cb. 00000100:00000001:0.0:1788880510.964494:0:203394:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:0.0:1788880510.964496:0:203394:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 000000007e98b8cb. 00000020:00000010:0.0:1788880510.964499:0:203394:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000fdf771c7. 00000020:00000010:0.0:1788880510.964504:0:203394:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 0000000038bba271. 00000800:00000010:1.0:1788880510.964506:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000020:00000010:0.0:1788880510.964507:0:203394:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 00000000d01614d5. 00000100:00000040:0.0:1788880510.964514:0:203394:0:(service.c:1432:ptlrpc_at_set_timer()) armed ldlm_cbd at +6s 00000800:00000200:1.0:1788880510.964514:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880510.964517:0:203394:0:(service.c:2003:ptlrpc_server_request_add()) Process entered 00000100:00000001:0.0:1788880510.964520:0:203394:0:(service.c:1919:ptlrpc_server_hpreq_init()) Process entered 00000100:00000001:0.0:1788880510.964522:0:203394:0:(service.c:1949:ptlrpc_server_hpreq_init()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880510.964523:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880510.964528:0:203394:0:(service.c:2101:ptlrpc_server_request_add()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880510.964528:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:0.0:1788880510.964541:0:203394:0:(service.c:2445:ptlrpc_server_handle_req_in()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.964545:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.964550:0:203394:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 00000100:00000001:1.0:1788880510.964551:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306147141 00000100:00000001:0.0:1788880510.964554:0:203394:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 00000100:00100000:0.0:1788880510.964560:0:203394:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.111@tcp, seq: 4 00000100:00000001:1.0:1788880510.964560:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306146768 00000100:00000040:0.0:1788880510.964572:0:203394:0:(service.c:2255:ptlrpc_server_request_get()) RPC GETting export 00000000ea058c72 : new rpc_count 1 00000100:00000001:0.0:1788880510.964575:0:203394:0:(service.c:2257:ptlrpc_server_request_get()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000100:00000040:0.0:1788880510.964590:0:203394:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Interpret req@000000002a42fbe1 x1875776788097024/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:131/0 lens 328/0 e 0 to 0 dl 1788880521 ref 1 fl New:/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880510.964593:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306147141 00000100:00000001:1.0:1788880510.964600:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306146768 00000100:00000001:0.0:1788880510.964604:0:203394:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000100:00000001:1.0:1788880510.964606:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306147141 00000020:00000010:0.0:1788880510.964609:0:203394:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 000000007e98b8cb. 00000100:00000001:1.0:1788880510.964611:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306146768 00000100:00000001:0.0:1788880510.964612:0:203394:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880510.964615:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306147141 00000020:00000010:0.0:1788880510.964617:0:203394:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 000000007e98b8cb. 00000100:00000001:1.0:1788880510.964621:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306146768 00000100:00100000:0.0:1788880510.964625:0:203394:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@000000002a42fbe1 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358207:x1875776788097024:12345-192.168.204.111@tcp:104: 00000100:00000001:1.0:1788880510.964629:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880510.964631:0:203394:0:(service.c:2563:ptlrpc_server_handle_request()) got req 1875776788097024 00000800:00000200:2.0:1788880510.964631:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00010000:00000001:0.0:1788880510.964635:0:203394:0:(ldlm_lockd.c:2360:ldlm_callback_handler()) Process entered 00000100:00000001:1.0:1788880510.964635:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880510.964639:0:203394:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000100:00000001:1.0:1788880510.964641:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880510.964641:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000020:00000001:0.0:1788880510.964642:0:203394:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880510.964646:0:203394:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=2 00000100:00000001:1.0:1788880510.964648:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880510.964649:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000020:00000001:0.0:1788880510.964650:0:203394:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:0.0:1788880510.964655:0:203394:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00000400:00000200:2.0:1788880510.964659:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00010000:00000001:0.0:1788880510.964660:0:203394:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:0.0:1788880510.964664:0:203394:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880510.964664:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000400:00000010:2.0:1788880510.964668:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00010000:00010000:0.0:1788880510.964669:0:203394:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:2.0:1788880510.964674:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880510.964679:0:203394:0:(pack_generic.c:296:lustre_pack_reply_v2()) Process entered 00000100:00000200:2.0:1788880510.964680:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880510.964683:0:203394:0:(sec.c:2568:sptlrpc_svc_alloc_rs()) Process entered 02000000:00000010:0.0:1788880510.964687:0:203394:0:(sec_null.c:305:null_alloc_rs()) kmalloced '(rs)': 520 at 000000008ca60ac8. 02000000:00000001:0.0:1788880510.964690:0:203394:0:(sec.c:2605:sptlrpc_svc_alloc_rs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.964694:0:203394:0:(pack_generic.c:331:lustre_pack_reply_v2()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880510.964695:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00010000:00000001:0.0:1788880510.964697:0:203394:0:(ldlm_lib.c:3317:target_pack_pool_reply()) Process entered 00010000:00000001:0.0:1788880510.964700:0:203394:0:(ldlm_lib.c:3327:target_pack_pool_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880510.964700:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880510.964705:0:203394:0:(import.c:2149:obd_at_measure()) add 1 to 0000000013afe31a time=15 v=5 (1 0 1 0) 00000100:00000001:0.0:1788880510.964713:0:203394:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000001:2.0:1788880510.964717:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880510.964718:0:203394:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 13 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880510.964720:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880510.964726:0:203394:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 02000000:00000001:0.0:1788880510.964729:0:203394:0:(sec.c:2619:sptlrpc_svc_wrap_reply()) Process entered 02000000:00000001:0.0:1788880510.964731:0:203394:0:(sec.c:2630:sptlrpc_svc_wrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.964733:0:203394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880510.964736:0:203394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880510.964738:0:203394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ff62fd67. 00000100:00000200:0.0:1788880510.964742:0:203394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 16, xid 1875776788097024, offset 224 00000400:00000200:0.0:1788880510.964746:0:203394:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880510.964752:0:203394:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880510.964756:0:203394:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [253:253:256:4294967295] 192.168.204.111@tcp LPNI seq info [253:253:8:4294967295] 00000400:00000200:0.0:1788880510.964763:0:203394:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880510.964767:0:203394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880510.964771:0:203394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000004e0f6846. 00000800:00000200:0.0:1788880510.964775:0:203394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880510.964779:0:203394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880510.964781:0:203394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000004e0f6846 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880510.964790:0:203394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880510.964792:0:203394:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00010000:00000001:0.0:1788880510.964795:0:203394:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:0.0:1788880510.964798:0:203394:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at 0000000011281a1b. 00010000:00000001:0.0:1788880510.964800:0:203394:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:0.0:1788880510.964809:0:203394:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00010000:00000001:0.0:1788880510.964811:0:203394:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880510.964812:0:203394:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880510.964814:0:203394:0:(ldlm_lockd.c:2533:ldlm_callback_handler()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880510.964819:0:203394:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776788097024/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:131/0 lens 328/224 e 0 to 0 dl 1788880521 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788880510.964828:0:203394:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@000000002a42fbe1 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358207:x1875776788097024:12345-192.168.204.111@tcp:104: Request processed in 223us (455us total) trans 0 rc 0/0 00000100:00100000:0.0:1788880510.964835:0:203394:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.111@tcp, seq: 4 00000020:00000001:1.0:1788880510.964839:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880510.964842:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880510.964845:0:203394:0:(service.c:1113:ptlrpc_server_finish_active_request()) RPC PUTting export 00000000ea058c72 : new rpc_count 0 00010000:00010000:1.0:1788880510.964846:0:203398:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00000001:0.0:1788880510.964851:0:203394:0:(service.c:1955:ptlrpc_server_hpreq_fini()) Process entered 00010000:00000001:1.0:1788880510.964851:0:203398:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00000100:00000001:0.0:1788880510.964854:0:203394:0:(service.c:1966:ptlrpc_server_hpreq_fini()) Process leaving 00010000:00000001:1.0:1788880510.964855:0:203398:0:(ldlm_lockd.c:1905:ldlm_handle_bl_callback()) Process entered 00000020:00000010:0.0:1788880510.964859:0:203394:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 0000000038bba271. 00010000:00010000:1.0:1788880510.964859:0:203398:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00000010:0.0:1788880510.964865:0:203394:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 00000000d01614d5. 00000800:00000200:3.0:1788880510.964869:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00010000:00010000:1.0:1788880510.964871:0:203398:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock 00000000408d8a8f already unused, calling callback (00000000387296d6) 00000020:00000010:0.0:1788880510.964872:0:203394:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000fdf771c7. 00000800:00000010:3.0:1788880510.964873:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000004e0f6846. 10000000:00000001:1.0:1788880510.964877:0:203398:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 00000020:00000040:0.0:1788880510.964879:0:203394:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 2 10000000:00010000:1.0:1788880510.964880:0:203398:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000400:00000200:3.0:1788880510.964881:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880510.964884:0:203394:0:(service.c:2635:ptlrpc_server_handle_request()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880510.964890:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00010000:00000001:1.0:1788880510.964896:0:203398:0:(ldlm_request.c:1770:ldlm_cli_cancel()) Process entered 00000400:00000200:3.0:1788880510.964897:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ff62fd67 00010000:00000001:1.0:1788880510.964898:0:203398:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880510.964901:0:203398:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000400:00000010:3.0:1788880510.964901:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ff62fd67. 00000020:00000040:1.0:1788880510.964905:0:203398:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=3 00000100:00000001:3.0:1788880510.964907:0:201278:0:(events.c:386:reply_out_callback()) Process entered 00000020:00000001:1.0:1788880510.964908:0:203398:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00000100:00000001:3.0:1788880510.964909:0:201278:0:(events.c:398:reply_out_callback()) Process leaving 00010000:00000001:1.0:1788880510.964912:0:203398:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880510.964915:0:203398:0:(ldlm_request.c:1386:ldlm_cli_cancel_local()) Process entered 00010000:00010000:1.0:1788880510.964917:0:203398:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00000001:1.0:1788880510.964928:0:203398:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:1.0:1788880510.964930:0:203398:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1788880510.964934:0:203398:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:00000001:1.0:1788880510.964937:0:203398:0:(mgc_request.c:719:mgc_requeue_add()) Process entered 10000000:00000040:1.0:1788880510.964938:0:203398:0:(mgc_request.c:723:mgc_requeue_add()) log lustre-client: requeue (r=2 sp=0 st=1) 10000000:00000040:1.0:1788880510.964939:0:203398:0:(mgc_request.c:726:mgc_requeue_add()) lockh 0x0 00010000:00000001:1.0:1788880510.964940:0:203398:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00010000:00000001:1.0:1788880510.964941:0:203398:0:(ldlm_lock.c:617:__ldlm_handle2lock()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880510.964950:0:203398:0:(mgc_request.c:742:mgc_requeue_add()) Process leaving 10000000:00000001:1.0:1788880510.964951:0:203398:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:1.0:1788880510.964952:0:203398:0:(mgc_request.c:123:config_log_put()) log lustre-client(00000000c2974351) refs 2 10000000:00000001:1.0:1788880510.964954:0:203398:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:1.0:1788880510.964955:0:203398:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880510.964956:0:203398:0:(ldlm_lock.c:2502:ldlm_lock_cancel()) Process entered 00010000:00000001:1.0:1788880510.964957:0:203398:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:1.0:1788880510.964958:0:203398:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:1.0:1788880510.964960:0:203398:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1788880510.964961:0:203398:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880510.964963:0:203398:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object 00000000408d8a8f with handle 0x805d548a36a886c8 from hash 00010000:00000001:1.0:1788880510.964965:0:203398:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:1.0:1788880510.964966:0:203398:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880510.964967:0:203398:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880510.964967:0:203398:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:1.0:1788880510.964968:0:203398:0:(ldlm_lock.c:2538:ldlm_lock_cancel()) Process leaving 00010000:00000001:1.0:1788880510.964969:0:203398:0:(ldlm_request.c:1422:ldlm_cli_cancel_local()) Process leaving (rc=70368744177664 : 70368744177664 : 400000000000) 00010000:00000001:1.0:1788880510.964971:0:203398:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:1.0:1788880510.964972:0:203398:0:(ldlm_request.c:1527:ldlm_cli_cancel_req()) Process entered 00000100:00000010:1.0:1788880510.964976:0:203398:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c27d4622. 00000020:00000040:1.0:1788880510.964978:0:203398:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880510.964980:0:203398:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880510.964981:0:203398:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880510.964984:0:203398:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880510.964991:0:203398:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000005578e134. 00000100:00000001:1.0:1788880510.964994:0:203398:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880510.964996:0:203398:0:(ldlm_request.c:1461:ldlm_cancel_pack()) Process entered 00010000:00010000:1.0:1788880510.964998:0:203398:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880510.965003:0:203398:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00000001:1.0:1788880510.965004:0:203398:0:(ldlm_request.c:1500:ldlm_cancel_pack()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.965007:0:203398:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880510.965008:0:203398:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880510.965013:0:203398:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c27d4622] to pc [ptlrpcd_00_00+0] req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880510.965052:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880510.965055:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880510.965057:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880510.965061:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880510.965069:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880510.965071:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880510.965073:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880510.965075:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880510.965078:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880510.965080:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780663296:192.168.204.111@tcp:103: 00000100:00000001:0.0:1788880510.965083:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880510.965086:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880510.965087:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880510.965089:0:203398:0:(ldlm_request.c:1595:ldlm_cli_cancel_req()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880510.965090:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880510.965094:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000fdf771c7. 00010000:00000001:1.0:1788880510.965094:0:203398:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880510.965096:0:203398:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 02000000:00000001:0.0:1788880510.965098:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880510.965099:0:203398:0:(ldlm_request.c:2504:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880510.965103:0:203398:0:(ldlm_request.c:1834:ldlm_cli_cancel()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880510.965104:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f9411509. 10000000:00000001:1.0:1788880510.965105:0:203398:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880510.965110:0:203398:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000400:00000010:0.0:1788880510.965111:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000200:0.0:1788880510.965117:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780663296, portal 18 00010000:00000001:1.0:1788880510.965119:0:203398:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00000100:00000001:0.0:1788880510.965121:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00010000:00010000:1.0:1788880510.965122:0:203398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886c8 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7ac2 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:0.0:1788880510.965124:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938049664 : -126823771501952 : ffff8ca78a268a80) 00000100:00000040:0.0:1788880510.965132:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000040:1.0:1788880510.965134:0:203398:0:(ldlm_resource.c:1704:ldlm_resource_putref()) putref res: 00000000b9a8b8b1 count: 0 00000100:00000001:0.0:1788880510.965142:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00010000:00000001:1.0:1788880510.965144:0:203398:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880510.965146:0:203398:0:(ldlm_lockd.c:1935:ldlm_handle_bl_callback()) Process leaving 00010000:00000010:1.0:1788880510.965148:0:203398:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at 0000000011281a1b. 00010000:00000001:1.0:1788880510.965152:0:203398:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880510.965154:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000020:00000001:1.0:1788880510.965156:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000400:00000010:0.0:1788880510.965158:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000096cf0c4a. 00000100:00000200:0.0:1788880510.965161:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 17, xid 1875776780663296, offset 0 00000100:00000001:3.0:1788880510.965163:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880510.965168:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000020:00000001:1.0:1788880510.965168:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880510.965169:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880510.965175:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:1.0:1788880510.965177:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000400:00000200:0.0:1788880510.965178:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880510.965182:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000020:00000001:1.0:1788880510.965185:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880510.965188:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00010000:00010000:1.0:1788880510.965189:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:1.0:1788880510.965191:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880510.965194:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880510.965195:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00010000:00010000:1.0:1788880510.965197:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000200:0.0:1788880510.965200:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880510.965204:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [254:254:256:4294967295] 192.168.204.111@tcp LPNI seq info [254:254:8:4294967295] 00000400:00000200:0.0:1788880510.965212:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880510.965217:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880510.965220:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000da26fd2f. 00000800:00000200:0.0:1788880510.965224:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880510.965228:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880510.965231:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000da26fd2f type 1, nob 424 niov 1 nkiov 1 00000100:00000001:0.0:1788880510.965271:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.965275:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306147141 00000100:00000001:0.0:1788880510.965277:0:201285:0:(niobuf.c:692:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, keep current deadline 4306147141 00000100:00000001:0.0:1788880510.965279:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306147141 00000100:00000001:0.0:1788880510.965281:0:201285:0:(niobuf.c:692:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, keep current deadline 4306147141 00000100:00000001:0.0:1788880510.965283:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306147141 00000100:00000001:0.0:1788880510.965285:0:201285:0:(niobuf.c:692:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, keep current deadline 4306147141 00000100:00000001:0.0:1788880510.965286:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306147141 00000100:00000001:0.0:1788880510.965288:0:201285:0:(niobuf.c:692:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, keep current deadline 4306147141 00000100:00000001:0.0:1788880510.965290:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880510.965292:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.965294:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.965296:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880510.965301:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880510.965308:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880510.965310:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880510.965311:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880510.965313:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.965315:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880510.965408:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880510.965411:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000da26fd2f. 00000400:00000200:3.0:1788880510.965414:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880510.965423:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880510.965426:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000096cf0c4a 00000400:00000010:3.0:1788880510.965427:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000096cf0c4a. 00000100:00000001:3.0:1788880510.965430:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880510.965433:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.965441:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880510.965443:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.965491:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880510.965493:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880510.967091:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880510.967100:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880510.967104:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880510.967106:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880510.967113:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880510.967124:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 18 MB=0x6aa0250b07980 00000400:00000200:2.0:1788880510.967130:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 12 from 12345-192.168.204.111@tcp of length 224/224 into md 0x8c1 [1] + 224 00000800:00000001:2.0:1788880510.967135:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880510.967141:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880510.967143:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880510.967146:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880510.967162:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880510.967166:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880510.967175:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880510.967189:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880510.967194:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880510.967207:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880510.967210:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880510.967213:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880510.967215:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880510.967218:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880510.967219:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000066432675. 00000400:00000200:1.0:1788880510.967222:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000065ad0e62 00000400:00000010:1.0:1788880510.967223:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000065ad0e62. 00000100:00000001:1.0:1788880510.967226:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880510.967229:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880510.967268:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880510.967274:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880510.967276:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.967278:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880510.967279:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880510.967281:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880510.967283:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880510.967286:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880510.967289:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000081111466 time=315 v=5 (5 0 0 0) 00000100:00001000:1.0:1788880510.967293:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=38 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880510.967295:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880510.967296:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880510.967298:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880510.967300:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880510.967302:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880510.967306:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880510.967312:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880510.967314:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880510.967321:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880510.967325:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880510.967331:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780663168:192.168.204.111@tcp:103: 00000100:00000001:1.0:1788880510.967336:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.967338:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880510.967340:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780663168/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880510.967353:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880510.967354:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880510.967356:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000046cece22. 02000000:00000001:1.0:1788880510.967359:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880510.967360:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880510.967361:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880510.967363:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880510.967364:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000b936126e. 02000000:00000001:1.0:1788880510.967367:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880510.967368:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880510.967369:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000100:00000001:1.0:1788880510.967372:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880510.967373:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.967374:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880510.967377:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880510.967378:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880510.967379:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880510.967380:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880510.968369:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880510.968376:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880510.968378:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880510.968380:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880510.968386:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880510.968393:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 18 MB=0x6aa0250b07a00 00000400:00000200:3.0:1788880510.968398:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 12 from 12345-192.168.204.111@tcp of length 224/224 into md 0x8cd [1] + 224 00000800:00000001:3.0:1788880510.968403:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880510.968434:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880510.968437:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880510.968441:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880510.968444:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880510.968448:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880510.968456:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880510.968468:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880510.968473:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880510.968476:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880510.968510:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880510.968513:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880510.968516:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880510.968518:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880510.968519:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000f9411509. 00000400:00000200:0.0:1788880510.968523:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000010:0.0:1788880510.968524:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880510.968528:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880510.968532:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880510.968545:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880510.968552:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880510.968553:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880510.968555:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880510.968557:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880510.968558:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880510.968560:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880510.968562:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880510.968565:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000081111466 time=15 v=5 (5 0 5 0) 00000100:00001000:0.0:1788880510.968568:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=38 v=5 (1 1 1 0) 00000100:00000001:0.0:1788880510.968570:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880510.968571:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880510.968573:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880510.968574:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.968590:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880510.968593:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880510.968599:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880510.968602:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880510.968603:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880510.968614:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880510.968621:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780663296:192.168.204.111@tcp:103: 00000100:00000001:0.0:1788880510.968625:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880510.968627:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880510.968630:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c27d4622 x1875776780663296/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880526 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880510.968636:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880510.968638:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880510.968640:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000fdf771c7. 02000000:00000001:0.0:1788880510.968642:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880510.968644:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880510.968645:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880510.968647:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880510.968649:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000005578e134. 02000000:00000001:0.0:1788880510.968651:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880510.968652:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880510.968653:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c27d4622. 00000100:00000001:0.0:1788880510.968656:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880510.968657:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880510.968659:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880510.968663:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880510.968664:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880510.968666:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880510.968668:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000010:1.1F:1788880510.970267:0:0:0:(ldlm_resource.c:1555:__ldlm_resource_free()) slab-freed '(res)': 256 at 00000000b9a8b8b1. 00010000:00000010:2.1F:1788880510.970273:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 00000000d9c80304. 00010000:00000010:1.1:1788880510.970282:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 00000000408d8a8f. 00000800:00000001:3.0:1788880511.089741:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880511.089754:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880511.089770:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880511.089771:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880511.089781:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880511.089805:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 328 into portal 15 MB=0x6aa025121fd00 00000400:00000200:3.0:1788880511.089811:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index f from 12345-192.168.204.111@tcp of length 328/328 into md 0xd [2] + 1640 00000800:00000001:3.0:1788880511.089817:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880511.089824:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880511.089826:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880511.089828:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880511.089834:0:201280:0:(events.c:286:request_in_callback()) Process entered 00000100:00000200:3.0:1788880511.089835:0:201280:0:(events.c:296:request_in_callback()) event type 2, status 0, service ldlm_cbd 00000100:00000010:3.0:1788880511.089840:0:201280:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000100:00000040:3.0:1788880511.089843:0:201280:0:(events.c:337:request_in_callback()) incoming req@00000000c1ccbaa0 x1875776788102400 msgsize 328 00000100:00100000:3.0:1788880511.089845:0:201280:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.111@tcp (source: 12345-192.168.204.111@tcp) 00000100:00000001:3.0:1788880511.089859:0:201280:0:(events.c:373:request_in_callback()) Process leaving 00000800:00000001:3.0:1788880511.089865:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880511.089866:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880511.089922:0:203393:0:(service.c:2282:ptlrpc_server_handle_req_in()) Process entered 00000100:00100000:0.0:1788880511.089926:0:203393:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875776788102400 02000000:00000001:0.0:1788880511.089929:0:203393:0:(sec.c:2497:sptlrpc_svc_unwrap_request()) Process entered 00000100:00000001:0.0:1788880511.089932:0:203393:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880511.089934:0:203393:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880511.089938:0:203393:0:(sec.c:2546:sptlrpc_svc_unwrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880511.089942:0:203393:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875776788102400 00000020:00000001:0.0:1788880511.089945:0:203393:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:0.0:1788880511.089947:0:203393:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000001:0.0:1788880511.089949:0:203393:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880511.089958:0:203393:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=3 00000020:00000001:0.0:1788880511.089961:0:203393:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000020:00000001:0.0:1788880511.089963:0:203393:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000100:00000001:0.0:1788880511.089968:0:203393:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:0.0:1788880511.089971:0:203393:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 000000007e98b8cb. 00000100:00000001:0.0:1788880511.089974:0:203393:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:0.0:1788880511.089976:0:203393:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 000000007e98b8cb. 00000020:00000010:0.0:1788880511.089982:0:203393:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000fdf771c7. 00000020:00000010:0.0:1788880511.089987:0:203393:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 0000000038bba271. 00000020:00000010:0.0:1788880511.089991:0:203393:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 00000000d01614d5. 00000100:00000040:0.0:1788880511.090000:0:203393:0:(service.c:1432:ptlrpc_at_set_timer()) armed ldlm_cbd at +6s 00000100:00000001:0.0:1788880511.090003:0:203393:0:(service.c:2003:ptlrpc_server_request_add()) Process entered 00000100:00000001:0.0:1788880511.090004:0:203393:0:(service.c:1919:ptlrpc_server_hpreq_init()) Process entered 00000100:00000001:0.0:1788880511.090006:0:203393:0:(service.c:1949:ptlrpc_server_hpreq_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880511.090013:0:203393:0:(service.c:2101:ptlrpc_server_request_add()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880511.090030:0:203393:0:(service.c:2445:ptlrpc_server_handle_req_in()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880511.090040:0:203393:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 00000100:00000001:0.0:1788880511.090042:0:203393:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 00000100:00100000:0.0:1788880511.090047:0:203393:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.111@tcp, seq: 5 00000100:00000040:0.0:1788880511.090050:0:203393:0:(service.c:2255:ptlrpc_server_request_get()) RPC GETting export 00000000ea058c72 : new rpc_count 1 00000100:00000001:0.0:1788880511.090052:0:203393:0:(service.c:2257:ptlrpc_server_request_get()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000100:00000040:0.0:1788880511.090057:0:203393:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Interpret req@00000000c1ccbaa0 x1875776788102400/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:132/0 lens 328/0 e 0 to 0 dl 1788880522 ref 1 fl New:/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880511.090067:0:203393:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:0.0:1788880511.090070:0:203393:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 000000007e98b8cb. 00000100:00000001:0.0:1788880511.090071:0:203393:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:0.0:1788880511.090073:0:203393:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 000000007e98b8cb. 00000100:00100000:0.0:1788880511.090077:0:203393:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@00000000c1ccbaa0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358231:x1875776788102400:12345-192.168.204.111@tcp:104: 00000100:00000200:0.0:1788880511.090080:0:203393:0:(service.c:2563:ptlrpc_server_handle_request()) got req 1875776788102400 00010000:00000001:0.0:1788880511.090083:0:203393:0:(ldlm_lockd.c:2360:ldlm_callback_handler()) Process entered 00010000:00000001:0.0:1788880511.090089:0:203393:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788880511.090096:0:203393:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:0.0:1788880511.090098:0:203393:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000000e3a19ac refcount=2 00000020:00000001:0.0:1788880511.090100:0:203393:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:0.0:1788880511.090103:0:203393:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00010000:00000001:0.0:1788880511.090108:0:203393:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:0.0:1788880511.090111:0:203393:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880511.090115:0:203393:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:0.0:1788880511.090125:0:203393:0:(pack_generic.c:296:lustre_pack_reply_v2()) Process entered 02000000:00000001:0.0:1788880511.090128:0:203393:0:(sec.c:2568:sptlrpc_svc_alloc_rs()) Process entered 02000000:00000010:0.0:1788880511.090131:0:203393:0:(sec_null.c:305:null_alloc_rs()) kmalloced '(rs)': 520 at 00000000556dc547. 02000000:00000001:0.0:1788880511.090133:0:203393:0:(sec.c:2605:sptlrpc_svc_alloc_rs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880511.090135:0:203393:0:(pack_generic.c:331:lustre_pack_reply_v2()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880511.090139:0:203393:0:(ldlm_lib.c:3317:target_pack_pool_reply()) Process entered 00010000:00000001:0.0:1788880511.090141:0:203393:0:(ldlm_lib.c:3327:target_pack_pool_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880511.090145:0:203393:0:(import.c:2149:obd_at_measure()) add 1 to 0000000013afe31a time=16 v=5 (1 0 1 0) 00000100:00000001:0.0:1788880511.090149:0:203393:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:0.0:1788880511.090152:0:203393:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 13 to 192.168.204.111@tcp 00000100:00000001:0.0:1788880511.090154:0:203393:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 02000000:00000001:0.0:1788880511.090166:0:203393:0:(sec.c:2619:sptlrpc_svc_wrap_reply()) Process entered 02000000:00000001:0.0:1788880511.090168:0:203393:0:(sec.c:2630:sptlrpc_svc_wrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880511.090171:0:203393:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880511.090173:0:203393:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880511.090178:0:203393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000200:0.0:1788880511.090182:0:203393:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 16, xid 1875776788102400, offset 224 00000400:00000200:0.0:1788880511.090190:0:203393:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880511.090199:0:203393:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880511.090204:0:203393:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [255:255:256:4294967295] 192.168.204.111@tcp LPNI seq info [255:255:8:4294967295] 00000400:00000200:0.0:1788880511.090218:0:203393:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880511.090223:0:203393:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880511.090228:0:203393:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000800:00000200:0.0:1788880511.090275:0:203393:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880511.090281:0:203393:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880511.090285:0:203393:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880511.090304:0:203393:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880511.090307:0:203393:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00010000:00000001:0.0:1788880511.090311:0:203393:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:0.0:1788880511.090314:0:203393:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at 00000000081730d7. 00010000:00000001:0.0:1788880511.090317:0:203393:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:0.0:1788880511.090341:0:203393:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00010000:00000001:0.0:1788880511.090344:0:203393:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880511.090346:0:203393:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880511.090348:0:203393:0:(ldlm_lockd.c:2533:ldlm_callback_handler()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880511.090353:0:203393:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776788102400/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:132/0 lens 328/224 e 0 to 0 dl 1788880522 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788880511.090362:0:203393:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@00000000c1ccbaa0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358231:x1875776788102400:12345-192.168.204.111@tcp:104: Request processed in 293us (517us total) trans 0 rc 0/0 00000020:00000001:3.0:1788880511.090365:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880511.090369:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880511.090372:0:203393:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.111@tcp, seq: 5 00010000:00010000:3.0:1788880511.090374:0:203399:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00000040:0.0:1788880511.090376:0:203393:0:(service.c:1113:ptlrpc_server_finish_active_request()) RPC PUTting export 00000000ea058c72 : new rpc_count 0 00010000:00000001:3.0:1788880511.090378:0:203399:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00000100:00000001:0.0:1788880511.090380:0:203393:0:(service.c:1955:ptlrpc_server_hpreq_fini()) Process entered 00010000:00000001:3.0:1788880511.090382:0:203399:0:(ldlm_lockd.c:1905:ldlm_handle_bl_callback()) Process entered 00000100:00000001:0.0:1788880511.090383:0:203393:0:(service.c:1966:ptlrpc_server_hpreq_fini()) Process leaving 00010000:00010000:3.0:1788880511.090386:0:203399:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00000010:0.0:1788880511.090388:0:203393:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 0000000038bba271. 00000020:00000010:0.0:1788880511.090393:0:203393:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 00000000d01614d5. 00000020:00000010:0.0:1788880511.090399:0:203393:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000fdf771c7. 00000020:00000040:0.0:1788880511.090403:0:203393:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 2 00010000:00010000:3.0:1788880511.090404:0:203399:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock 000000000e3a19ac already unused, calling callback (00000000387296d6) 00000100:00000001:0.0:1788880511.090406:0:203393:0:(service.c:2635:ptlrpc_server_handle_request()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:3.0:1788880511.090411:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:3.0:1788880511.090414:0:203399:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000200:2.0:1788880511.090420:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00010000:00000001:3.0:1788880511.090421:0:203399:0:(ldlm_request.c:1770:ldlm_cli_cancel()) Process entered 00010000:00000001:3.0:1788880511.090424:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000800:00000010:2.0:1788880511.090426:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000020:00000001:3.0:1788880511.090427:0:203399:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000400:00000200:2.0:1788880511.090432:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000020:00000040:3.0:1788880511.090437:0:203399:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000000e3a19ac refcount=3 00000400:00000200:2.0:1788880511.090439:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000020:00000001:3.0:1788880511.090442:0:203399:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00000400:00000200:2.0:1788880511.090445:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00010000:00000001:3.0:1788880511.090447:0:203399:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280347648 : -126823429203968 : ffff8ca79e8d9800) 00000400:00000010:2.0:1788880511.090448:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00010000:00000001:3.0:1788880511.090452:0:203399:0:(ldlm_request.c:1386:ldlm_cli_cancel_local()) Process entered 00000100:00000001:2.0:1788880511.090454:0:201279:0:(events.c:386:reply_out_callback()) Process entered 00010000:00010000:3.0:1788880511.090456:0:203399:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:2.0:1788880511.090457:0:201279:0:(events.c:398:reply_out_callback()) Process leaving 10000000:00000001:3.0:1788880511.090466:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:3.0:1788880511.090468:0:203399:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788880511.090475:0:203399:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 10000000:00000001:3.0:1788880511.090478:0:203399:0:(mgc_request.c:719:mgc_requeue_add()) Process entered 10000000:00000040:3.0:1788880511.090479:0:203399:0:(mgc_request.c:723:mgc_requeue_add()) log lustre-cliir: requeue (r=2 sp=0 st=3) 10000000:00000040:3.0:1788880511.090481:0:203399:0:(mgc_request.c:726:mgc_requeue_add()) lockh 0x0 00010000:00000001:3.0:1788880511.090483:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00010000:00000001:3.0:1788880511.090484:0:203399:0:(ldlm_lock.c:617:__ldlm_handle2lock()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:3.0:1788880511.090535:0:203399:0:(mgc_request.c:742:mgc_requeue_add()) Process leaving 10000000:00000001:3.0:1788880511.090537:0:203399:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:3.0:1788880511.090538:0:203399:0:(mgc_request.c:123:config_log_put()) log lustre-cliir(0000000008e74578) refs 2 10000000:00000001:3.0:1788880511.090541:0:203399:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:3.0:1788880511.090542:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880511.090545:0:203399:0:(ldlm_lock.c:2502:ldlm_lock_cancel()) Process entered 00010000:00000001:3.0:1788880511.090547:0:203399:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:3.0:1788880511.090549:0:203399:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:3.0:1788880511.090550:0:203399:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:3.0:1788880511.090552:0:203399:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880511.090554:0:203399:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object 000000000e3a19ac with handle 0x805d548a36a886c1 from hash 00010000:00000001:3.0:1788880511.090556:0:203399:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:3.0:1788880511.090558:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:3.0:1788880511.090559:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:3.0:1788880511.090560:0:203399:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:3.0:1788880511.090562:0:203399:0:(ldlm_lock.c:2538:ldlm_lock_cancel()) Process leaving 00010000:00000001:3.0:1788880511.090585:0:203399:0:(ldlm_request.c:1422:ldlm_cli_cancel_local()) Process leaving (rc=70368744177664 : 70368744177664 : 400000000000) 00010000:00000001:3.0:1788880511.090589:0:203399:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:3.0:1788880511.090596:0:203399:0:(ldlm_request.c:1527:ldlm_cli_cancel_req()) Process entered 00000100:00000010:3.0:1788880511.090601:0:203399:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880511.090604:0:203399:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880511.090607:0:203399:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880511.090609:0:203399:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880511.090613:0:203399:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880511.090617:0:203399:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000000cc126c4. 00000100:00000001:3.0:1788880511.090622:0:203399:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880511.090624:0:203399:0:(ldlm_request.c:1461:ldlm_cancel_pack()) Process entered 00010000:00010000:3.0:1788880511.090627:0:203399:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788880511.090642:0:203399:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00000001:3.0:1788880511.090644:0:203399:0:(ldlm_request.c:1500:ldlm_cancel_pack()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880511.090650:0:203399:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880511.090655:0:203399:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880511.090660:0:203399:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_01+1] req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880511.090688:0:203399:0:(ldlm_request.c:1595:ldlm_cli_cancel_req()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:3.0:1788880511.090691:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:3.0:1788880511.090692:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:3.0:1788880511.090693:0:203399:0:(ldlm_request.c:2504:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880511.090695:0:203399:0:(ldlm_request.c:1834:ldlm_cli_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:3.0:1788880511.090696:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880511.090698:0:203399:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:3.0:1788880511.090704:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00010000:3.0:1788880511.090705:0:203399:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.111@tcp lock: 000000000e3a19ac/0x805d548a36a886c1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7abb expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000040:3.0:1788880511.090713:0:203399:0:(ldlm_resource.c:1704:ldlm_resource_putref()) putref res: 00000000035e4197 count: 0 00000100:00000001:2.0:1788880511.090718:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880511.090722:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880511.090724:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880511.090726:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:3.0:1788880511.090726:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000100:00000040:2.0:1788880511.090730:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880511.090731:0:203399:0:(ldlm_lockd.c:1935:ldlm_handle_bl_callback()) Process leaving 00000100:00000001:0.0:1788880511.090732:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000010:3.0:1788880511.090739:0:203399:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at 00000000081730d7. 00000100:00000001:2.0:1788880511.090745:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00010000:00000001:3.0:1788880511.090746:0:203399:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880511.090748:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880511.090750:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880511.090753:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880511.090754:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00010000:00010000:3.0:1788880511.090755:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880511.090758:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 02000000:00000001:2.0:1788880511.090759:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000020:00000001:3.0:1788880511.090760:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880511.090763:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880511.090764:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788880511.090768:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780663424:192.168.204.111@tcp:103: 00000020:00000001:3.0:1788880511.090768:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880511.090770:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880511.090775:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:2.0:1788880511.090776:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880511.090783:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880511.090785:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880511.090788:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880511.090792:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ca284ac2. 02000000:00000001:2.0:1788880511.090793:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880511.090798:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000400:00000010:2.0:1788880511.090802:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000200:2.0:1788880511.090806:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780663424, portal 18 00000100:00000001:2.0:1788880511.090809:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880511.090810:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000040:2.0:1788880511.090814:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880527 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880511.090821:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880511.090828:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880511.090831:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001e963647. 00000100:00000200:2.0:1788880511.090833:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 17, xid 1875776780663424, offset 0 00000400:00000200:2.0:1788880511.090837:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880511.090843:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880511.090847:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880511.090851:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880511.090854:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880511.090856:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880511.090859:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [256:256:256:4294967295] 192.168.204.111@tcp LPNI seq info [256:256:8:4294967295] 00000400:00000200:2.0:1788880511.090866:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880511.090870:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880511.090872:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 00000800:00000200:2.0:1788880511.090875:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880511.090878:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880511.090880:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:2.0:1788880511.090935:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880511.090940:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306147267 00000100:00000001:2.0:1788880511.090946:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306147141 00000100:00000001:2.0:1788880511.090961:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306147267 00000100:00000001:2.0:1788880511.090964:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306147141 00000100:00000001:2.0:1788880511.090970:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306147267 00000100:00000001:2.0:1788880511.090973:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306147141 00000100:00000001:2.0:1788880511.090975:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306147267 00000100:00000001:2.0:1788880511.090977:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306147141 00000100:00000001:2.0:1788880511.090979:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880511.090982:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880511.090984:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880511.090986:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880511.090996:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880511.090997:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880511.090999:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880511.091001:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880511.091002:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880511.091004:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880511.091006:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880511.091009:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000400:00000200:3.0:1788880511.091012:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880511.091016:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880511.091018:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001e963647 00000400:00000010:3.0:1788880511.091020:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001e963647. 00000100:00000001:3.0:1788880511.091023:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880511.091026:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880527 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880511.091036:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880511.091038:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880527 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880511.091045:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880511.091046:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880511.093625:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880511.093649:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880511.093653:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880511.093656:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880511.093665:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880511.093678:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 18 MB=0x6aa0250b07a80 00000400:00000200:3.0:1788880511.093683:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 12 from 12345-192.168.204.111@tcp of length 224/224 into md 0x8d9 [1] + 224 00000800:00000001:3.0:1788880511.093688:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880511.093693:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880511.093705:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880511.093710:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880511.093716:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880511.093722:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880527 ref 1 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880511.093747:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880527 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880511.093763:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880511.093770:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880511.093773:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880511.093806:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880511.093811:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880511.093817:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880511.093818:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880511.093820:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:2.0:1788880511.093824:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880511.093826:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880511.093830:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880511.093834:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880527 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880511.093843:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880527 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880511.093849:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880511.093851:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880511.093853:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880511.093856:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880511.093858:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880511.093860:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880511.093862:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880511.093868:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000081111466 time=16 v=5 (5 0 5 0) 00000100:00001000:2.0:1788880511.093871:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=39 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880511.093874:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880511.093875:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880511.093877:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880511.093879:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880511.093881:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880511.093884:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880527 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880511.093890:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880511.093893:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880511.093900:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880511.093904:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880527 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880511.093911:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780663424:192.168.204.111@tcp:103: 00000100:00000001:2.0:1788880511.093916:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880511.093919:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880511.093921:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780663424/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880527 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880511.093927:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880511.093929:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880511.093931:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ca284ac2. 02000000:00000001:2.0:1788880511.093935:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880511.093937:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880511.093939:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880511.093941:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880511.093943:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000000cc126c4. 02000000:00000001:2.0:1788880511.093946:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880511.093947:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880511.093949:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:2.0:1788880511.093952:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880511.093953:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880511.093955:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880511.093960:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880511.093962:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880511.093964:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880511.093966:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000010:3.1:1788880511.096278:0:0:0:(ldlm_resource.c:1555:__ldlm_resource_free()) slab-freed '(res)': 256 at 00000000035e4197. 00010000:00000010:3.1:1788880511.096291:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 000000000e3a19ac. 00010000:00000001:1.0:1788880511.678354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880511.678363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880511.678370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880511.678371:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880511.678400:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880511.678402:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880511.678405:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880512.702345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880512.702352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880512.702372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880512.702373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880512.702425:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880512.702428:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880512.702430:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880513.726368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880513.726381:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880513.726384:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880513.726387:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880513.726403:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880513.726406:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880513.726408:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880513.726410:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880513.726415:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880513.726416:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880513.726445:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880513.726449:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880513.726452:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880514.750356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880514.750363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880514.750369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880514.750370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880514.750431:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880514.750435:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880514.750439:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880515.710544:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880515.710552:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880515.710558:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000020:00000040:2.0:1788880515.710563:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880515.710568:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880515.710571:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880515.710575:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880515.710582:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c54a5c6f. 00000100:00000001:2.0:1788880515.710596:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880515.710607:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880515.710617:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880515.710620:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880515.710627:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_02+2] req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880515.710655:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880515.710656:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880515.710660:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880515.710661:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880515.710666:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880515.710667:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000010:2.0:1788880515.710671:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 00000100:00000040:1.0:1788880515.710675:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:2.0:1788880515.710675:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880515.710680:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880515.710683:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880515.710687:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:2.0:1788880515.710687:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.710689:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880515.710694:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880515.710696:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880515.710698:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880515.710701:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780663552:192.168.204.111@tcp:400: 02000000:00000010:2.0:1788880515.710702:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 00000100:00000001:1.0:1788880515.710707:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880515.710712:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880515.710715:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880515.710720:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880515.710720:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880515.710725:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880515.710731:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005d52ca5c. 00000100:00000001:2.0:1788880515.710732:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:1.0:1788880515.710735:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.710736:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880515.710741:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_03+3] req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880515.710752:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000066432675. 00000100:00000001:2.0:1788880515.710753:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:2.0:1788880515.710756:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:1.0:1788880515.710759:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000065ad0e62. 00000100:00080000:2.0:1788880515.710760:0:53635:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00000001:2.0:1788880515.710764:0:53635:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00000200:1.0:1788880515.710765:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780663552, portal 25 00000100:00080000:2.0:1788880515.710767:0:53635:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:1.0:1788880515.710768:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880515.710771:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 00000100:00000001:2.0:1788880515.710774:0:53635:0:(import.c:485:import_select_connection()) Process entered 00000100:00000040:1.0:1788880515.710778:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880531 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880515.710786:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00080000:2.0:1788880515.710786:0:53635:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 11474 00000100:00000040:1.0:1788880515.710791:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880515.710794:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000002fa6ff6. 00000100:00000200:1.0:1788880515.710796:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780663552, offset 0 00000400:00000200:2.0:1788880515.710798:0:53635:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000400:00000200:1.0:1788880515.710804:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00001000:2.0:1788880515.710804:0:53635:0:(import.c:2149:obd_at_measure()) add 15 to 0000000040f48a4f time=17 v=10 (10 1 1 0) 00000400:00000200:1.0:1788880515.710814:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880515.710818:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00001000:2.0:1788880515.710820:0:53635:0:(import.c:2215:obd_at_measure()) AT 0000000040f48a4f change: old=10 new=15 delta=5 (val=15) hist 15 1 1 0 00000400:00000200:1.0:1788880515.710825:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00080000:2.0:1788880515.710826:0:53635:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: tried all connections, increasing latency to 10s 00000400:00000200:1.0:1788880515.710830:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1788880515.710832:0:53635:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000400:00000200:1.0:1788880515.710837:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880515.710837:0:53635:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000400:00000200:1.0:1788880515.710843:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [257:257:256:4294967295] 192.168.204.111@tcp LPNI seq info [257:257:8:4294967295] 00000100:00000040:2.0:1788880515.710848:0:53635:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880515.710851:0:53635:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000001:2.0:1788880515.710854:0:53635:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880515.710858:0:53635:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:2.0:1788880515.710861:0:53635:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000400:00000200:1.0:1788880515.710862:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000020:00000040:2.0:1788880515.710866:0:53635:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 00000020:00000001:2.0:1788880515.710870:0:53635:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000800:00000200:1.0:1788880515.710871:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000020:00000001:2.0:1788880515.710874:0:53635:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000800:00000010:1.0:1788880515.710878:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000040:2.0:1788880515.710879:0:53635:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880515.710884:0:53635:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880515.710885:0:53635:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000001:2.0:1788880515.710889:0:53635:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000800:00000200:1.0:1788880515.710890:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000020:00000040:2.0:1788880515.710894:0:53635:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 00000100:00080000:2.0:1788880515.710900:0:53635:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: import 00000000b98b8a4e using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000800:00000200:1.0:1788880515.710903:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880515.710905:0:53635:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880515.710909:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880515.710909:0:53635:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:2.0:1788880515.710917:0:53635:0:(sec.c:1628:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880515.710920:0:53635:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.710923:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.710924:0:53635:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:2.0:1788880515.710929:0:53635:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 2097152, lost: 0. 00000100:00000001:1.0:1788880515.710931:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306151887 00000008:00000001:2.0:1788880515.710937:0:53635:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.710940:0:53635:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.710941:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306147267 00000100:00000001:1.0:1788880515.710946:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306151887 00000100:00000001:1.0:1788880515.710950:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306147267 00000100:00000001:1.0:1788880515.710953:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306151887 00000100:00000010:2.0:1788880515.710957:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880515.710962:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880515.710964:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306147267 00000100:00000001:1.0:1788880515.710967:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306151887 00000100:00000001:1.0:1788880515.710970:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306147267 02000000:00000001:2.0:1788880515.710973:0:53635:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 00000100:00000001:1.0:1788880515.710975:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880515.710976:0:53635:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.710982:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.710982:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880515.710986:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880515.710987:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880515.710989:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.710992:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:2.0:1788880515.710995:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 00000000de3147fa. 00000100:00000001:2.0:1788880515.711000:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.711002:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880515.711005:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:2.0:1788880515.711006:0:53635:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880515.711009:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880515.711012:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880515.711014:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880515.711019:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:1.0:1788880515.711023:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880515.711024:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880515.711026:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880515.711026:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.711030:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880515.711032:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_rcv+-1] req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880515.711034:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880515.711037:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880515.711040:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880515.711047:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.711049:0:53635:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880515.711053:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880515.711059:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880515.711065:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880515.711066:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11484) 00000100:00000001:1.0:1788880515.711068:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880515.711072:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880515.711074:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880515.711075:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880515.711076:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.711078:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880515.711082:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780663680:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880515.711087:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880515.711090:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880515.711091:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880515.711093:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880515.711096:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000020450263. 02000000:00000001:1.0:1788880515.711097:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880515.711101:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b34466e7. 00000400:00000010:1.0:1788880515.711111:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:1.0:1788880515.711114:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780663680, portal 10 00000100:00000001:1.0:1788880515.711115:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880515.711116:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938435456 : -126823771116160 : ffff8ca78a2c6d80) 00000100:00000040:1.0:1788880515.711122:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880531 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880515.711129:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880515.711132:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880515.711135:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000200:1.0:1788880515.711137:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780663680, offset 0 00000400:00000200:1.0:1788880515.711142:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880515.711149:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880515.711153:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880515.711153:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:1.0:1788880515.711157:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000010:2.0:1788880515.711158:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:1.0:1788880515.711162:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880515.711165:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880515.711166:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880515.711173:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [258:258:255:4294967295] 192.168.204.111@tcp LPNI seq info [258:258:7:4294967295] 00000400:00000200:2.0:1788880515.711176:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880515.711183:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880515.711188:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880515.711190:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000002fa6ff6 00000800:00000010:1.0:1788880515.711191:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000400:00000010:2.0:1788880515.711194:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000002fa6ff6. 00000800:00000200:1.0:1788880515.711198:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880515.711202:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:1.0:1788880515.711203:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880515.711206:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000200:2.0:1788880515.711209:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880531 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880515.711210:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880515.711215:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 346242 usecs (347 jiffies) 00000100:00000001:1.0:1788880515.711219:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306147163 00000100:00000001:1.0:1788880515.711223:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306151887 00000100:00000001:1.0:1788880515.711227:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306147163 00000100:00000001:2.0:1788880515.711228:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880515.711230:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306151887 00000100:00000040:2.0:1788880515.711257:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880531 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.711266:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.711271:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306147163 00000100:00000001:2.0:1788880515.711277:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880515.711278:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306151887 00000100:00000001:1.0:1788880515.711282:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306147163 00000100:00000001:1.0:1788880515.711284:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306151887 00000100:00000001:1.0:1788880515.711286:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880515.711289:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.711292:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.711294:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880515.711296:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880515.711298:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:2.0:1788880515.711302:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880515.711305:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880515.711307:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:2.0:1788880515.711308:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:2.0:1788880515.711310:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880515.711313:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880531 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.711318:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880515.711320:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880531 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.711324:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.711325:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880515.711333:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880515.711334:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880515.711336:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880515.711338:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.711342:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880515.711343:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880515.711345:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880515.711347:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880515.711348:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880515.711349:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780663808:192.168.204.111@tcp:38: 00000100:00000001:2.0:1788880515.711352:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880515.711354:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880515.711355:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880515.711356:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880515.711358:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000005e22417f. 02000000:00000001:2.0:1788880515.711360:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880515.711362:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000400:00000010:2.0:1788880515.711365:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000200:2.0:1788880515.711372:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780663808, portal 10 00000100:00000001:2.0:1788880515.711373:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880515.711374:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000100:00000040:2.0:1788880515.711379:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880535 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.711383:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880515.711385:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880515.711386:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c3a5d4ae. 00000100:00000200:2.0:1788880515.711389:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780663808, offset 0 00000400:00000200:2.0:1788880515.711392:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880515.711398:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880515.711400:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880515.711402:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880515.711405:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880515.711408:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880515.711411:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [259:259:256:4294967295] 192.168.204.111@tcp LPNI seq info [259:259:8:4294967295] 00000400:00000200:2.0:1788880515.711417:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880515.711420:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880515.711423:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000011e1ab42. 00000800:00000200:2.0:1788880515.711426:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880515.711429:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880515.711431:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000011e1ab42 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:2.0:1788880515.711439:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880515.711441:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 21886 usecs (22 jiffies) 00000100:00000001:2.0:1788880515.711443:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306146838 00000100:00000001:2.0:1788880515.711445:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306151887 00000100:00000001:2.0:1788880515.711446:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306146838 00000100:00000001:2.0:1788880515.711447:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306151887 00000100:00000001:2.0:1788880515.711449:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306146838 00000100:00000001:2.0:1788880515.711454:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306151887 00000100:00000001:2.0:1788880515.711455:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306146838 00000100:00000001:2.0:1788880515.711457:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306151887 00000100:00000001:2.0:1788880515.711458:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880515.711459:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.711461:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.711462:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880515.711466:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880515.711467:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:2.0:1788880515.711468:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880515.711469:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880515.711470:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.711471:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880515.711526:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880515.711530:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000011e1ab42. 00000400:00000200:0.0:1788880515.711534:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880515.711540:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880515.711543:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c3a5d4ae 00000400:00000010:0.0:1788880515.711550:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c3a5d4ae. 00000100:00000001:0.0:1788880515.711554:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880515.711557:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880535 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880515.711567:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880515.711570:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880535 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880515.711578:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880515.711580:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880515.712716:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880515.712728:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880515.712731:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880515.712733:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880515.712740:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880515.712752:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b07b00 00000400:00000200:0.0:1788880515.712761:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x8e1 [1] + 224 00000800:00000001:0.0:1788880515.712767:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880515.712773:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880515.712775:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880515.712779:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880515.712784:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880515.712789:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880515.712800:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880515.712813:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880515.712820:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880515.712824:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880515.712951:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880515.712955:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880515.712962:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880515.712964:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880515.712966:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000066432675. 00000400:00000200:2.0:1788880515.712971:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000065ad0e62 00000400:00000010:2.0:1788880515.712973:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000065ad0e62. 00000100:00000001:2.0:1788880515.712977:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880515.712981:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880515.712990:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.712996:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880515.712998:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880515.713001:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880515.713003:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880515.713006:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880515.713008:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880515.713011:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880515.713015:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=43 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880515.713019:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=43 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880515.713022:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880515.713030:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880515.713034:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880515.713036:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.713039:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880515.713044:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.713050:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880515.713052:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880515.713054:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880515.713058:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880515.713065:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780663552:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880515.713070:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.713072:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880515.713075:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780663552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.713082:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880515.713084:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880515.713087:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005d52ca5c. 02000000:00000001:2.0:1788880515.713093:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880515.713094:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880515.713097:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880515.713099:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880515.713101:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c54a5c6f. 02000000:00000001:2.0:1788880515.713105:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880515.713107:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880515.713108:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000001:2.0:1788880515.713111:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880515.713113:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880515.713115:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.713117:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880515.713119:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880515.713120:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.713122:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880515.714630:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880515.714640:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880515.714643:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880515.714645:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880515.714652:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880515.714664:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b07b80 00000400:00000200:2.0:1788880515.714671:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x8e9 [1] + 224 00000800:00000001:2.0:1788880515.714676:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880515.714681:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880515.714684:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880515.714688:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880515.714692:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880515.714697:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880515.714706:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.714724:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880515.714730:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880515.714733:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880515.714752:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880515.714756:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880515.714760:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880515.714762:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880515.714764:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b34466e7. 00000400:00000200:1.0:1788880515.714768:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:1.0:1788880515.714770:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:1.0:1788880515.714774:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880515.714778:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880515.714788:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880515.714795:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880515.714797:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880515.714799:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880515.714801:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880515.714803:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880515.714804:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880515.714807:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880515.714813:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=33 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880515.714817:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=33 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880515.714819:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880515.714820:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880515.714822:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880515.714827:0:201287:0:(ldlm_request.c:1723:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.714839:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880515.714841:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880515.714843:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880515.714844:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880515.714848:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880515.714855:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880515.714858:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880515.714860:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880515.714863:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880515.714871:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780663680:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880515.714875:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880515.714878:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880515.714880:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000175072aa x1875776780663680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880531 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880515.714887:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880515.714889:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880515.714891:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000020450263. 02000000:00000001:1.0:1788880515.714896:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880515.714897:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880515.714899:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880515.714902:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880515.714904:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 02000000:00000001:1.0:1788880515.714907:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880515.714908:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880515.714910:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000175072aa. 00000100:00000001:1.0:1788880515.714913:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880515.714915:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880515.714917:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880515.714923:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880515.714924:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880515.714927:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880515.714934:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880515.720385:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880515.720408:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880515.720413:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880515.720416:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880515.720431:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880515.720465:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 416 into portal 10 MB=0x6aa0250b07c00 00000400:00000200:0.0:1788880515.720476:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 416/416 into md 0x8f1 [1] + 0 00000800:00000001:0.0:1788880515.720501:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880515.720511:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880515.720515:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880515.720520:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880515.720530:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880515.720539:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880535 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880515.720553:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=416 offset=0 replen=544 req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880535 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880515.720566:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880515.720580:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880515.720583:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880515.721692:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880515.721700:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880515.721708:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880515.721711:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880515.721713:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:2.0:1788880515.721722:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880515.721724:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880515.721729:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880515.721739:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880535 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880515.721749:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880535 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.721757:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880515.721772:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880515.721775:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880515.721779:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880515.721781:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880515.721783:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880515.721786:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880515.721796:0:201284:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=43 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880515.721800:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=17 v=15 (15 1 1 0) 00000100:00000001:2.0:1788880515.721803:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880515.721804:0:201284:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880515.721806:0:201284:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880515.721811:0:201284:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880535 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880515.721818:0:201284:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.721822:0:201284:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880515.721824:0:201284:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880515.721826:0:201284:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880515.721827:0:201284:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880515.721831:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880535 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880515.721837:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880515.721840:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880515.721841:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880515.721844:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00080000:2.0:1788880515.721849:0:201284:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0001-mdc-ffff8ca787527000: connect to target with instance 5 00000020:00000001:2.0:1788880515.721852:0:201284:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:2.0:1788880515.721854:0:201284:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:2.0:1788880515.721856:0:201284:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880515.721860:0:201284:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 00000020:00000001:2.0:1788880515.721862:0:201284:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000020:00000001:2.0:1788880515.721864:0:201284:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000100:00001000:2.0:1788880515.721867:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=1788880515 v=0 (0 0 0 0) 00000100:00001000:2.0:1788880515.721869:0:201284:0:(import.c:2215:obd_at_measure()) AT 0000000040f48a4f change: old=0 new=5 delta=5 (val=1) hist 1 0 0 0 00000100:00000040:2.0:1788880515.721881:0:201284:0:(obd_cksum.h:77:obd_cksum_types_supported_client()) Crypto hash speed: crc 1651, crc32c 9963, adler 1371 00000020:00000040:2.0:1788880515.721886:0:201284:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 00000100:00000001:2.0:1788880515.721889:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 00000002:00000001:2.0:1788880515.721891:0:201284:0:(mdc_request.c:2879:mdc_import_event()) Process entered 00000008:00000020:2.0:1788880515.721894:0:201284:0:(osc_request.c:1062:osc_init_grant()) lustre-MDT0001-mdc-ffff8ca787527000, setting cl_avail_grant: 2097152 cl_lost_grant: 0. chunk bits: 12 cl_max_extent_pages: 16384 00000002:00000001:2.0:1788880515.721898:0:201284:0:(mdc_request.c:3107:mdc_init_ea_size()) Process entered 00000002:00000001:2.0:1788880515.721900:0:201284:0:(mdc_request.c:3114:mdc_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880515.721901:0:201284:0:(obd_class.h:1243:obd_notify()) Process entered 00800000:00000001:2.0:1788880515.721904:0:201284:0:(lmv_obd.c:188:lmv_notify()) Process entered 00000080:00000001:2.0:1788880515.721907:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:2.0:1788880515.721910:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880515.721912:0:201284:0:(lmv_obd.c:223:lmv_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880515.721913:0:201284:0:(obd_class.h:1256:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880515.721915:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:2.0:1788880515.721916:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880515.721917:0:201284:0:(mdc_request.c:2953:mdc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.721919:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00080000:2.0:1788880515.721922:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from CONNECTING to FULL 00000100:00000001:2.0:1788880515.721924:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 00000002:00000001:2.0:1788880515.721925:0:201284:0:(mdc_request.c:2879:mdc_import_event()) Process entered 00000002:00000001:2.0:1788880515.721926:0:201284:0:(obd_class.h:1243:obd_notify()) Process entered 00800000:00000001:2.0:1788880515.721926:0:201284:0:(lmv_obd.c:188:lmv_notify()) Process entered 00800000:00000001:2.0:1788880515.721927:0:201284:0:(lmv_obd.c:82:lmv_set_mdc_active()) Process entered 00800000:00000040:2.0:1788880515.721928:0:201284:0:(lmv_obd.c:85:lmv_set_mdc_active()) Searching in lmv 00000000b999beb9 for uuid lustre-MDT0001_UUID (activate=1) 00800000:00000040:2.0:1788880515.721931:0:201284:0:(lmv_obd.c:91:lmv_set_mdc_active()) Target idx 0 is lustre-MDT0000_UUID conn 0x805d548a36a8869e 00800000:00000040:2.0:1788880515.721932:0:201284:0:(lmv_obd.c:91:lmv_set_mdc_active()) Target idx 1 is lustre-MDT0001_UUID conn 0x805d548a36a886a5 00800000:00000040:2.0:1788880515.721934:0:201284:0:(lmv_obd.c:106:lmv_set_mdc_active()) Found OBD lustre-MDT0001-mdc-ffff8ca787527000=ea27006f-52a8-4007-b69a-dcac8279445a device 4 (000000001b5d8280) type mdc at LMV idx 1 00800000:00000040:2.0:1788880515.721937:0:201284:0:(lmv_obd.c:111:lmv_set_mdc_active()) OBD 000000001b5d8280 already active! 00800000:00000001:2.0:1788880515.721938:0:201284:0:(lmv_obd.c:112:lmv_set_mdc_active()) Process leaving via out_lmv_lock (rc=0 : 0 : 0x0) 00000080:00000001:2.0:1788880515.721940:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:2.0:1788880515.721941:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:2.0:1788880515.721942:0:201284:0:(lmv_obd.c:223:lmv_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:2.0:1788880515.721943:0:201284:0:(obd_class.h:1256:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1788880515.721945:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:2.0:1788880515.721945:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880515.721953:0:201284:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=5 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000002:00000001:2.0:1788880515.722198:0:201284:0:(mdc_request.c:2953:mdc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.722201:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00000001:2.0:1788880515.722204:0:201284:0:(import.c:1648:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00000001:2.0:1788880515.722207:0:201284:0:(import.c:1754:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.722210:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880515.722214:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880535 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880515.722220:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780663808:192.168.204.111@tcp:38: 00000100:00000001:2.0:1788880515.722223:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880515.722225:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880515.722226:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780663808/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880535 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000002:00000001:0.0:1788880515.722442:0:205874:0:(mdc_request.c:2638:mdc_kuc_reregister_thread()) Process entered 00000020:00000001:0.0:1788880515.722445:0:205874:0:(kernelcomm.c:1325:libcfs_kkuc_group_foreach()) Process entered 00000020:00000001:0.0:1788880515.722448:0:205874:0:(kernelcomm.c:1339:libcfs_kkuc_group_foreach()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880515.722451:0:205874:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880515.722453:0:205874:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880515.722456:0:205874:0:(genops.c:1251:class_import_put()) Process leaving 00000002:00000001:0.0:1788880515.722456:0:205874:0:(mdc_request.c:2648:mdc_kuc_reregister_thread()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.722560:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880515.722563:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880515.722566:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000005e22417f. 02000000:00000001:2.0:1788880515.722570:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880515.722572:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880515.722574:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880515.722576:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880515.722578:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 00000000de3147fa. 02000000:00000001:2.0:1788880515.722582:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880515.722583:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880515.722585:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:2.0:1788880515.722589:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880515.722590:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880515.722592:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880515.722599:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880515.722601:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880515.722604:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880515.722606:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880515.774351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880515.774358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880515.774364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880515.774365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880515.774384:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880515.774387:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880515.774390:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00000001:1.0:1788880516.735295:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880516.735301:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-cliir (0000000008e74578) refs 2 10000000:00000001:1.0:1788880516.735306:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880516.735309:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000001:1.0:1788880516.735310:0:203401:0:(mgc_request.c:120:config_log_put()) Process leaving 10000000:00000001:1.0:1788880516.735311:0:203401:0:(mgc_request.c:601:do_requeue()) Process entered 10000000:01000000:1.0:1788880516.735313:0:203401:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:00000001:1.0:1788880516.735316:0:203401:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:1.0:1788880516.735319:0:203401:0:(mgc_request.c:1801:mgc_process_log()) Process log lustre-cliir-ffff8ca787527000 from 13 10000000:00000001:1.0:1788880516.735322:0:203401:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:1.0:1788880516.735324:0:203401:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00000100:00000010:1.0:1788880516.735331:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000020:00000040:1.0:1788880516.735335:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880516.735340:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880516.735343:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880516.735348:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880516.735354:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000ab1d1f2f. 00000100:00000001:1.0:1788880516.735362:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.735367:0:203401:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:1.0:1788880516.735371:0:203401:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:1.0:1788880516.735378:0:203401:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at 00000000b9a8b8b1. 00010000:00000001:1.0:1788880516.735382:0:203401:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:1.0:1788880516.735386:0:203401:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 00000000408d8a8f. 00000020:00000001:1.0:1788880516.735390:0:203401:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:1.0:1788880516.735393:0:203401:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000408d8a8f with handle 0x805d548a36a886dd to hash 00000020:00000001:1.0:1788880516.735401:0:203401:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:1.0:1788880516.735403:0:203401:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880516.735406:0:203401:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880516.735411:0:203401:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1788880516.735421:0:203401:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880516.735425:0:203401:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880516.735434:0:203401:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880516.735449:0:203401:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880516.735456:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880516.735458:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880516.735465:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000edab0b56. 00000100:00000001:1.0:1788880516.735468:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063674752 : -126823645876864 : ffff8ca791a36d80) 00000100:00000001:1.0:1788880516.735483:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.735485:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000001:1.0:1788880516.735490:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880516.735492:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.735495:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880516.735497:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880516.735508:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.735517:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880516.735520:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.735524:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880516.735526:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880516.735530:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880516.735556:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780663936:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880516.735561:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880516.735567:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880516.735569:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.735572:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880516.735577:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000845fe3a7. 02000000:00000001:1.0:1788880516.735580:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880516.735592:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b34466e7. 00000400:00000010:1.0:1788880516.735599:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:1.0:1788880516.735606:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780663936, portal 25 00000100:00000001:1.0:1788880516.735608:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.735609:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000040:1.0:1788880516.735615:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.735623:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880516.735626:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880516.735629:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000030b8c48c. 00000100:00000200:1.0:1788880516.735631:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780663936, offset 0 00000400:00000200:1.0:1788880516.735638:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880516.735653:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880516.735658:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880516.735664:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880516.735668:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880516.735670:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880516.735674:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [260:260:256:4294967295] 192.168.204.111@tcp LPNI seq info [260:260:8:4294967295] 00000400:00000200:1.0:1788880516.735688:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880516.735696:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880516.735701:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000800:00000200:1.0:1788880516.735708:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880516.735716:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880516.735720:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 424 niov 1 nkiov 1 00000100:00000001:1.0:1788880516.735733:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.735738:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306152912 00000100:00000001:1.0:1788880516.735745:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306151887 00000100:00000001:1.0:1788880516.735751:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306152912 00000100:00000001:1.0:1788880516.735755:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306151887 00000100:00000001:1.0:1788880516.735759:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306152912 00000100:00000001:1.0:1788880516.735769:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306151887 00000100:00000001:1.0:1788880516.735773:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306152912 00000100:00000001:1.0:1788880516.735778:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306151887 00000100:00000001:1.0:1788880516.735781:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880516.735785:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.735787:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880516.735789:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880516.735791:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 00000000edab0b56 going to sleep for 11 seconds 00000100:00000001:1.0:1788880516.735794:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880516.735796:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880516.735833:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880516.735838:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:0.0:1788880516.735842:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.735847:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880516.735852:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000030b8c48c 00000400:00000010:0.0:1788880516.735855:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000030b8c48c. 00000100:00000001:0.0:1788880516.735859:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880516.735863:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.735872:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880516.735875:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.735888:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880516.735890:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880516.737436:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.737447:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880516.737450:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.737451:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880516.737457:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880516.737468:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b07c80 00000400:00000200:3.0:1788880516.737523:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0x8f9 [1] + 224 00000800:00000001:3.0:1788880516.737530:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.737544:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880516.737546:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880516.737550:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880516.737556:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880516.737562:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880516.737572:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880516.737597:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880516.737605:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.737609:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.737625:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880516.737633:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880516.737635:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880516.737637:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b34466e7. 00000400:00000200:1.0:1788880516.737641:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:1.0:1788880516.737643:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:1.0:1788880516.737655:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880516.737659:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880516.737672:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.737679:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880516.737681:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.737683:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880516.737686:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880516.737688:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880516.737690:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.737693:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880516.737699:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=44 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880516.737703:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=44 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880516.737706:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880516.737708:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.737710:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880516.737713:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.737721:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880516.737725:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.737732:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880516.737734:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880516.737736:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880516.737739:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880516.737747:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780663936:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880516.737751:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.737753:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.737756:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880516.737757:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.737760:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.737766:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880516.737768:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000edab0b56. 00000100:00000001:1.0:1788880516.737772:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880516.737774:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.737777:0:203401:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:1.0:1788880516.737779:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880516.737781:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880516.737784:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=4 00000020:00000001:1.0:1788880516.737786:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880516.737789:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000040:1.0:1788880516.737794:0:203401:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 00000000408d8a8f, remote cookie: 0x92f11317f60a7bd3, flags: 0x0 00010000:00000001:1.0:1788880516.737798:0:203401:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:1.0:1788880516.737801:0:203401:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:1.0:1788880516.737803:0:203401:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:1.0:1788880516.737805:0:203401:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:1.0:1788880516.737806:0:203401:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:1.0:1788880516.737808:0:203401:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x2:0x0].0x0 (00000000b9a8b8b1) refcount = 1 00010000:00010000:1.0:1788880516.737818:0:203401:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880516.737825:0:203401:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:1.0:1788880516.737827:0:203401:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:1.0:1788880516.737828:0:203401:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788880516.737831:0:203401:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:1.0:1788880516.737833:0:203401:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880516.737836:0:203401:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880516.737842:0:203401:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:1.0:1788880516.737844:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880516.737845:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880516.737846:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880516.737847:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880516.737848:0:203401:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.737851:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.737854:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780663936/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.737861:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880516.737864:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880516.737866:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000845fe3a7. 02000000:00000001:1.0:1788880516.737869:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880516.737871:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880516.737873:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880516.737875:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880516.737877:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000ab1d1f2f. 02000000:00000001:1.0:1788880516.737882:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880516.737883:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880516.737884:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:1.0:1788880516.737888:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880516.737890:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:1.0:1788880516.737892:0:203401:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880516.737894:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880516.737895:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-cliir (0000000008e74578) refs 3 10000000:00000001:1.0:1788880516.737902:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.737904:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880516.737905:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880516.737907:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=3 00000020:00000001:1.0:1788880516.737909:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880516.737911:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880516.737913:0:203401:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:1.0:1788880516.737915:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880516.737916:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880516.737917:0:203401:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880516.737919:0:203401:0:(mgc_request.c:1519:mgc_process_recover_log()) Process entered 10000000:00000010:1.0:1788880516.737923:0:203401:0:(mgc_request.c:1531:mgc_process_recover_log()) kmalloced '(folios)': 32 at 00000000782a973c. 00000100:00000010:1.0:1788880516.737938:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000020:00000040:1.0:1788880516.737941:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880516.737943:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880516.737946:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880516.737949:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880516.737953:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000ab1d1f2f. 00000100:00000001:1.0:1788880516.737957:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.737961:0:203401:0:(client.c:227:ptlrpc_prep_bulk_imp()) Process entered 00000100:00000010:1.0:1788880516.737964:0:203401:0:(client.c:169:ptlrpc_new_bulk()) kmalloced '(desc)': 984 at 00000000845fe3a7. 00000100:00000010:1.0:1788880516.737966:0:203401:0:(client.c:174:ptlrpc_new_bulk()) kmalloced '(desc->bd_vec)': 64 at 0000000088249268. 00000020:00000040:1.0:1788880516.737968:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880516.737973:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880516.737974:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880516.737978:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000edab0b56. 00000100:00000001:1.0:1788880516.737980:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063674752 : -126823645876864 : ffff8ca791a36d80) 00000100:00000001:1.0:1788880516.737982:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.737983:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000001:1.0:1788880516.737986:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880516.737989:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.737990:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880516.737991:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880516.737994:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.738012:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880516.738014:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.738017:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880516.738019:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880516.738021:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880516.738023:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664064:192.168.204.111@tcp:256: 00000100:00000001:1.0:1788880516.738027:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880516.738030:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880516.738032:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.738034:0:203401:0:(niobuf.c:346:ptlrpc_register_bulk()) Process entered 00000400:00000010:1.0:1788880516.738039:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b34466e7. 00000400:00000010:1.0:1788880516.738043:0:203401:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 00000000199f9b06. 00000100:00000200:1.0:1788880516.738047:0:203401:0:(niobuf.c:453:ptlrpc_register_bulk()) Setup 1 bulk put-sink buffers: 4 pages 16384 bytes, mbits x0x6aa0250b07d00-0x6aa0250b07d00, portal 33 00000100:00000001:1.0:1788880516.738050:0:203401:0:(niobuf.c:455:ptlrpc_register_bulk()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.738052:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880516.738055:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e28573ff. 02000000:00000001:1.0:1788880516.738057:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880516.738059:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a3419a3. 00000400:00000010:1.0:1788880516.738061:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:1.0:1788880516.738064:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780664064, portal 25 00000100:00000001:1.0:1788880516.738066:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.738067:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000040:1.0:1788880516.738071:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.738078:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880516.738080:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880516.738082:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000ae22117. 00000100:00000200:1.0:1788880516.738084:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 304 bytes to portal 26, xid 1875776780664064, offset 0 00000400:00000200:1.0:1788880516.738089:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880516.738095:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880516.738100:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880516.738109:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880516.738113:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880516.738116:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880516.738119:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [261:261:256:4294967295] 192.168.204.111@tcp LPNI seq info [261:261:8:4294967295] 00000400:00000200:1.0:1788880516.738128:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880516.738142:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 304 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880516.738147:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000800:00000200:1.0:1788880516.738151:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880516.738158:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880516.738160:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 400 niov 1 nkiov 1 00000100:00000001:1.0:1788880516.738172:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.738175:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306152914 00000100:00000001:1.0:1788880516.738181:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306152912 00000100:00000001:1.0:1788880516.738184:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306152914 00000100:00000001:1.0:1788880516.738188:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306152912 00000100:00000001:1.0:1788880516.738190:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306152914 00000100:00000001:1.0:1788880516.738194:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306152912 00000100:00000001:1.0:1788880516.738196:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306152914 00000100:00000001:1.0:1788880516.738199:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306152912 00000100:00000001:1.0:1788880516.738202:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880516.738204:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.738206:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880516.738207:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880516.738209:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 00000000edab0b56 going to sleep for 11 seconds 00000100:00000001:1.0:1788880516.738212:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880516.738214:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880516.738260:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880516.738265:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:0.0:1788880516.738269:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.738273:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880516.738276:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000ae22117 00000400:00000010:0.0:1788880516.738278:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000ae22117. 00000100:00000001:0.0:1788880516.738282:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880516.738285:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.738292:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880516.738295:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.738301:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880516.738302:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880516.739252:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.739260:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880516.739264:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.739266:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.739272:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880516.739282:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 44 into portal 33 MB=0x6aa0250b07d00 00000400:00000200:0.0:1788880516.739288:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 21 from 12345-192.168.204.111@tcp of length 44/44 into md 0x901 [4] + 0 00000400:00000010:0.0:1788880516.739291:0:201280:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b34466e7. 00000400:00000200:0.0:1788880516.739295:0:201280:0:(lib-md.c:42:lnet_md_unlink()) Queueing unlink of md 00000000199f9b06 00000800:00000001:0.0:1788880516.739298:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.739303:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.739305:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.739308:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880516.739311:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000199f9b06 00000400:00000010:0.0:1788880516.739313:0:201280:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 00000000199f9b06. 00000100:00000001:0.0:1788880516.739316:0:201280:0:(events.c:174:client_bulk_callback()) Process entered 00000100:00000200:0.0:1788880516.739324:0:201280:0:(events.c:192:client_bulk_callback()) event type 2, status 0, req 00000000ca5e4c87 desc 00000000845fe3a7 mbits 1875776780664064 00000100:00000001:0.0:1788880516.739332:0:201280:0:(events.c:217:client_bulk_callback()) Process leaving 00000400:00000200:0.0:1788880516.739336:0:201280:0:(lib-md.c:96:lnet_cpt_of_md()) Couldn't resolve first page of md 0000000000000000 with offset 0 00000400:00000200:0.0:1788880516.739339:0:201280:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880516.739345:0:201280:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [262:262:256:4294967295] 192.168.204.111@tcp LPNI seq info [262:262:8:4294967295] 00000100:00000001:1.0:1788880516.739356:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880516.739362:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880516.739365:0:201280:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) 192.168.204.111@tcp : ACK try# 0 00000800:00000200:0.0:1788880516.739377:0:201280:0:(socklnd_cb.c:931:ksocknal_send()) sending 0 bytes in 4 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880516.739382:0:201280:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 296 at 000000002f5492f5. 00000800:00000200:0.0:1788880516.739385:0:201280:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880516.739388:0:201280:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880516.739396:0:201280:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000002f5492f5 type 0, nob 96 niov 1 nkiov 0 00000800:00000001:3.0:1788880516.739428:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000200:0.0:1788880516.739445:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880516.739449:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 296 at 000000002f5492f5. 00000400:00000200:0.0:1788880516.739453:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000001:3.0:1788880516.739453:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.739460:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: ACK: OK 00000800:00000001:0.0:1788880516.740185:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.740191:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880516.740192:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.740193:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.740197:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880516.740202:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 240 into portal 25 MB=0x6aa0250b07d00 00000400:00000200:0.0:1788880516.740206:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 240/240 into md 0x905 [1] + 224 00000800:00000001:0.0:1788880516.740209:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.740212:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.740213:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.740214:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880516.740217:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880516.740221:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880516.740227:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=240 offset=224 replen=240 req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.740270:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880516.740277:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.740280:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.740320:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880516.740326:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880516.740334:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880516.740336:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a3419a3. 00000400:00000200:1.0:1788880516.740340:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:1.0:1788880516.740342:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:1.0:1788880516.740345:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880516.740349:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880516.740358:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.740366:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880516.740367:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.740370:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880516.740372:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880516.740374:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880516.740375:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.740377:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880516.740382:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=44 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880516.740385:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=44 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880516.740388:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880516.740390:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.740392:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880516.740394:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.740396:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880516.740399:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Bulk req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880516.740409:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Bulk to Interpret req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 2 fl Bulk:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.740416:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880516.740418:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880516.740432:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880516.740439:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664064:192.168.204.111@tcp:256: 00000100:00000001:1.0:1788880516.740452:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.740455:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.740457:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880516.740459:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.740461:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.740467:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880516.740480:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000edab0b56. 00000100:00000001:1.0:1788880516.740483:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880516.740485:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:1.0:1788880516.740489:0:203401:0:(mgc_request.c:1599:mgc_process_recover_log()) Latest version 13, more 0. 10000000:00000001:1.0:1788880516.740492:0:203401:0:(mgc_request.c:1308:mgc_apply_recover_logs()) Process entered 10000000:00000010:1.0:1788880516.740506:0:203401:0:(mgc_request.c:1331:mgc_apply_recover_logs()) kmalloced '(nidlist)': 640 at 000000009ef860f0. 10000000:00000010:1.0:1788880516.740509:0:203401:0:(mgc_request.c:1343:mgc_apply_recover_logs()) kmalloced '(obdname)': 128 at 000000002d7511ca. 10000000:00000200:1.0:1788880516.740514:0:203401:0:(mgc_request.c:1241:mgc_build_nidlist()) use 1 from 1 NIDs 10000000:00000040:1.0:1788880516.740517:0:203401:0:(mgc_request.c:1440:mgc_apply_recover_logs()) Looking for OBD lustre-MDT0001-mdc-ffff8ca787527000 00010000:00000001:1.0:1788880516.740522:0:203401:0:(ldlm_lib.c:159:client_import_add_nids_to_conn()) Process entered 00000100:00000001:1.0:1788880516.740524:0:203401:0:(lustre_peer.c:229:class_check_uuid()) Process entered 00000100:00000040:1.0:1788880516.740528:0:203401:0:(lustre_peer.c:232:class_check_uuid()) check if uuid 192.168.204.111@tcp has 192.168.204.111@tcp. 00000100:00000001:1.0:1788880516.740530:0:203401:0:(lustre_peer.c:251:class_check_uuid()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.740532:0:203401:0:(lustre_peer.c:189:class_add_nids_to_uuid()) Process entered 00000100:00000200:1.0:1788880516.740533:0:203401:0:(lustre_peer.c:194:class_add_nids_to_uuid()) Comparing 192.168.204.111@tcp with 192.168.204.111@tcp 00000100:00000200:1.0:1788880516.740535:0:203401:0:(lustre_peer.c:201:class_add_nids_to_uuid()) Updating UUID '192.168.204.111@tcp' 00000400:00000200:1.0:1788880516.740538:0:203401:0:(peer.c:1363:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:1.0:1788880516.740540:0:203401:0:(api-ni.c:3274:LNetNIInit()) refs 1 00000400:00000200:1.0:1788880516.740543:0:203401:0:(peer.c:1377:LNetAddPeer()) Adding NID[0]: 192.168.204.111@tcp 00000400:00000200:1.0:1788880516.740547:0:203401:0:(peer.c:1822:lnet_peer_add()) peer 192.168.204.111@tcp NID flags 0x100001: -17 00000400:00000200:1.0:1788880516.740554:0:203401:0:(peer.c:2890:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:1.0:1788880516.740557:0:203401:0:(peer.c:2899:lnet_discover_peer_locked()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp: 0. pending discovery 00000400:00000200:1.0:1788880516.740560:0:203401:0:(peer.c:1431:LNetAddPeer()) A peer exists for NID 192.168.204.111@tcp 00000100:00000001:1.0:1788880516.740562:0:203401:0:(lustre_peer.c:219:class_add_nids_to_uuid()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.740564:0:203401:0:(ldlm_lib.c:174:client_import_add_nids_to_conn()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:1.0:1788880516.740566:0:203401:0:(mgc_request.c:1486:mgc_apply_recover_logs()) ir apply logs 13/13 for lustre-MDT0001-mdc-ffff8ca787527000 10000000:00000010:1.0:1788880516.740568:0:203401:0:(mgc_request.c:1491:mgc_apply_recover_logs()) kfreed 'obdname': 128 at 000000002d7511ca. 10000000:00000010:1.0:1788880516.740577:0:203401:0:(mgc_request.c:1493:mgc_apply_recover_logs()) kfreed 'nidlist': 640 at 000000009ef860f0. 10000000:00000001:1.0:1788880516.740580:0:203401:0:(mgc_request.c:1495:mgc_apply_recover_logs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.740582:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.740585:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780664064/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880532 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.740592:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880516.740594:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880516.740595:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e28573ff. 02000000:00000001:1.0:1788880516.740599:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880516.740601:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880516.740602:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880516.740604:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880516.740606:0:203401:0:(client.c:312:ptlrpc_free_bulk()) Process entered 00000020:00000001:1.0:1788880516.740607:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880516.740608:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880516.740610:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000010:1.0:1788880516.740612:0:203401:0:(client.c:333:ptlrpc_free_bulk()) kvfree 'desc->bd_vec': 64 at 0000000088249268. 00000100:00000010:1.0:1788880516.740615:0:203401:0:(client.c:334:ptlrpc_free_bulk()) kfreed 'desc': 984 at 00000000845fe3a7. 00000100:00000001:1.0:1788880516.740617:0:203401:0:(client.c:335:ptlrpc_free_bulk()) Process leaving 02000000:00000010:1.0:1788880516.740619:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000ab1d1f2f. 02000000:00000001:1.0:1788880516.740623:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880516.740625:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880516.740626:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:1.0:1788880516.740630:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880516.740632:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000010:1.0:1788880516.740647:0:203401:0:(mgc_request.c:1651:mgc_process_recover_log()) kvfree 'folios': 32 at 00000000782a973c. 10000000:01000000:1.0:1788880516.740650:0:203401:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'lustre-cliir' succeeded (0). 10000000:00000001:1.0:1788880516.740653:0:203401:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:1.0:1788880516.740656:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880516.740658:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880516.740661:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=3 00000020:00000001:1.0:1788880516.740663:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880516.740666:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880516.740668:0:203401:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:1.0:1788880516.740671:0:203401:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880516.740686:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880516.740687:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:1.0:1788880516.740689:0:203401:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880516.740695:0:203401:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:1.0:1788880516.740696:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880516.740697:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:1.0:1788880516.740699:0:203401:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880516.740701:0:203401:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880516.740703:0:203401:0:(mgc_request.c:621:do_requeue()) Process leaving 10000000:00000001:1.0:1788880516.740705:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880516.740706:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-client (00000000c2974351) refs 2 10000000:00000001:1.0:1788880516.740708:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880516.740710:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:1.0:1788880516.740711:0:203401:0:(mgc_request.c:123:config_log_put()) log lustre-cliir(0000000008e74578) refs 3 10000000:00000001:1.0:1788880516.740713:0:203401:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:1.0:1788880516.740714:0:203401:0:(mgc_request.c:601:do_requeue()) Process entered 10000000:01000000:1.0:1788880516.740715:0:203401:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:00000001:1.0:1788880516.740717:0:203401:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:1.0:1788880516.740718:0:203401:0:(mgc_request.c:1801:mgc_process_log()) Process log lustre-client-ffff8ca787527000 from 42 10000000:00000001:1.0:1788880516.740721:0:203401:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:1.0:1788880516.740722:0:203401:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00000100:00000010:1.0:1788880516.740726:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000020:00000040:1.0:1788880516.740729:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880516.740732:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880516.740735:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880516.740738:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880516.740742:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000ab1d1f2f. 00000100:00000001:1.0:1788880516.740746:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.740750:0:203401:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:1.0:1788880516.740753:0:203401:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:1.0:1788880516.740758:0:203401:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at 00000000b425dd0a. 00010000:00000001:1.0:1788880516.740762:0:203401:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:1.0:1788880516.740765:0:203401:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 000000004fb52558. 00000020:00000001:1.0:1788880516.740774:0:203401:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:1.0:1788880516.740777:0:203401:0:(lustre_handles.c:93:class_handle_hash()) added object 000000004fb52558 with handle 0x805d548a36a886e4 to hash 00000020:00000001:1.0:1788880516.740779:0:203401:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:1.0:1788880516.740780:0:203401:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000001:1.0:1788880516.740783:0:203401:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000001:1.0:1788880516.740786:0:203401:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1788880516.740789:0:203401:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880516.740804:0:203401:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880516.740811:0:203401:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880516.740820:0:203401:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880516.740826:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880516.740827:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880516.740832:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000edab0b56. 00000100:00000001:1.0:1788880516.740834:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063674752 : -126823645876864 : ffff8ca791a36d80) 00000100:00000001:1.0:1788880516.740836:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.740837:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000001:1.0:1788880516.740841:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880516.740843:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.740844:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880516.740846:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880516.740849:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.740857:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880516.740858:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.740862:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880516.740863:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880516.740866:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880516.740876:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664192:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880516.740880:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880516.740884:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880516.740886:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.740888:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880516.740891:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000845fe3a7. 02000000:00000001:1.0:1788880516.740893:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880516.740897:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a3419a3. 00000400:00000010:1.0:1788880516.740901:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:1.0:1788880516.740905:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780664192, portal 25 00000100:00000001:1.0:1788880516.740908:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.740909:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000040:1.0:1788880516.740913:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.740920:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880516.740923:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880516.740925:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000487fae38. 00000100:00000200:1.0:1788880516.740928:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780664192, offset 0 00000400:00000200:1.0:1788880516.740933:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880516.740939:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880516.740944:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880516.740947:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880516.740950:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880516.740953:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880516.740956:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [263:263:256:4294967295] 192.168.204.111@tcp LPNI seq info [263:263:8:4294967295] 00000400:00000200:1.0:1788880516.740963:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880516.740968:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880516.740972:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000800:00000200:1.0:1788880516.740976:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880516.740990:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880516.740996:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 424 niov 1 nkiov 1 00000100:00000001:1.0:1788880516.741016:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.741020:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306152917 00000100:00000001:1.0:1788880516.741025:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306152914 00000100:00000001:1.0:1788880516.741029:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306152917 00000100:00000001:1.0:1788880516.741032:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306152914 00000100:00000001:1.0:1788880516.741035:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306152917 00000100:00000001:1.0:1788880516.741039:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306152914 00000100:00000001:1.0:1788880516.741042:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306152917 00000100:00000001:1.0:1788880516.741044:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306152914 00000100:00000001:1.0:1788880516.741046:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880516.741049:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.741051:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880516.741053:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880516.741055:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 00000000edab0b56 going to sleep for 11 seconds 00000100:00000001:1.0:1788880516.741058:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880516.741059:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880516.741097:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880516.741101:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:3.0:1788880516.741103:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880516.741106:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880516.741108:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000487fae38 00000400:00000010:3.0:1788880516.741109:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000487fae38. 00000100:00000001:3.0:1788880516.741112:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880516.741114:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880516.741120:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880516.741121:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880516.741125:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880516.741126:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880516.742285:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.742292:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880516.742295:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.742297:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.742304:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880516.742312:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b07d80 00000400:00000200:0.0:1788880516.742317:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0x90d [1] + 224 00000800:00000001:0.0:1788880516.742321:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.742326:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.742327:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.742331:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880516.742334:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880516.742338:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880516.742346:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.742361:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880516.742368:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.742371:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.742380:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880516.742385:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880516.742387:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880516.742388:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a3419a3. 00000400:00000200:1.0:1788880516.742391:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:1.0:1788880516.742392:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:1.0:1788880516.742394:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880516.742397:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880516.742404:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.742408:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880516.742409:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.742411:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880516.742413:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880516.742414:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880516.742420:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.742422:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880516.742426:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=44 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880516.742428:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=44 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880516.742430:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880516.742431:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.742432:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880516.742433:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.742435:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880516.742437:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.742441:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880516.742443:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880516.742444:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880516.742446:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880516.742452:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664192:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880516.742456:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.742458:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.742459:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880516.742460:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.742462:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.742466:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880516.742468:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000edab0b56. 00000100:00000001:1.0:1788880516.742483:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880516.742485:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.742487:0:203401:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:1.0:1788880516.742490:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880516.742491:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880516.742493:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000004fb52558 refcount=4 00000020:00000001:1.0:1788880516.742494:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000001:1.0:1788880516.742496:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000040:1.0:1788880516.742504:0:203401:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 000000004fb52558, remote cookie: 0x92f11317f60a7bda, flags: 0x0 00010000:00000001:1.0:1788880516.742506:0:203401:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:1.0:1788880516.742509:0:203401:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:1.0:1788880516.742510:0:203401:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:1.0:1788880516.742512:0:203401:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:1.0:1788880516.742513:0:203401:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:1.0:1788880516.742515:0:203401:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x0:0x0].0x0 (00000000b425dd0a) refcount = 1 00010000:00010000:1.0:1788880516.742517:0:203401:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880516.742521:0:203401:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:1.0:1788880516.742523:0:203401:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:1.0:1788880516.742524:0:203401:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788880516.742526:0:203401:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:1.0:1788880516.742528:0:203401:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880516.742530:0:203401:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880516.742533:0:203401:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:1.0:1788880516.742535:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880516.742536:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880516.742537:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880516.742537:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880516.742539:0:203401:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.742540:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.742542:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780664192/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.742546:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880516.742547:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880516.742549:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000845fe3a7. 02000000:00000001:1.0:1788880516.742551:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880516.742553:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880516.742554:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880516.742556:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880516.742557:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000ab1d1f2f. 02000000:00000001:1.0:1788880516.742563:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880516.742565:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880516.742566:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:1.0:1788880516.742569:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880516.742570:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:1.0:1788880516.742571:0:203401:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880516.742573:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880516.742574:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-client (00000000c2974351) refs 3 10000000:00000001:1.0:1788880516.742575:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.742577:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880516.742578:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880516.742579:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000004fb52558 refcount=3 00000020:00000001:1.0:1788880516.742581:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000001:1.0:1788880516.742582:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000001:1.0:1788880516.742583:0:203401:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:1.0:1788880516.742584:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880516.742585:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880516.742586:0:203401:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880516.742588:0:203401:0:(mgc_request.c:1668:mgc_process_cfg_log()) Process entered 10000000:00000010:1.0:1788880516.742590:0:203401:0:(mgc_request.c:1682:mgc_process_cfg_log()) kmalloced '(env)': 56 at 0000000088249268. 00000020:00000010:1.0:1788880516.742592:0:203401:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000ab1d1f2f. 00000020:00000010:1.0:1788880516.742595:0:203401:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000e28573ff. 00000040:00000010:1.0:1788880516.742599:0:203401:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 000000008e83276d. 00000020:00000010:1.0:1788880516.742603:0:203401:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 000000005257fdb2. 00000020:00000010:1.0:1788880516.742607:0:203401:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 000000005578e134. 80000000:00000010:1.0:1788880516.742611:0:203401:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 0000000069720654. 00040000:00000010:1.0:1788880516.742616:0:203401:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000cd5d51f4. 20000000:00000010:1.0:1788880516.742620:0:203401:0:(mgs_handler.c:1787:mgs_key_init()) kmalloced '(value)': 392 at 000000005d52ca5c. 10000000:00000040:1.0:1788880516.742623:0:203401:0:(lustre_log.h:387:llog_ctxt_get()) GETting ctxt 000000006d5d0799 : new refcount 2 00000020:00000001:1.0:1788880516.742626:0:203401:0:(obd_config.c:2120:class_config_parse_llog()) Process entered 00000020:00000040:1.0:1788880516.742629:0:203401:0:(obd_config.c:2122:class_config_parse_llog()) looking up llog lustre-client 00000040:00000001:1.0:1788880516.742631:0:203401:0:(llog.c:1538:llog_open()) Process entered 00000040:00000010:1.0:1788880516.742634:0:203401:0:(llog.c:42:llog_alloc_handle()) kmalloced '(loghandle)': 352 at 0000000046cece22. 00000040:00000001:1.0:1788880516.742637:0:203401:0:(llog_client.c:70:llog_client_open()) Process entered 00000020:00000040:1.0:1788880516.742639:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788880516.742642:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000020:00000040:1.0:1788880516.742650:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880516.742654:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880516.742657:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880516.742659:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880516.742663:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000001be0e14c. 00000100:00000001:1.0:1788880516.742667:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.742672:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880516.742673:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880516.742676:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000edab0b56. 00000100:00000001:1.0:1788880516.742678:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063674752 : -126823645876864 : ffff8ca791a36d80) 00000100:00000001:1.0:1788880516.742681:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.742682:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000001:1.0:1788880516.742686:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880516.742688:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.742690:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880516.742691:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880516.742695:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.742702:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880516.742703:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.742707:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880516.742709:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880516.742711:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880516.742714:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664320:192.168.204.111@tcp:501: 00000100:00000001:1.0:1788880516.742717:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880516.742721:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880516.742722:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.742724:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880516.742727:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b2a347c7. 02000000:00000001:1.0:1788880516.742728:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880516.742731:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a3419a3. 00000400:00000010:1.0:1788880516.742733:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:1.0:1788880516.742737:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780664320, portal 25 00000100:00000001:1.0:1788880516.742738:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.742742:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000040:1.0:1788880516.742746:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.742751:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880516.742753:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880516.742755:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000487fae38. 00000100:00000200:1.0:1788880516.742757:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 512 bytes to portal 26, xid 1875776780664320, offset 0 00000400:00000200:1.0:1788880516.742762:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880516.742768:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880516.742772:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880516.742774:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880516.742777:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880516.742779:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880516.742783:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [264:264:256:4294967295] 192.168.204.111@tcp LPNI seq info [264:264:8:4294967295] 00000400:00000200:1.0:1788880516.742791:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880516.742795:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 512 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880516.742798:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000800:00000200:1.0:1788880516.742803:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880516.742809:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880516.742812:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 608 niov 1 nkiov 1 00000100:00000001:1.0:1788880516.742823:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.742826:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306152919 00000100:00000001:1.0:1788880516.742833:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306152917 00000100:00000001:1.0:1788880516.742838:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306152919 00000100:00000001:1.0:1788880516.742842:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306152917 00000100:00000001:1.0:1788880516.742845:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306152919 00000100:00000001:1.0:1788880516.742848:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306152917 00000100:00000001:1.0:1788880516.742852:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306152919 00000100:00000001:1.0:1788880516.742856:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306152917 00000100:00000001:1.0:1788880516.742862:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880516.742864:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.742866:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880516.742867:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880516.742868:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 00000000edab0b56 going to sleep for 11 seconds 00000100:00000001:1.0:1788880516.742870:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880516.742871:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880516.742881:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880516.742885:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:0.0:1788880516.742888:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.742893:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880516.742896:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000487fae38 00000400:00000010:0.0:1788880516.742897:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000487fae38. 00000100:00000001:0.0:1788880516.742900:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880516.742903:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.742929:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880516.742931:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.742935:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880516.742936:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880516.743871:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.743878:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880516.743880:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.743881:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880516.743886:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880516.743894:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 272 into portal 25 MB=0x6aa0250b07e00 00000400:00000200:3.0:1788880516.743898:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 272/272 into md 0x915 [1] + 224 00000800:00000001:3.0:1788880516.743901:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.743904:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880516.743906:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880516.743915:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880516.743920:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880516.743924:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880516.743939:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=272 offset=224 replen=272 req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880516.743954:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880516.743960:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.743963:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.743989:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880516.743995:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880516.743997:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880516.743998:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a3419a3. 00000400:00000200:1.0:1788880516.744002:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:1.0:1788880516.744005:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:1.0:1788880516.744008:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880516.744012:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880516.744022:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.744029:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880516.744031:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.744033:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880516.744035:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880516.744037:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880516.744046:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.744049:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880516.744053:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=44 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880516.744056:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=44 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880516.744058:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880516.744059:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.744061:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880516.744063:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.744066:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880516.744069:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.744076:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880516.744084:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880516.744086:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880516.744090:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880516.744097:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664320:192.168.204.111@tcp:501: 00000100:00000001:1.0:1788880516.744101:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.744110:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.744112:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880516.744114:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.744117:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.744123:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880516.744125:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000edab0b56. 00000100:00000001:1.0:1788880516.744128:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880516.744129:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880516.744133:0:203401:0:(llog_client.c:122:llog_client_open()) Process leaving 00000020:00000001:1.0:1788880516.744135:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880516.744137:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880516.744139:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880516.744141:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.744143:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780664320/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.744149:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880516.744151:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880516.744152:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b2a347c7. 02000000:00000001:1.0:1788880516.744155:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880516.744157:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880516.744158:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880516.744159:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880516.744161:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000001be0e14c. 02000000:00000001:1.0:1788880516.744164:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880516.744165:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880516.744167:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:1.0:1788880516.744169:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880516.744175:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000040:00000001:1.0:1788880516.744177:0:203401:0:(llog.c:1563:llog_open()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880516.744180:0:203401:0:(llog.c:415:llog_init_handle()) Process entered 00000040:00000010:1.0:1788880516.744185:0:203401:0:(llog.c:420:llog_init_handle()) kmalloced '(llh)': 8192 at 00000000807cae5b. 00000040:00000001:1.0:1788880516.744187:0:203401:0:(llog.c:363:llog_read_header()) Process entered 00000040:00000001:1.0:1788880516.744189:0:203401:0:(llog_client.c:269:llog_client_read_header()) Process entered 00000020:00000040:1.0:1788880516.744191:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788880516.744195:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000020:00000040:1.0:1788880516.744198:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880516.744202:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880516.744204:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880516.744207:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880516.744212:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000001be0e14c. 00000100:00000001:1.0:1788880516.744216:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.744219:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880516.744221:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880516.744258:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000edab0b56. 00000100:00000001:1.0:1788880516.744261:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063674752 : -126823645876864 : ffff8ca791a36d80) 00000100:00000001:1.0:1788880516.744264:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.744266:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000001:1.0:1788880516.744270:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880516.744272:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.744274:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880516.744276:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880516.744280:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.744288:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880516.744289:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.744292:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880516.744294:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880516.744297:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880516.744301:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664448:192.168.204.111@tcp:503: 00000100:00000001:1.0:1788880516.744304:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880516.744308:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880516.744310:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.744317:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880516.744329:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 16384 at 00000000c7e198ec. 02000000:00000001:1.0:1788880516.744332:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880516.744335:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a3419a3. 00000400:00000010:1.0:1788880516.744340:0:203401:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 00000000afd8648c. 00000100:00000200:1.0:1788880516.744344:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 16384 bytes, xid 1875776780664448, portal 25 00000100:00000001:1.0:1788880516.744346:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.744347:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000040:1.0:1788880516.744351:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.744358:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880516.744361:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880516.744364:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:1.0:1788880516.744366:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 272 bytes to portal 26, xid 1875776780664448, offset 0 00000400:00000200:1.0:1788880516.744372:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880516.744378:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880516.744383:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880516.744387:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880516.744390:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880516.744393:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880516.744396:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [265:265:256:4294967295] 192.168.204.111@tcp LPNI seq info [265:265:8:4294967295] 00000400:00000200:1.0:1788880516.744408:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880516.744413:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 272 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880516.744417:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000800:00000200:1.0:1788880516.744422:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880516.744428:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880516.744431:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 368 niov 1 nkiov 1 00000100:00000001:1.0:1788880516.744443:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.744447:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306152921 00000100:00000001:1.0:1788880516.744452:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306152919 00000100:00000001:1.0:1788880516.744455:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306152921 00000100:00000001:1.0:1788880516.744464:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306152919 00000100:00000001:1.0:1788880516.744468:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306152921 00000100:00000001:1.0:1788880516.744483:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306152919 00000100:00000001:1.0:1788880516.744488:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306152921 00000100:00000001:1.0:1788880516.744492:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306152919 00000100:00000001:1.0:1788880516.744495:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880516.744501:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.744504:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880516.744506:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880516.744508:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 00000000edab0b56 going to sleep for 11 seconds 00000100:00000001:1.0:1788880516.744511:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880516.744513:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880516.744538:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880516.744542:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:0.0:1788880516.744546:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.744553:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880516.744556:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:0.0:1788880516.744558:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:0.0:1788880516.744562:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880516.744566:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.744575:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880516.744578:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880532 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.744584:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880516.744586:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880516.745628:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.745636:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880516.745639:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.745642:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.745648:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880516.745657:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 8416 into portal 25 MB=0x6aa0250b07e80 00000400:00000200:0.0:1788880516.745663:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 8416/8416 into md 0x91d [4] + 224 00000800:00000001:0.0:1788880516.745667:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.745674:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.745677:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.745680:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880516.745684:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880516.745694:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880532 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880516.745703:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=8416 offset=224 replen=8416 req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.745718:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880516.745725:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.745728:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.745747:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880516.745752:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880516.745754:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880516.745755:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a3419a3. 00000400:00000200:1.0:1788880516.745759:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000afd8648c 00000400:00000010:1.0:1788880516.745760:0:203401:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 00000000afd8648c. 00000100:00000001:1.0:1788880516.745763:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880516.745767:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880516.745775:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.745781:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880516.745782:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.745785:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880516.745786:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880516.745792:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880516.745794:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.745796:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880516.745800:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=44 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880516.745803:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=44 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880516.745805:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880516.745812:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.745814:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880516.745816:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.745818:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880516.745821:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.745828:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880516.745831:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880516.745832:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880516.745836:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880532 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880516.745843:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664448:192.168.204.111@tcp:503: 00000100:00000001:1.0:1788880516.745847:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.745849:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.745850:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880516.745852:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.745854:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880532 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.745859:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880516.745861:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000edab0b56. 00000100:00000001:1.0:1788880516.745864:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880516.745865:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880516.745871:0:203401:0:(llog_client.c:319:llog_client_read_header()) Process leaving 00000100:00000001:1.0:1788880516.745872:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.745874:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780664448/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880532 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.745880:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880516.745881:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880516.745882:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 16384 at 00000000c7e198ec. 02000000:00000001:1.0:1788880516.745890:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880516.745892:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880516.745894:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880516.745896:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880516.745901:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000001be0e14c. 02000000:00000001:1.0:1788880516.745905:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880516.745907:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880516.745908:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:1.0:1788880516.745911:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880516.745912:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880516.745914:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880516.745915:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880516.745917:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000040:00000001:1.0:1788880516.745919:0:203401:0:(llog.c:403:llog_read_header()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880516.745921:0:203401:0:(llog.c:481:llog_init_handle()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880516.745924:0:203401:0:(llog.c:951:llog_process_or_fork()) Process entered 00000040:00000010:1.0:1788880516.745926:0:203401:0:(llog.c:953:llog_process_or_fork()) kmalloced '(lpi)': 96 at 00000000d9f2f285. 00000040:00001000:1.0:1788880516.745928:0:203401:0:(llog.c:968:llog_process_or_fork()) Processing [0xa:0x3:0x0] flags 0x004 startcat -1 startidx -1 first_idx 41 last_idx 0 read_mode 0 00000040:00000001:1.0:1788880516.746170:0:205883:0:(llog.c:552:llog_process_thread()) Process entered 00000020:00000010:1.0:1788880516.746175:0:205883:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000001be0e14c. 00000020:00000010:1.0:1788880516.746180:0:205883:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000b2a347c7. 00000040:00000010:1.0:1788880516.746184:0:205883:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000bd87bd8d. 00000020:00000010:1.0:1788880516.746187:0:205883:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 000000007f7d1e46. 00000020:00000010:1.0:1788880516.746192:0:205883:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 000000008f8293fd. 00000020:00000010:1.0:1788880516.746196:0:205883:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 000000009e1ad299. 80000000:00000010:1.0:1788880516.746199:0:205883:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 0000000045676c6b. 40000000:00000010:1.0:1788880516.746202:0:205883:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 0000000032e6aa73. 00100000:00000010:1.0:1788880516.746207:0:205883:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 00000000453588fe. 00040000:00000010:1.0:1788880516.746211:0:205883:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 0000000007660b3e. 00040000:00000010:1.0:1788880516.746215:0:205883:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 00000000a71f1a29. 00040000:00000010:1.0:1788880516.746218:0:205883:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 000000008d264e32. 00000004:00000010:1.0:1788880516.746222:0:205883:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 00000000a3e174ee. 00000004:00000010:1.0:1788880516.746226:0:205883:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 0000000013189f47. 00000004:00000010:1.0:1788880516.746230:0:205883:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 000000006f45b8ae. 00002000:00000010:1.0:1788880516.746251:0:205883:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 00000000845fe3a7. 00000004:00000010:1.0:1788880516.746254:0:205883:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 000000009ef860f0. 00000020:00000010:1.0:1788880516.746257:0:205883:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 000000007bbc051d. 00000040:00000010:1.0:1788880516.746264:0:205883:0:(llog.c:578:llog_process_thread()) kmalloced '(buf)': 8192 at 0000000073d230ef. 00000040:00001000:1.0:1788880516.746267:0:205883:0:(llog.c:615:llog_process_thread()) index: 42 last_index 64767 00000040:00000001:1.0:1788880516.746273:0:205883:0:(lustre_log.h:475:llog_next_block()) Process entered 00000040:00000001:1.0:1788880516.746274:0:205883:0:(llog_client.c:140:llog_client_next_block()) Process entered 00000020:00000040:1.0:1788880516.746276:0:205883:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788880516.746278:0:205883:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000020:00000040:1.0:1788880516.746279:0:205883:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880516.746281:0:205883:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880516.746283:0:205883:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880516.746285:0:205883:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880516.746288:0:205883:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000004ffd965d. 00000100:00000001:1.0:1788880516.746292:0:205883:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.746295:0:205883:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880516.746296:0:205883:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880516.746299:0:205883:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000afd8648c. 00000100:00000001:1.0:1788880516.746300:0:205883:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063672064 : -126823645879552 : ffff8ca791a36300) 00000100:00000001:1.0:1788880516.746302:0:205883:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.746303:0:205883:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000001:1.0:1788880516.746306:0:205883:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880516.746307:0:205883:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.746308:0:205883:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880516.746309:0:205883:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880516.746313:0:205883:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.746319:0:205883:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880516.746320:0:205883:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.746322:0:205883:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880516.746323:0:205883:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880516.746325:0:205883:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880516.746327:0:205883:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job llog_process_th:fa4856d5-51a8-499e-b3f8-b477743a3dec:205883:1875776780664576:192.168.204.111@tcp:502: 00000100:00000001:1.0:1788880516.746330:0:205883:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880516.746332:0:205883:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880516.746334:0:205883:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.746335:0:205883:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880516.746343:0:205883:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 16384 at 00000000c7e198ec. 02000000:00000001:1.0:1788880516.746344:0:205883:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880516.746349:0:205883:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a3419a3. 00000400:00000010:1.0:1788880516.746352:0:205883:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 00000000fa054719. 00000100:00000200:1.0:1788880516.746354:0:205883:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 16384 bytes, xid 1875776780664576, portal 25 00000100:00000001:1.0:1788880516.746356:0:205883:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880516.746356:0:205883:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000040:1.0:1788880516.746359:0:205883:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.746363:0:205883:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880516.746364:0:205883:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880516.746366:0:205883:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000ae22117. 00000100:00000200:1.0:1788880516.746367:0:205883:0:(niobuf.c:94:ptl_send_buf()) Sending 272 bytes to portal 26, xid 1875776780664576, offset 0 00000400:00000200:1.0:1788880516.746371:0:205883:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880516.746376:0:205883:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880516.746379:0:205883:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880516.746381:0:205883:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880516.746383:0:205883:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880516.746385:0:205883:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880516.746387:0:205883:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [266:266:256:4294967295] 192.168.204.111@tcp LPNI seq info [266:266:8:4294967295] 00000400:00000200:1.0:1788880516.746393:0:205883:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880516.746396:0:205883:0:(socklnd_cb.c:931:ksocknal_send()) sending 272 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880516.746398:0:205883:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000800:00000200:1.0:1788880516.746401:0:205883:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880516.746406:0:205883:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880516.746408:0:205883:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 368 niov 1 nkiov 1 00000100:00000001:1.0:1788880516.746416:0:205883:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.746418:0:205883:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306152923 00000100:00000001:1.0:1788880516.746422:0:205883:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306152921 00000100:00000001:1.0:1788880516.746424:0:205883:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306152923 00000100:00000001:1.0:1788880516.746426:0:205883:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306152921 00000100:00000001:1.0:1788880516.746428:0:205883:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306152923 00000100:00000001:1.0:1788880516.746434:0:205883:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306152921 00000100:00000001:1.0:1788880516.746437:0:205883:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306152923 00000100:00000001:1.0:1788880516.746439:0:205883:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306152921 00000100:00000001:1.0:1788880516.746442:0:205883:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880516.746444:0:205883:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.746445:0:205883:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880516.746447:0:205883:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880516.746448:0:205883:0:(client.c:2702:ptlrpc_set_wait()) set 00000000afd8648c going to sleep for 11 seconds 00000100:00000001:1.0:1788880516.746450:0:205883:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880516.746451:0:205883:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880516.746500:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880516.746504:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:3.0:1788880516.746508:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880516.746513:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880516.746516:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000ae22117 00000400:00000010:3.0:1788880516.746518:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000ae22117. 00000100:00000001:3.0:1788880516.746521:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880516.746524:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880516.746533:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880516.746535:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880532 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880516.746542:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880516.746543:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880516.747871:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.747883:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880516.747887:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.747890:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.747897:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880516.747908:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 8472 into portal 25 MB=0x6aa0250b07f00 00000400:00000200:0.0:1788880516.747915:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 8472/8472 into md 0x925 [4] + 224 00000800:00000001:0.0:1788880516.747921:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.747985:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.747988:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.747992:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880516.747998:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880516.748003:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880532 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880516.748012:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=8472 offset=224 replen=8472 req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.748028:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880516.748040:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.748043:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.748061:0:205883:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880516.748071:0:205883:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880516.748072:0:205883:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880516.748073:0:205883:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a3419a3. 00000400:00000200:1.0:1788880516.748076:0:205883:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000fa054719 00000400:00000010:1.0:1788880516.748077:0:205883:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 00000000fa054719. 00000100:00000001:1.0:1788880516.748079:0:205883:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880516.748083:0:205883:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880516.748088:0:205883:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.748092:0:205883:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880516.748093:0:205883:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.748095:0:205883:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880516.748097:0:205883:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880516.748098:0:205883:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880516.748099:0:205883:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880516.748101:0:205883:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880516.748105:0:205883:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=44 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880516.748109:0:205883:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=44 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880516.748111:0:205883:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880516.748112:0:205883:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.748114:0:205883:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880516.748116:0:205883:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.748125:0:205883:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880516.748129:0:205883:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.748136:0:205883:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880516.748139:0:205883:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880516.748140:0:205883:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880516.748143:0:205883:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880532 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880516.748150:0:205883:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job llog_process_th:fa4856d5-51a8-499e-b3f8-b477743a3dec:205883:1875776780664576:192.168.204.111@tcp:502: 00000100:00000001:1.0:1788880516.748155:0:205883:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880516.748156:0:205883:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880516.748158:0:205883:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880516.748160:0:205883:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.748161:0:205883:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880532 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.748165:0:205883:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880516.748167:0:205883:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000afd8648c. 00000100:00000001:1.0:1788880516.748169:0:205883:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880516.748170:0:205883:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880516.748175:0:205883:0:(llog_client.c:198:llog_client_next_block()) Process leaving 00000100:00000001:1.0:1788880516.748176:0:205883:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880516.748178:0:205883:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780664576/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880532 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880516.748182:0:205883:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880516.748183:0:205883:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880516.748185:0:205883:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 16384 at 00000000c7e198ec. 02000000:00000001:1.0:1788880516.748191:0:205883:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880516.748193:0:205883:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880516.748194:0:205883:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880516.748195:0:205883:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880516.748196:0:205883:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000004ffd965d. 02000000:00000001:1.0:1788880516.748199:0:205883:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880516.748200:0:205883:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880516.748204:0:205883:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:1.0:1788880516.748206:0:205883:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880516.748207:0:205883:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880516.748208:0:205883:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880516.748209:0:205883:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880516.748210:0:205883:0:(genops.c:1251:class_import_put()) Process leaving 00000040:00000001:1.0:1788880516.748211:0:205883:0:(lustre_log.h:485:llog_next_block()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:1.0:1788880516.748213:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022000 type=0x10620000 idx=1 00000040:00001000:1.0:1788880516.748216:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 1 00000040:00001000:1.0:1788880516.748217:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0220e0 type=0x10620000 idx=2 00000040:00001000:1.0:1788880516.748218:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 2 00000040:00001000:1.0:1788880516.748218:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022158 type=0x10620000 idx=3 00000040:00001000:1.0:1788880516.748219:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 3 00000040:00001000:1.0:1788880516.748220:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022200 type=0x10620000 idx=4 00000040:00001000:1.0:1788880516.748220:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 4 00000040:00001000:1.0:1788880516.748221:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0222e0 type=0x10620000 idx=5 00000040:00001000:1.0:1788880516.748222:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 5 00000040:00001000:1.0:1788880516.748228:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0223c0 type=0x10620000 idx=6 00000040:00001000:1.0:1788880516.748229:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 6 00000040:00001000:1.0:1788880516.748230:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022438 type=0x10620000 idx=7 00000040:00001000:1.0:1788880516.748231:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 7 00000040:00001000:1.0:1788880516.748264:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0224e0 type=0x10620000 idx=8 00000040:00001000:1.0:1788880516.748267:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 8 00000040:00001000:1.0:1788880516.748268:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0225c0 type=0x10620000 idx=9 00000040:00001000:1.0:1788880516.748270:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 9 00000040:00001000:1.0:1788880516.748271:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0226a0 type=0x10620000 idx=10 00000040:00001000:1.0:1788880516.748273:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 10 00000040:00001000:1.0:1788880516.748275:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022718 type=0x10620000 idx=11 00000040:00001000:1.0:1788880516.748276:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 11 00000040:00001000:1.0:1788880516.748278:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0227f8 type=0x10620000 idx=12 00000040:00001000:1.0:1788880516.748279:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 12 00000040:00001000:1.0:1788880516.748280:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0228d8 type=0x10620000 idx=13 00000040:00001000:1.0:1788880516.748281:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 13 00000040:00001000:1.0:1788880516.748283:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022930 type=0x10620000 idx=14 00000040:00001000:1.0:1788880516.748284:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 14 00000040:00001000:1.0:1788880516.748284:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0229b0 type=0x10620000 idx=15 00000040:00001000:1.0:1788880516.748290:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 15 00000040:00001000:1.0:1788880516.748291:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022a40 type=0x10620000 idx=16 00000040:00001000:1.0:1788880516.748291:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 16 00000040:00001000:1.0:1788880516.748292:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022ae0 type=0x10620000 idx=17 00000040:00001000:1.0:1788880516.748293:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 17 00000040:00001000:1.0:1788880516.748294:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022bc0 type=0x10620000 idx=18 00000040:00001000:1.0:1788880516.748294:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 18 00000040:00001000:1.0:1788880516.748295:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022ca0 type=0x10620000 idx=19 00000040:00001000:1.0:1788880516.748296:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 19 00000040:00001000:1.0:1788880516.748296:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022cf0 type=0x10620000 idx=20 00000040:00001000:1.0:1788880516.748297:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 20 00000040:00001000:1.0:1788880516.748298:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022dd0 type=0x10620000 idx=21 00000040:00001000:1.0:1788880516.748298:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 21 00000040:00001000:1.0:1788880516.748299:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022eb0 type=0x10620000 idx=22 00000040:00001000:1.0:1788880516.748300:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 22 00000040:00001000:1.0:1788880516.748300:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022f08 type=0x10620000 idx=23 00000040:00001000:1.0:1788880516.748301:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 23 00000040:00001000:1.0:1788880516.748302:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be022f88 type=0x10620000 idx=24 00000040:00001000:1.0:1788880516.748302:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 24 00000040:00001000:1.0:1788880516.748303:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023018 type=0x10620000 idx=25 00000040:00001000:1.0:1788880516.748304:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 25 00000040:00001000:1.0:1788880516.748304:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0230b8 type=0x10620000 idx=26 00000040:00001000:1.0:1788880516.748305:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 26 00000040:00001000:1.0:1788880516.748306:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023198 type=0x10620000 idx=27 00000040:00001000:1.0:1788880516.748306:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 27 00000040:00001000:1.0:1788880516.748307:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023278 type=0x10620000 idx=28 00000040:00001000:1.0:1788880516.748308:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 28 00000040:00001000:1.0:1788880516.748308:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0232c8 type=0x10620000 idx=29 00000040:00001000:1.0:1788880516.748309:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 29 00000040:00001000:1.0:1788880516.748310:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0233a8 type=0x10620000 idx=30 00000040:00001000:1.0:1788880516.748310:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 30 00000040:00001000:1.0:1788880516.748311:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023488 type=0x10620000 idx=31 00000040:00001000:1.0:1788880516.748312:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 31 00000040:00001000:1.0:1788880516.748312:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0234e0 type=0x10620000 idx=32 00000040:00001000:1.0:1788880516.748313:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 32 00000040:00001000:1.0:1788880516.748316:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023560 type=0x10620000 idx=33 00000040:00001000:1.0:1788880516.748317:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 33 00000040:00001000:1.0:1788880516.748318:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0235f0 type=0x10620000 idx=34 00000040:00001000:1.0:1788880516.748319:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 34 00000040:00001000:1.0:1788880516.748320:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023670 type=0x10620000 idx=35 00000040:00001000:1.0:1788880516.748321:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 35 00000040:00001000:1.0:1788880516.748321:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023750 type=0x10620000 idx=36 00000040:00001000:1.0:1788880516.748322:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 36 00000040:00001000:1.0:1788880516.748322:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023830 type=0x10620000 idx=37 00000040:00001000:1.0:1788880516.748323:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 37 00000040:00001000:1.0:1788880516.748324:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023880 type=0x10620000 idx=38 00000040:00001000:1.0:1788880516.748325:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 38 00000040:00001000:1.0:1788880516.748325:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023960 type=0x10620000 idx=39 00000040:00001000:1.0:1788880516.748326:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 39 00000040:00001000:1.0:1788880516.748326:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023a40 type=0x10620000 idx=40 00000040:00001000:1.0:1788880516.748327:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 40 00000040:00001000:1.0:1788880516.748328:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023a98 type=0x10620000 idx=41 00000040:00001000:1.0:1788880516.748329:0:205883:0:(llog.c:767:llog_process_thread()) skipping lrh_index 41 00000040:00001000:1.0:1788880516.748329:0:205883:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be023b18 type=0x10620000 idx=42 00000040:00001000:1.0:1788880516.748330:0:205883:0:(llog.c:784:llog_process_thread()) lrh_index: 42 lrh_len: 144 (1256 remains) 00000040:00001000:1.0:1788880516.748332:0:205883:0:(llog.c:805:llog_process_thread()) index: 42, lh_last_idx: 57 synced_idx: 0 lgh_last_idx: 57 00000020:00000001:1.0:1788880516.748334:0:205883:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:1.0:1788880516.748339:0:205883:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 39 at 0000000022045bfb. 00000020:01000000:1.0:1788880516.748341:0:205883:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf003, instance name: lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000010:1.0:1788880516.748344:0:205883:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 136 at 00000000afd8648c. 00000020:00000080:1.0:1788880516.748346:0:205883:0:(obd_config.c:1470:class_process_config()) processing cmd: cf003 00000020:00000001:1.0:1788880516.748351:0:205883:0:(obd_config.c:742:class_setup()) Process entered 00000020:00020000:1.0:1788880516.748353:0:205883:0:(obd_config.c:761:class_setup()) mdc: device 4 already setup: rc = -17 00000020:00000001:1.0:1788880516.751169:0:205883:0:(obd_config.c:762:class_setup()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00000020:00000001:1.0:1788880516.751176:0:205883:0:(obd_config.c:1618:class_process_config()) Process leaving via out (rc=18446744073709551599 : -17 : 0xffffffffffffffef) 00000020:00000080:1.0:1788880516.751180:0:205883:0:(obd_config.c:1470:class_process_config()) processing cmd: cf002 00000020:00000001:1.0:1788880516.751187:0:205883:0:(obd_config.c:863:class_detach()) Process entered 00000020:00020000:1.0:1788880516.751189:0:205883:0:(obd_config.c:866:class_detach()) OBD device 4 still set up 00000020:00000001:1.0:1788880516.753768:0:205883:0:(obd_config.c:867:class_detach()) Process leaving (rc=18446744073709551600 : -16 : fffffffffffffff0) 00000020:00000001:1.0:1788880516.753792:0:205883:0:(obd_config.c:1622:class_process_config()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000010:1.0:1788880516.753795:0:205883:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 136 at 00000000afd8648c. 00000020:00000010:1.0:1788880516.753802:0:205883:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 39 at 0000000022045bfb. 00000020:00020000:1.0:1788880516.753804:0:205883:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.111@tcp: cfg command failed: rc = -17 00000020:00000001:1.0:1788880516.758008:0:205883:0:(obd_config.c:2423:class_config_dump_handler()) Process entered 00000020:00000010:1.0:1788880516.758021:0:205883:0:(obd_config.c:2425:class_config_dump_handler()) kmalloced '(outstr)': 256 at 00000000d5454cc1. 00000020:00000001:1.0:1788880516.758028:0:205883:0:(obd_config.c:2354:class_config_parse_rec()) Process entered 00000020:00000001:1.0:1788880516.758033:0:205883:0:(obd_config.c:2413:class_config_parse_rec()) Process leaving (rc=79 : 79 : 4f) 00000020:02000400:1.0:1788880516.758035:0:205883:0:(obd_config.c:2431:class_config_dump_handler()) cmd=cf003 0:lustre-MDT0001-mdc 1:lustre-MDT0001_UUID 2:192.168.204.111@tcp 00000020:00000010:1.0:1788880516.760783:0:205883:0:(obd_config.c:2437:class_config_dump_handler()) kfreed 'outstr': 256 at 00000000d5454cc1. 00000020:00000001:1.0:1788880516.760796:0:205883:0:(obd_config.c:2438:class_config_dump_handler()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880516.760799:0:205883:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00000040:00000001:1.0:1788880516.760804:0:205883:0:(llog.c:843:llog_process_thread()) Process leaving via out (rc=18446744073709551599 : -17 : 0xffffffffffffffef) 00000040:00080000:1.0:1788880516.760807:0:205883:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 42 count 57 00000040:00000010:1.0:1788880516.760810:0:205883:0:(llog.c:899:llog_process_thread()) kvfree 'buf': 8192 at 0000000073d230ef. 00000020:00000010:1.0:1788880516.760818:0:205883:0:(lu_object.c:2058:lu_env_remove()) kfreed_rcu 'lei': 40 at 000000007bbc051d. 00000020:00000010:1.0:1788880516.760836:0:205883:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000b2a347c7. 00000040:00000010:1.0:1788880516.760843:0:205883:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000bd87bd8d. 00000020:00000010:1.0:1788880516.760845:0:205883:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 000000007f7d1e46. 00000020:00000010:1.0:1788880516.760849:0:205883:0:(tgt_main.c:741:tgt_key_fini()) slab-freed '(info)': 1288 at 000000008f8293fd. 00000020:00000010:1.0:1788880516.760854:0:205883:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 000000009e1ad299. 80000000:00000010:1.0:1788880516.760859:0:205883:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 0000000045676c6b. 40000000:00000010:1.0:1788880516.760862:0:205883:0:(fid_handler.c:479:seq_key_fini()) kfreed 'info': 48 at 0000000032e6aa73. 00100000:00000010:1.0:1788880516.760867:0:205883:0:(lfsck_lib.c:43:lfsck_key_fini()) kfreed 'info': 1800 at 00000000453588fe. 00040000:00000010:1.0:1788880516.760871:0:205883:0:(lquota_lib.c:35:lquota_key_fini()) kfreed 'info': 248 at 0000000007660b3e. 00040000:00000010:1.0:1788880516.760875:0:205883:0:(qmt_dev.c:418:qmt_key_fini()) kfreed 'info': 1256 at 00000000a71f1a29. 00040000:00000010:1.0:1788880516.760878:0:205883:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 000000008d264e32. 00000004:00000010:1.0:1788880516.760882:0:205883:0:(mdd_device.c:2420:mdd_key_fini()) kfreed 'info': 1872 at 00000000a3e174ee. 00000004:00000010:1.0:1788880516.760886:0:205883:0:(mdt_handler.c:8495:mdt_key_fini()) kfreed 'info': 2568 at 0000000013189f47. 00000004:00000010:1.0:1788880516.760892:0:205883:0:(lod_dev.c:2378:lod_key_fini()) kfreed 'info': 1224 at 000000006f45b8ae. 00002000:00000010:1.0:1788880516.760895:0:205883:0:(ofd_dev.c:3002:ofd_key_fini()) kfreed 'info': 816 at 00000000845fe3a7. 00000004:00000010:1.0:1788880516.760899:0:205883:0:(osp_dev.c:1888:osp_key_fini()) kfreed 'info': 616 at 000000009ef860f0. 00000020:00000010:1.0:1788880516.760902:0:205883:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 000000001be0e14c. 00000040:00000010:2.0:1788880516.760958:0:203401:0:(llog.c:994:llog_process_or_fork()) kfreed 'lpi': 96 at 00000000d9f2f285. 00000040:00000001:2.0:1788880516.760965:0:203401:0:(llog.c:995:llog_process_or_fork()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00000020:01000000:2.0:1788880516.760969:0:203401:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 42-42 (rc=-17) 00000040:00000010:2.0:1788880516.760988:0:203401:0:(llog.c:70:llog_free_handle()) kvfree 'loghandle->lgh_hdr': 8192 at 00000000807cae5b. 00000040:00000010:2.0:1788880516.760993:0:203401:0:(llog.c:72:llog_free_handle()) kfreed 'loghandle': 352 at 0000000046cece22. 00000020:00000001:2.0:1788880516.760996:0:203401:0:(obd_config.c:2151:class_config_parse_llog()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 10000000:00000001:2.0:1788880516.761000:0:203401:0:(mgc_request.c:1714:mgc_process_cfg_log()) Process leaving via out_pop (rc=18446744073709551599 : -17 : 0xffffffffffffffef) 00000020:00000010:2.0:1788880516.761004:0:203401:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000e28573ff. 00000040:00000010:2.0:1788880516.761008:0:203401:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 000000008e83276d. 00000020:00000010:2.0:1788880516.761011:0:203401:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 000000005257fdb2. 00000020:00000010:2.0:1788880516.761014:0:203401:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 000000005578e134. 80000000:00000010:2.0:1788880516.761016:0:203401:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 0000000069720654. 00040000:00000010:2.0:1788880516.761018:0:203401:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 00000000cd5d51f4. 20000000:00000010:2.0:1788880516.761022:0:203401:0:(mgs_handler.c:1787:mgs_key_fini()) kfreed 'info': 392 at 000000005d52ca5c. 00000020:00000010:2.0:1788880516.761025:0:203401:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000ab1d1f2f. 10000000:00000010:2.0:1788880516.761027:0:203401:0:(mgc_request.c:1730:mgc_process_cfg_log()) kfreed 'env': 56 at 0000000088249268. 10000000:01000000:2.0:1788880516.761030:0:203401:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'lustre-client' failed (-17). 10000000:00000001:2.0:1788880516.761037:0:203401:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:2.0:1788880516.761040:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:2.0:1788880516.761042:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880516.761045:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000004fb52558 refcount=3 00000020:00000001:2.0:1788880516.761047:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000001:2.0:1788880516.761049:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000001:2.0:1788880516.761051:0:203401:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:2.0:1788880516.761055:0:203401:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880516.761063:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880516.761064:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:2.0:1788880516.761066:0:203401:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880516.761071:0:203401:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:2.0:1788880516.761072:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880516.761073:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:2.0:1788880516.761074:0:203401:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880516.761081:0:203401:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 10000000:00020000:2.0:1788880516.761084:0:203401:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 10000000:00000001:2.0:1788880516.763575:0:203401:0:(mgc_request.c:621:do_requeue()) Process leaving 10000000:00000001:2.0:1788880516.763581:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:2.0:1788880516.763583:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-sptlrpc (000000007b8fd88a) refs 2 10000000:00000001:2.0:1788880516.763588:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880516.763591:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:2.0:1788880516.763592:0:203401:0:(mgc_request.c:123:config_log_put()) log lustre-client(00000000c2974351) refs 3 10000000:00000001:2.0:1788880516.763594:0:203401:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:2.0:1788880516.763595:0:203401:0:(mgc_request.c:601:do_requeue()) Process entered 10000000:01000000:2.0:1788880516.763597:0:203401:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:00000001:2.0:1788880516.763598:0:203401:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:2.0:1788880516.763600:0:203401:0:(mgc_request.c:1801:mgc_process_log()) Process log lustre-sptlrpc-ffff8ca792d4cd40 from 1 10000000:00000001:2.0:1788880516.763602:0:203401:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:2.0:1788880516.763604:0:203401:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00000010:2.0:1788880516.763612:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880516.763616:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880516.763620:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880516.763622:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880516.763626:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880516.763634:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000ee42a892. 00000100:00000001:2.0:1788880516.763641:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880516.763645:0:203401:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:2.0:1788880516.763648:0:203401:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000040:2.0:1788880516.763653:0:203401:0:(ldlm_resource.c:1656:ldlm_resource_getref()) getref res: 00000000b425dd0a count: 2 00010000:00000001:2.0:1788880516.763656:0:203401:0:(ldlm_resource.c:1595:ldlm_resource_get()) Process leaving via found (rc=0 : 0 : 0x0) 00010000:00000001:2.0:1788880516.763659:0:203401:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:2.0:1788880516.763661:0:203401:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 000000003520ed0c. 00000020:00000001:2.0:1788880516.763666:0:203401:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880516.763668:0:203401:0:(lustre_handles.c:93:class_handle_hash()) added object 000000003520ed0c with handle 0x805d548a36a886eb to hash 00000020:00000001:2.0:1788880516.763671:0:203401:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:2.0:1788880516.763671:0:203401:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 00010000:00000001:2.0:1788880516.763674:0:203401:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 00010000:00000001:2.0:1788880516.763677:0:203401:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:2.0:1788880516.763679:0:203401:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880516.763681:0:203401:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788880516.763697:0:203401:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788880516.763705:0:203401:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:2.0:1788880516.763709:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:2.0:1788880516.763711:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:2.0:1788880516.763716:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000018bd9dcf. 00000100:00000001:2.0:1788880516.763718:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250663051648 : -126823046499968 : ffff8ca7b55d3180) 00000100:00000001:2.0:1788880516.763719:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880516.763720:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000100:00000001:2.0:1788880516.763723:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880516.763725:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880516.763726:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:2.0:1788880516.763727:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880516.763735:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.763743:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880516.763745:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880516.763749:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880516.763751:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880516.763753:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880516.763756:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664704:192.168.204.111@tcp:101: 00000100:00000001:2.0:1788880516.763760:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880516.763766:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880516.763768:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880516.763770:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880516.763774:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000005e22417f. 02000000:00000001:2.0:1788880516.763776:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880516.763783:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000400:00000010:2.0:1788880516.763789:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000200:2.0:1788880516.763798:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780664704, portal 25 00000100:00000001:2.0:1788880516.763799:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880516.763800:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000100:00000040:2.0:1788880516.763803:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.763809:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880516.763811:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880516.763813:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008f790e4a. 00000100:00000200:2.0:1788880516.763816:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780664704, offset 0 00000400:00000200:2.0:1788880516.763822:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880516.763834:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880516.763839:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880516.763843:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880516.763846:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880516.763848:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880516.763851:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [267:267:256:4294967295] 192.168.204.111@tcp LPNI seq info [267:267:8:4294967295] 00000400:00000200:2.0:1788880516.763862:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880516.763868:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880516.763873:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000800:00000200:2.0:1788880516.763879:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880516.763898:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880516.763902:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 424 niov 1 nkiov 1 00000100:00000001:2.0:1788880516.763920:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880516.763924:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306152940 00000100:00000001:2.0:1788880516.763931:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306152923 00000100:00000001:2.0:1788880516.763936:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306152940 00000100:00000001:2.0:1788880516.763938:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306152923 00000100:00000001:2.0:1788880516.763941:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306152940 00000100:00000001:2.0:1788880516.763944:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306152923 00000100:00000001:2.0:1788880516.763953:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306152940 00000100:00000001:2.0:1788880516.763956:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306152923 00000100:00000001:2.0:1788880516.763958:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880516.763960:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880516.763962:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880516.763964:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:2.0:1788880516.763965:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 0000000018bd9dcf going to sleep for 11 seconds 00000100:00000001:2.0:1788880516.763968:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880516.763971:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880516.764058:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880516.764062:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000400:00000200:0.0:1788880516.764065:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.764069:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880516.764073:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008f790e4a 00000400:00000010:0.0:1788880516.764074:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008f790e4a. 00000100:00000001:0.0:1788880516.764082:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880516.764085:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.764092:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880516.764094:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.764098:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880516.764099:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880516.765508:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.765519:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880516.765522:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.765525:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880516.765533:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880516.765545:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b07f80 00000400:00000200:3.0:1788880516.765553:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0x92d [1] + 224 00000800:00000001:3.0:1788880516.765559:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.765572:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880516.765574:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880516.765576:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880516.765580:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880516.765588:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880516.765595:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880516.765606:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880516.765612:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880516.765614:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880516.765625:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880516.765631:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880516.765632:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880516.765634:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:2.0:1788880516.765637:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880516.765638:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880516.765640:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880516.765643:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880516.765650:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.765654:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880516.765655:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880516.765657:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880516.765659:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880516.765660:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880516.765662:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880516.765663:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880516.765668:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=44 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880516.765670:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=44 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880516.765672:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880516.765672:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880516.765674:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880516.765676:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880516.765677:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880516.765679:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.765689:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880516.765691:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880516.765692:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880516.765694:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880516.765698:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664704:192.168.204.111@tcp:101: 00000100:00000001:2.0:1788880516.765700:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880516.765702:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880516.765703:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:2.0:1788880516.765704:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880516.765706:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.765709:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880516.765711:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000018bd9dcf. 00000100:00000001:2.0:1788880516.765713:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:2.0:1788880516.765714:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880516.765716:0:203401:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:2.0:1788880516.765719:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:2.0:1788880516.765720:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880516.765722:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000003520ed0c refcount=4 00000020:00000001:2.0:1788880516.765723:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 00010000:00000001:2.0:1788880516.765725:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 00010000:00000040:2.0:1788880516.765729:0:203401:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 000000003520ed0c, remote cookie: 0x92f11317f60a7be1, flags: 0x0 00010000:00000001:2.0:1788880516.765731:0:203401:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:2.0:1788880516.765735:0:203401:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:2.0:1788880516.765737:0:203401:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:2.0:1788880516.765738:0:203401:0:(ldlm_lock.c:1006:search_granted_lock()) Process leaving 00010000:00000001:2.0:1788880516.765739:0:203401:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:2.0:1788880516.765741:0:203401:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x0:0x0].0x0 (00000000b425dd0a) refcount = 2 00010000:00000040:2.0:1788880516.765744:0:203401:0:(ldlm_resource.c:1879:ldlm_resource_dump()) Granted locks (in reverse order): 00010000:00000040:2.0:1788880516.765746:0:203401:0:(ldlm_resource.c:1882:ldlm_resource_dump()) ### ### ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 1/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788880516.765758:0:203401:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880516.765764:0:203401:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:2.0:1788880516.765765:0:203401:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:2.0:1788880516.765767:0:203401:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:2.0:1788880516.765770:0:203401:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:2.0:1788880516.765772:0:203401:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880516.765775:0:203401:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880516.765780:0:203401:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:2.0:1788880516.765782:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880516.765783:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788880516.765784:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880516.765785:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788880516.765786:0:203401:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880516.765788:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880516.765791:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780664704/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880532 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.765795:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880516.765797:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880516.765799:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000005e22417f. 02000000:00000001:2.0:1788880516.765801:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880516.765802:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880516.765803:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880516.765805:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880516.765806:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000ee42a892. 02000000:00000001:2.0:1788880516.765810:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880516.765811:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880516.765812:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:2.0:1788880516.765814:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880516.765815:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:2.0:1788880516.765816:0:203401:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880516.765821:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:2.0:1788880516.765822:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-sptlrpc (000000007b8fd88a) refs 3 10000000:00000001:2.0:1788880516.765823:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880516.765824:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:2.0:1788880516.765825:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880516.765826:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000003520ed0c refcount=3 00000020:00000001:2.0:1788880516.765827:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 00010000:00000001:2.0:1788880516.765829:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 00010000:00000001:2.0:1788880516.765830:0:203401:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:2.0:1788880516.765831:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880516.765831:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788880516.765832:0:203401:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880516.765833:0:203401:0:(mgc_request.c:1668:mgc_process_cfg_log()) Process entered 10000000:00000010:2.0:1788880516.765835:0:203401:0:(mgc_request.c:1682:mgc_process_cfg_log()) kmalloced '(env)': 56 at 00000000a1e4f73a. 00000020:00000010:2.0:1788880516.765838:0:203401:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000ee42a892. 00000020:00000010:2.0:1788880516.765841:0:203401:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000009cadd275. 00000040:00000010:2.0:1788880516.765844:0:203401:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000c21a3ff8. 00000020:00000010:2.0:1788880516.765847:0:203401:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000a8e57426. 00000020:00000010:2.0:1788880516.765851:0:203401:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 0000000078217d0d. 80000000:00000010:2.0:1788880516.765854:0:203401:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000ccadc63b. 00040000:00000010:2.0:1788880516.765858:0:203401:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000feb2a015. 20000000:00000010:2.0:1788880516.765860:0:203401:0:(mgs_handler.c:1787:mgs_key_init()) kmalloced '(value)': 392 at 000000001f6835d8. 10000000:00000040:2.0:1788880516.765863:0:203401:0:(lustre_log.h:387:llog_ctxt_get()) GETting ctxt 000000006d5d0799 : new refcount 2 00000020:00000001:2.0:1788880516.765865:0:203401:0:(obd_config.c:2120:class_config_parse_llog()) Process entered 00000020:00000040:2.0:1788880516.765867:0:203401:0:(obd_config.c:2122:class_config_parse_llog()) looking up llog lustre-sptlrpc 00000040:00000001:2.0:1788880516.765868:0:203401:0:(llog.c:1538:llog_open()) Process entered 00000040:00000010:2.0:1788880516.765870:0:203401:0:(llog.c:42:llog_alloc_handle()) kmalloced '(loghandle)': 352 at 00000000ed94d533. 00000040:00000001:2.0:1788880516.765873:0:203401:0:(llog_client.c:70:llog_client_open()) Process entered 00000020:00000040:2.0:1788880516.765874:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:2.0:1788880516.765876:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880516.765878:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880516.765880:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880516.765882:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880516.765884:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880516.765887:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000000784e3b9. 00000100:00000001:2.0:1788880516.765893:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880516.765896:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:2.0:1788880516.765898:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:2.0:1788880516.765900:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000018bd9dcf. 00000100:00000001:2.0:1788880516.765901:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250663051648 : -126823046499968 : ffff8ca7b55d3180) 00000100:00000001:2.0:1788880516.765902:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880516.765903:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000100:00000001:2.0:1788880516.765906:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880516.765907:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880516.765908:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:2.0:1788880516.765908:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880516.765910:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.765915:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880516.765916:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880516.765917:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880516.765919:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880516.765920:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880516.765921:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664832:192.168.204.111@tcp:501: 00000100:00000001:2.0:1788880516.765923:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880516.765925:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880516.765926:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880516.765928:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880516.765930:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000746bfd07. 02000000:00000001:2.0:1788880516.765932:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880516.765935:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000400:00000010:2.0:1788880516.765938:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000200:2.0:1788880516.765941:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780664832, portal 25 00000100:00000001:2.0:1788880516.765943:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880516.765944:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000100:00000040:2.0:1788880516.765947:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.765953:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880516.765960:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880516.765962:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001c5b8d0f. 00000100:00000200:2.0:1788880516.765965:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 512 bytes to portal 26, xid 1875776780664832, offset 0 00000400:00000200:2.0:1788880516.765970:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880516.765977:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880516.765981:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880516.765984:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880516.765987:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880516.765989:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880516.765992:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [268:268:256:4294967295] 192.168.204.111@tcp LPNI seq info [268:268:8:4294967295] 00000400:00000200:2.0:1788880516.766001:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880516.766006:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 512 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880516.766009:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000045330cf1. 00000800:00000200:2.0:1788880516.766014:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880516.766018:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880516.766021:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000045330cf1 type 1, nob 608 niov 1 nkiov 1 00000100:00000001:2.0:1788880516.766038:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880516.766042:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306152942 00000100:00000001:2.0:1788880516.766046:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306152940 00000100:00000001:2.0:1788880516.766049:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306152942 00000100:00000001:2.0:1788880516.766051:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306152940 00000100:00000001:2.0:1788880516.766054:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306152942 00000100:00000001:2.0:1788880516.766057:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306152940 00000100:00000001:2.0:1788880516.766060:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306152942 00000100:00000001:2.0:1788880516.766063:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306152940 00000100:00000001:2.0:1788880516.766065:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880516.766067:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880516.766069:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880516.766070:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:2.0:1788880516.766077:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 0000000018bd9dcf going to sleep for 11 seconds 00000100:00000001:2.0:1788880516.766080:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880516.766081:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880516.766106:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880516.766109:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000045330cf1. 00000400:00000200:0.0:1788880516.766112:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.766116:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880516.766119:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001c5b8d0f 00000400:00000010:0.0:1788880516.766120:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001c5b8d0f. 00000100:00000001:0.0:1788880516.766123:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880516.766129:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.766137:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880516.766138:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.766144:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880516.766145:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880516.767323:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.767330:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880516.767332:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.767333:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.767339:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880516.767347:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 272 into portal 25 MB=0x6aa0250b08000 00000400:00000200:0.0:1788880516.767352:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 272/272 into md 0x935 [1] + 224 00000800:00000001:0.0:1788880516.767355:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.767359:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880516.767361:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880516.767363:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880516.767373:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880516.767376:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880516.767383:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=272 offset=224 replen=272 req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880516.767394:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880516.767400:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880516.767402:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880516.767420:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880516.767427:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880516.767429:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880516.767430:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:2.0:1788880516.767434:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880516.767436:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880516.767439:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880516.767443:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880516.767451:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.767457:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880516.767458:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880516.767460:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880516.767463:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880516.767464:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880516.767466:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880516.767479:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880516.767484:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=44 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880516.767487:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=44 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880516.767490:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:2.0:1788880516.767493:0:203401:0:(client.c:1497:ptlrpc_check_status()) @@@ check status: rc = -2 req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/0 rc 0/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.767499:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000100:00000001:2.0:1788880516.767501:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000100:00000040:2.0:1788880516.767504:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Rpc:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.767511:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551614 : -2 : 0xfffffffffffffffe) 00000100:00000001:2.0:1788880516.767513:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880516.767514:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880516.767525:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Interpret:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880516.767533:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780664832:192.168.204.111@tcp:501: 00000100:00000001:2.0:1788880516.767537:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880516.767539:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000100:00000001:2.0:1788880516.767541:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:2.0:1788880516.767543:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880516.767546:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 2 fl Complete:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.767552:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880516.767553:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000018bd9dcf. 00000100:00000001:2.0:1788880516.767557:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:2.0:1788880516.767558:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000040:00000001:2.0:1788880516.767560:0:203401:0:(llog_client.c:114:llog_client_open()) Process leaving via out (rc=18446744073709551614 : -2 : 0xfffffffffffffffe) 00000020:00000001:2.0:1788880516.767564:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880516.767565:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880516.767567:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880516.767568:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880516.767570:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780664832/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880532 ref 1 fl Complete:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880516.767576:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880516.767578:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880516.767580:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000746bfd07. 02000000:00000001:2.0:1788880516.767582:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880516.767583:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880516.767584:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880516.767585:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880516.767587:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000000784e3b9. 02000000:00000001:2.0:1788880516.767589:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880516.767591:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880516.767592:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:2.0:1788880516.767595:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880516.767597:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000040:00001000:2.0:1788880516.767605:0:203401:0:(llog.c:1559:llog_open()) MGC192.168.204.111@tcp: Failed to open llog lustre-sptlrpc: rc -2 00000040:00000010:2.0:1788880516.767608:0:203401:0:(llog.c:72:llog_free_handle()) kfreed 'loghandle': 352 at 00000000ed94d533. 00000040:00000001:2.0:1788880516.767611:0:203401:0:(llog.c:1563:llog_open()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000020:00000001:2.0:1788880516.767613:0:203401:0:(obd_config.c:2125:class_config_parse_llog()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 10000000:00000001:2.0:1788880516.767616:0:203401:0:(mgc_request.c:1714:mgc_process_cfg_log()) Process leaving via out_pop (rc=18446744073709551614 : -2 : 0xfffffffffffffffe) 00000020:00000010:2.0:1788880516.767622:0:203401:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 000000009cadd275. 00000040:00000010:2.0:1788880516.767627:0:203401:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000c21a3ff8. 00000020:00000010:2.0:1788880516.767631:0:203401:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 00000000a8e57426. 00000020:00000010:2.0:1788880516.767637:0:203401:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 0000000078217d0d. 80000000:00000010:2.0:1788880516.767642:0:203401:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 00000000ccadc63b. 00040000:00000010:2.0:1788880516.767647:0:203401:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 00000000feb2a015. 20000000:00000010:2.0:1788880516.767651:0:203401:0:(mgs_handler.c:1787:mgs_key_fini()) kfreed 'info': 392 at 000000001f6835d8. 00000020:00000010:2.0:1788880516.767654:0:203401:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000ee42a892. 10000000:00000010:2.0:1788880516.767656:0:203401:0:(mgc_request.c:1730:mgc_process_cfg_log()) kfreed 'env': 56 at 00000000a1e4f73a. 10000000:01000000:2.0:1788880516.767659:0:203401:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 10000000:00000001:2.0:1788880516.767662:0:203401:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:2.0:1788880516.767664:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:2.0:1788880516.767666:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880516.767668:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000003520ed0c refcount=3 00000020:00000001:2.0:1788880516.767670:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 00010000:00000001:2.0:1788880516.767672:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 00010000:00000001:2.0:1788880516.767675:0:203401:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:2.0:1788880516.767678:0:203401:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880516.767685:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880516.767686:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:2.0:1788880516.767687:0:203401:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880516.767692:0:203401:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:2.0:1788880516.767693:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880516.767694:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:2.0:1788880516.767701:0:203401:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880516.767702:0:203401:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 10000000:00000001:2.0:1788880516.767705:0:203401:0:(mgc_request.c:621:do_requeue()) Process leaving 10000000:00000001:2.0:1788880516.767706:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:2.0:1788880516.767708:0:203401:0:(mgc_request.c:123:config_log_put()) log lustre-sptlrpc(000000007b8fd88a) refs 3 10000000:00000001:2.0:1788880516.767710:0:203401:0:(mgc_request.c:147:config_log_put()) Process leaving 00010000:00000001:1.0:1788880516.798369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880516.798378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.798385:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880516.798388:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880516.798391:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880516.798394:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.798398:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880516.798400:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.798402:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880516.798404:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880516.798454:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880516.798460:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880516.798463:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:3.0:1788880517.246702:0:203394:0:(service.c:1710:ptlrpc_at_check_timed()) Process entered 00000100:00000001:3.0:1788880517.246710:0:203394:0:(service.c:1721:ptlrpc_at_check_timed()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880517.822536:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880517.822543:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880517.822549:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880517.822550:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880517.822568:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880517.822570:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880517.822572:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880518.846338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880518.846343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880518.846348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880518.846349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880518.846402:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880518.846406:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880518.846409:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880519.870354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880519.870361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880519.870367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880519.870368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880519.870388:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880519.870391:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880519.870394:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880520.830324:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880520.830334:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880520.830340:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880520.830347:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880520.830351:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880520.830353:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880520.830358:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880520.830365:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e53a8f34. 00000100:00000001:2.0:1788880520.830372:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880520.830388:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880520.830408:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880520.830410:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880520.830414:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_00+0] req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880520.830445:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880520.830448:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880520.830451:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880520.830453:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000020:00000040:2.0:1788880520.830456:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880520.830458:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880520.830460:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880520.830462:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880520.830465:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 00000100:00000001:2.0:1788880520.830469:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880520.830471:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880520.830474:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880520.830476:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880520.830485:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880520.830494:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880520.830495:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880520.830499:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880520.830506:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880520.830508:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880520.830514:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880520.830516:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880520.830520:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_01+1] req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880520.830521:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880520.830525:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880520.830529:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880520.830535:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780664960:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880520.830539:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880520.830544:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880520.830547:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880520.830548:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880520.830551:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880520.830553:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880520.830556:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000010:0.0:1788880520.830560:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003a17641d. 00000100:00000010:2.0:1788880520.830561:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 02000000:00000001:0.0:1788880520.830566:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880520.830566:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880520.830571:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880520.830572:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880520.830574:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880520.830575:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f9411509. 02000000:00000010:2.0:1788880520.830578:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 00000100:00000001:2.0:1788880520.830581:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880520.830584:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f4d84419. 00000100:00000040:2.0:1788880520.830585:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000200:0.0:1788880520.830592:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780664960, portal 25 00000100:00000001:2.0:1788880520.830594:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880520.830597:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880520.830597:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880520.830601:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000100:00000040:2.0:1788880520.830604:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fd6b7ee1] to pc [ptlrpcd_00_02+2] req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880520.830610:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880536 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880520.830623:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880520.830632:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880520.830636:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000dc289a80. 00000100:00000200:0.0:1788880520.830638:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780664960, offset 0 00000100:00000001:2.0:1788880520.830643:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880520.830645:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880520.830647:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880520.830652:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880520.830655:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880520.830657:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11489) 00000100:00000001:3.0:1788880520.830659:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880520.830661:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880520.830663:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:0.0:1788880520.830669:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880520.830670:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880520.830670:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880520.830674:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000400:00000200:0.0:1788880520.830675:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880520.830677:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880520.830679:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880520.830681:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880520.830682:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880520.830688:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880520.830689:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880520.830692:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:0.0:1788880520.830695:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [269:269:256:4294967295] 192.168.204.111@tcp LPNI seq info [269:269:8:4294967295] 00000100:00000001:3.0:1788880520.830695:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880520.830698:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880520.830702:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880520.830702:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880520.830706:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880520.830706:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880520.830709:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880520.830712:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880520.830716:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880520.830717:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00100000:3.0:1788880520.830720:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780665088:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880520.830729:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880520.830731:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000200:0.0:1788880520.830733:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880520.830734:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880520.830734:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880520.830738:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880520.830739:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880520.830740:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000010:0.0:1788880520.830742:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f0a0057b. 02000000:00000001:2.0:1788880520.830743:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000010:3.0:1788880520.830745:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000948807ec. 02000000:00000001:2.0:1788880520.830749:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880520.830749:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880520.830753:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00100000:2.0:1788880520.830756:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780665216:192.168.204.111@tcp:400: 00000400:00000010:3.0:1788880520.830760:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009c9e5278. 00000100:00000001:2.0:1788880520.830761:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:3.0:1788880520.830764:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000800:00000200:0.0:1788880520.830766:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:2.0:1788880520.830766:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000200:3.0:1788880520.830768:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780665088, portal 10 02000000:00000001:2.0:1788880520.830769:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880520.830772:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f0a0057b type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880520.830772:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:2.0:1788880520.830773:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880520.830775:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 02000000:00000010:2.0:1788880520.830780:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000006eecdb68. 00000100:00000040:3.0:1788880520.830783:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880520.830784:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880520.830789:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880520.830790:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000100:00000001:3.0:1788880520.830795:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:2.0:1788880520.830796:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000001:0.0:1788880520.830797:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306157007 00000100:00000040:3.0:1788880520.830799:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788880520.830801:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780665216, portal 10 00000400:00000010:3.0:1788880520.830804:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a165c7f9. 00000100:00000001:2.0:1788880520.830805:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880520.830807:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000100:00000001:0.0:1788880520.830808:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306152942 00000100:00000200:3.0:1788880520.830810:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780665088, offset 0 00000100:00000001:0.0:1788880520.830815:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306157007 00000100:00000040:2.0:1788880520.830815:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880520.830820:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880520.830822:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306152942 00000100:00000001:2.0:1788880520.830829:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880520.830830:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306157007 00000400:00000200:3.0:1788880520.830830:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:2.0:1788880520.830835:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880520.830837:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306152942 00000400:00000200:3.0:1788880520.830838:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:2.0:1788880520.830839:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000841b943f. 00000100:00000001:0.0:1788880520.830842:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306157007 00000100:00000200:2.0:1788880520.830844:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780665216, offset 0 00000400:00000200:3.0:1788880520.830845:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880520.830850:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880520.830855:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880520.830856:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306152942 00000400:00000200:3.0:1788880520.830857:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880520.830862:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880520.830864:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [270:270:255:4294967295] 192.168.204.111@tcp LPNI seq info [270:270:7:4294967295] 00000100:00000001:0.0:1788880520.830867:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880520.830867:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880520.830871:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f0a0057b. 00000100:00000001:0.0:1788880520.830872:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880520.830875:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880520.830878:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880520.830883:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880520.830884:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880520.830886:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880520.830888:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880520.830891:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880520.830894:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880520.830894:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880520.830897:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880520.830904:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880520.830907:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000088542ca3. 00000400:00000200:2.0:1788880520.830909:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788880520.830913:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880520.830915:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:3.0:1788880520.830919:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880520.830920:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880520.830922:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000088542ca3 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880520.830930:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880520.830933:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880520.830935:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [271:271:254:4294967295] 192.168.204.111@tcp LPNI seq info [271:271:6:4294967295] 00000400:00000200:2.0:1788880520.830942:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880520.830947:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880520.830949:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000dc289a80 00000400:00000010:1.0:1788880520.830952:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000dc289a80. 00000800:00000010:2.0:1788880520.830952:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000100:00000001:3.0:1788880520.830959:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880520.830960:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:2.0:1788880520.830962:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:3.0:1788880520.830963:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 341693 usecs (342 jiffies) 00000100:00000001:3.0:1788880520.830968:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306152278 00000100:00000200:1.0:1788880520.830971:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880536 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880520.830972:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880520.830972:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306157007 00000800:00000200:2.0:1788880520.830977:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880520.830977:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306152278 00000100:00000001:3.0:1788880520.830983:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306157007 00000100:00000001:2.0:1788880520.830984:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880520.830987:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 21768 usecs (22 jiffies) 00000100:00000001:1.0:1788880520.830991:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880520.830991:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306151958 00000100:00000001:3.0:1788880520.830993:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306152278 00000100:00000001:2.0:1788880520.830995:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306157007 00000100:00000040:1.0:1788880520.830998:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880536 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880520.830998:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306157007 00000100:00000001:2.0:1788880520.830999:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306151958 00000100:00000001:2.0:1788880520.831005:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306157007 00000100:00000001:3.0:1788880520.831005:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306152278 00000100:00000001:2.0:1788880520.831011:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306151958 00000100:00000001:3.0:1788880520.831012:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306157007 00000100:00000001:1.0:1788880520.831015:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880520.831016:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306157007 00000100:00000001:1.0:1788880520.831018:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880520.831021:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880520.831025:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880520.831026:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306151958 00000100:00000001:3.0:1788880520.831029:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880520.831030:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306157007 00000100:00000001:3.0:1788880520.831033:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880520.831034:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880520.831036:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880520.831039:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000088542ca3. 00000100:00000001:2.0:1788880520.831040:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880520.831040:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788880520.831043:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880520.831043:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880520.831045:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880520.831047:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880520.831048:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880520.831051:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880520.831051:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880520.831052:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a165c7f9 00000400:00000010:1.0:1788880520.831055:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a165c7f9. 00000100:00000001:3.0:1788880520.831056:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880520.831058:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880520.831060:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880520.831062:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880520.831068:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880520.831070:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880520.831073:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880520.831074:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880520.831096:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880520.831098:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000400:00000200:1.0:1788880520.831099:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880520.831101:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880520.831103:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000841b943f 00000400:00000010:1.0:1788880520.831104:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000841b943f. 00000100:00000001:1.0:1788880520.831105:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880520.831107:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880520.831111:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880520.831112:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880520.831123:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880520.831124:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880520.832603:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880520.832613:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880520.832616:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880520.832618:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880520.832625:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880520.832635:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b08080 00000400:00000200:0.0:1788880520.832641:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x93d [1] + 224 00000800:00000001:0.0:1788880520.832647:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880520.832652:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880520.832654:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880520.832658:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880520.832662:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880520.832666:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880520.832675:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880520.832692:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880520.832699:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880520.832729:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880520.832732:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880520.832737:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880520.832739:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880520.832740:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000f9411509. 00000400:00000200:1.0:1788880520.832744:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f4d84419 00000400:00000010:1.0:1788880520.832745:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f4d84419. 00000100:00000001:1.0:1788880520.832748:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880520.832751:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880520.832760:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880520.832766:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880520.832767:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880520.832770:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880520.832772:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880520.832774:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880520.832776:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880520.832778:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880520.832784:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=48 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880520.832787:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=48 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880520.832789:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880520.832790:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880520.832793:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880520.832795:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880520.832797:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880520.832800:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880520.832804:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880520.832805:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880520.832807:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880520.832809:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880520.832823:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780664960:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880520.832828:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880520.832829:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880520.832831:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780664960/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880520.832842:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880520.832845:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880520.832847:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003a17641d. 00000800:00000001:0.0:1788880520.832851:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880520.832851:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880520.832854:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000800:00000001:0.0:1788880520.832855:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000040:1.0:1788880520.832858:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880520.832862:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880520.832864:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e53a8f34. 02000000:00000001:1.0:1788880520.832867:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880520.832869:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880520.832870:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:1.0:1788880520.832873:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880520.832874:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880520.832876:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880520.832880:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880520.832880:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880520.832884:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880520.832888:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880520.832888:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880520.832896:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08100 00000100:00000001:1.0:1788880520.832899:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880520.832903:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x945 [1] + 224 00000800:00000001:0.0:1788880520.832909:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880520.832912:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880520.832914:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880520.832923:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880520.832926:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880520.832929:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880520.832936:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880520.832949:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880520.832954:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880520.832957:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880520.832959:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880520.832961:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880520.832965:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880520.832970:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08180 00000400:00000200:0.0:1788880520.832974:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x949 [1] + 224 00000800:00000001:0.0:1788880520.832977:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880520.832981:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880520.832983:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880520.832987:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880520.832990:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880520.832993:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880520.833002:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880520.833002:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880520.833005:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880520.833009:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880520.833011:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880520.833012:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009c9e5278. 00000400:00000200:3.0:1788880520.833015:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00000100:00000001:0.0:1788880520.833016:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000010:3.0:1788880520.833016:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000100:00000001:3.0:1788880520.833018:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880520.833022:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880520.833023:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880520.833027:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880520.833030:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880520.833036:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880520.833037:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880520.833038:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880520.833039:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880520.833040:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880520.833041:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880520.833043:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880520.833047:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=38 v=5 (5 5 5 0) 00000100:00000001:2.0:1788880520.833053:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:3.0:1788880520.833054:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=38 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880520.833058:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880520.833058:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880520.833060:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788880520.833063:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880520.833065:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880520.833067:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00010000:00000001:3.0:1788880520.833070:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:2.0:1788880520.833072:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00010000:00000001:3.0:1788880520.833075:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880520.833078:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880520.833079:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000010:2.0:1788880520.833082:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:3.0:1788880520.833082:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880520.833085:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880520.833088:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1788880520.833089:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880520.833094:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880520.833095:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880520.833099:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880520.833100:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880520.833106:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880520.833114:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880520.833116:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780665088:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880520.833123:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880520.833125:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880520.833128:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780665088/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880520.833129:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880520.833132:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880520.833136:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880520.833138:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880520.833139:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:3.0:1788880520.833140:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880520.833142:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000010:3.0:1788880520.833142:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000948807ec. 00000100:00000001:2.0:1788880520.833145:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880520.833146:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:2.0:1788880520.833149:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880520.833149:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880520.833151:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880520.833153:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880520.833154:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 00000100:00001000:2.0:1788880520.833155:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=48 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880520.833160:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=5 v=5 (1 0 0 0) 02000000:00000001:3.0:1788880520.833161:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880520.833163:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000001:3.0:1788880520.833163:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788880520.833166:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880520.833169:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000010:3.0:1788880520.833169:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00010000:00080000:2.0:1788880520.833174:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880520.833174:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880520.833182:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788880520.833183:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880520.833187:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880520.833188:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880520.833190:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880520.833193:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880520.833194:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880520.833195:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880520.833197:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880520.833200:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880520.833201:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880520.833205:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880520.833213:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880520.833215:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880520.833216:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880520.833219:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880520.833258:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780665216:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880520.833262:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880520.833264:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880520.833266:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000fd6b7ee1 x1875776780665216/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880536 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880520.833272:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880520.833274:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880520.833275:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000006eecdb68. 02000000:00000001:2.0:1788880520.833279:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880520.833280:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880520.833281:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880520.833283:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880520.833285:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 02000000:00000001:2.0:1788880520.833287:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880520.833288:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880520.833289:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000fd6b7ee1. 00000100:00000001:2.0:1788880520.833292:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880520.833293:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880520.833295:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880520.833298:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880520.833299:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880520.833301:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880520.833303:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880520.894323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880520.894332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880520.894339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880520.894340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880520.894413:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880520.894416:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880520.894417:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880521.918406:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880521.918414:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880521.918422:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880521.918424:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880521.918449:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880521.918452:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880521.918455:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00000001:2.0:1788880522.878380:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000001:2.0:1788880522.878390:0:203401:0:(mgc_request.c:120:config_log_put()) Process leaving 00010000:00000001:1.0:1788880522.942386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880522.942396:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880522.942403:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880522.942405:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880522.942461:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880522.942466:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880522.942470:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880523.966363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880523.966372:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880523.966375:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880523.966378:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880523.966381:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880523.966384:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880523.966385:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880523.966388:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880523.966391:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880523.966392:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880523.966410:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880523.966413:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880523.966414:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880524.990391:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880524.990397:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880524.990401:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880524.990401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880524.990451:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880524.990455:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880524.990457:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880525.950363:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306157007, at 4306157056 00000100:00000040:2.0:1788880525.950363:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306157007, at 4306157056 00000100:00000040:0.0:1788880525.950364:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306157007, at 4306157056 00000100:00000010:1.0:1788880525.950381:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000007dfb4281. 00000100:00000010:2.0:1788880525.950386:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d5dcfb55. 00000100:00000010:0.0:1788880525.950388:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000dba5f48d. 00000100:00000040:3.0:1788880525.950403:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880525.950414:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880525.950428:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000020:00000040:3.0:1788880525.950436:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880525.950442:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880525.950446:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880525.950463:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880525.950475:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f92e72a2. 00000100:00000001:3.0:1788880525.950484:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880525.950498:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880525.950512:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880525.950515:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880525.950522:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_03+3] req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880525.950552:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880525.950554:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880525.950558:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1788880525.950574:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880525.950583:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880525.950584:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:3.0:1788880525.950584:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000100:00000001:1.0:1788880525.950590:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880525.950590:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:3.0:1788880525.950592:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880525.950598:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880525.950599:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:2.0:1788880525.950605:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880525.950606:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880525.950616:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000446e4ede. 00000100:00000001:3.0:1788880525.950621:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880525.950633:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880525.950635:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880525.950644:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880525.950645:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880525.950649:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_00+0] req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880525.950655:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880525.950666:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880525.950670:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880525.950674:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880525.950679:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780665344:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880525.950685:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880525.950691:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880525.950693:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880525.950695:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880525.950697:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880525.950701:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880525.950705:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880525.950705:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000087acc4de. 00000100:00000001:3.0:1788880525.950707:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880525.950710:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880525.950710:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880525.950713:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000010:3.0:1788880525.950714:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000400:00000010:2.0:1788880525.950718:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000020:00000040:3.0:1788880525.950720:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:1.0:1788880525.950722:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880525.950727:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000001:3.0:1788880525.950728:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880525.950734:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880525.950738:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880525.950739:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780665344, portal 25 00000100:00000001:2.0:1788880525.950747:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000010:3.0:1788880525.950748:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000088542ca3. 00000100:00000001:1.0:1788880525.950751:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880525.950751:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000100:00000001:3.0:1788880525.950754:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880525.950755:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880525.950761:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880525.950762:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880541 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880525.950764:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880525.950766:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880525.950769:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880525.950769:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880525.950778:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880525.950793:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780665472:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880525.950809:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880525.950818:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880525.950824:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880525.950826:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880525.950830:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000400:00000010:2.0:1788880525.950832:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000005509e00. 00000100:00000001:3.0:1788880525.950834:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880525.950835:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880525.950840:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780665344, offset 0 02000000:00000001:1.0:1788880525.950843:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880525.950847:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_01+1] req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880525.950850:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005578e134. 00000400:00000200:2.0:1788880525.950853:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880525.950857:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880525.950865:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a3419a3. 00000400:00000010:1.0:1788880525.950872:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000030b8c48c. 00000400:00000200:2.0:1788880525.950873:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:1.0:1788880525.950877:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780665472, portal 10 00000400:00000200:2.0:1788880525.950880:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880525.950881:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880525.950885:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000001:3.0:1788880525.950885:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880525.950886:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880525.950895:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880525.950896:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880525.950902:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880525.950902:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880525.950906:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880525.950917:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [272:272:256:4294967295] 192.168.204.111@tcp LPNI seq info [272:272:8:4294967295] 00000100:00000040:3.0:1788880525.950917:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880525.950919:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880525.950925:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880525.950930:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880525.950933:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11494) 00000100:00000040:0.0:1788880525.950935:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880525.950939:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880525.950945:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000065ad0e62. 00000100:00000200:1.0:1788880525.950950:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780665472, offset 0 00000400:00000200:2.0:1788880525.950950:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788880525.950959:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306157007, at 4306157056 00000400:00000200:1.0:1788880525.950964:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880525.950969:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880525.950973:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000010:3.0:1788880525.950975:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000837757ae. 00000100:00000001:0.0:1788880525.950977:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880525.950977:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000045330cf1. 00000400:00000200:1.0:1788880525.950980:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880525.950985:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880525.950989:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788880525.950990:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880525.950995:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:0.0:1788880525.950996:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880525.950996:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880525.951003:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00100000:0.0:1788880525.951006:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780665600:192.168.204.111@tcp:400: 00000800:00000200:2.0:1788880525.951009:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880525.951010:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880525.951016:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:2.0:1788880525.951017:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000045330cf1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880525.951018:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [273:273:255:4294967295] 192.168.204.111@tcp LPNI seq info [273:273:7:4294967295] 02000000:00000001:0.0:1788880525.951022:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880525.951027:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880525.951031:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880525.951035:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880525.951036:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000010:0.0:1788880525.951037:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000746bfd07. 02000000:00000001:0.0:1788880525.951043:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880525.951043:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306162127 00000800:00000200:1.0:1788880525.951048:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880525.951051:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ccadc63b. 00000400:00000010:0.0:1788880525.951052:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000878f51f0. 00000800:00000010:1.0:1788880525.951052:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000001:2.0:1788880525.951059:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000010:0.0:1788880525.951060:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000800:00000200:1.0:1788880525.951061:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:0.0:1788880525.951066:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780665600, portal 10 00000100:00000001:2.0:1788880525.951066:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306162127 00000800:00000200:1.0:1788880525.951070:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880525.951072:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:1.0:1788880525.951075:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000010:2.0:1788880525.951075:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000076a63e17. 00000100:00000001:0.0:1788880525.951076:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000001:2.0:1788880525.951081:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880525.951082:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880525.951086:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306162127 00000100:00000040:0.0:1788880525.951088:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788880525.951092:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f6d3b065. 00000100:00000040:1.0:1788880525.951094:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 337238 usecs (338 jiffies) 00000800:00000200:3.0:1788880525.951097:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880525.951099:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880525.951101:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306157394 00000800:00000010:3.0:1788880525.951105:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000045330cf1. 00000100:00000001:1.0:1788880525.951108:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306162127 00000100:00000001:2.0:1788880525.951109:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306162127 00000100:00000001:1.0:1788880525.951115:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306157394 00000400:00000200:3.0:1788880525.951115:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880525.951116:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:2.0:1788880525.951117:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006738d3fe. 00000100:00000040:0.0:1788880525.951127:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880525.951127:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306162127 00000100:00000001:2.0:1788880525.951129:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880525.951134:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306157394 00000400:00000010:0.0:1788880525.951135:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ee6cf6a3. 00000400:00000200:3.0:1788880525.951135:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880525.951137:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880525.951142:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306162127 00000100:00000200:0.0:1788880525.951145:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780665600, offset 0 00000100:00000001:1.0:1788880525.951151:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306157394 00000100:00000001:2.0:1788880525.951151:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880525.951154:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000005509e00 00000100:00000001:1.0:1788880525.951156:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306162127 00000100:00000001:2.0:1788880525.951157:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880525.951162:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000005509e00. 00000100:00000001:1.0:1788880525.951163:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880525.951163:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880525.951172:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880525.951174:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880525.951175:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880525.951177:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880525.951179:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880525.951181:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880525.951182:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880525.951187:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880541 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880525.951190:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880525.951190:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880525.951192:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880525.951195:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880525.951196:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880525.951204:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880525.951204:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880525.951205:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880525.951208:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880525.951214:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880525.951215:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880525.951219:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880525.951224:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880525.951225:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1788880525.951227:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880525.951263:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:3.0:1788880525.951267:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880541 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880525.951284:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [274:274:254:4294967295] 192.168.204.111@tcp LPNI seq info [274:274:6:4294967295] 00000100:00000001:3.0:1788880525.951287:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880525.951293:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:0.0:1788880525.951313:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880525.951320:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880525.951324:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b016e4f4. 00000800:00000200:0.0:1788880525.951329:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880525.951334:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880525.951338:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b016e4f4 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880525.951341:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880525.951344:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 21513 usecs (22 jiffies) 00000100:00000001:0.0:1788880525.951346:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306157079 00000100:00000001:0.0:1788880525.951348:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306162127 00000100:00000001:0.0:1788880525.951350:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306157079 00000100:00000001:0.0:1788880525.951351:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306162127 00000100:00000001:0.0:1788880525.951353:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306157079 00000100:00000001:0.0:1788880525.951355:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306162127 00000100:00000001:0.0:1788880525.951356:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306157079 00000100:00000001:0.0:1788880525.951358:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306162127 00000100:00000001:0.0:1788880525.951359:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880525.951361:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880525.951363:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880525.951365:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880525.951369:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880525.951371:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880525.951373:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880525.951374:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880525.951375:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880525.951377:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880525.951396:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880525.951400:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:3.0:1788880525.951403:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880525.951408:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880525.951410:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000065ad0e62 00000400:00000010:3.0:1788880525.951412:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000065ad0e62. 00000100:00000001:3.0:1788880525.951415:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880525.951418:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880525.951425:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880525.951427:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880525.951432:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880525.951440:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880525.951452:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880525.951454:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b016e4f4. 00000400:00000200:3.0:1788880525.951458:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880525.951461:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880525.951463:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ee6cf6a3 00000400:00000010:3.0:1788880525.951464:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ee6cf6a3. 00000100:00000001:3.0:1788880525.951467:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880525.951470:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880525.951476:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880525.951478:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880525.951483:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880525.951589:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880525.952831:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880525.952843:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880525.952846:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880525.952848:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880525.952854:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880525.952862:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b08200 00000400:00000200:1.0:1788880525.952866:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x955 [1] + 224 00000800:00000001:1.0:1788880525.952871:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880525.952880:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880525.952881:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880525.952884:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880525.952889:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880525.952899:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880525.952907:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880525.952920:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880525.952926:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880525.952929:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880525.952930:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880525.952931:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880525.952933:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880525.952937:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08300 00000400:00000200:1.0:1788880525.952942:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x965 [1] + 224 00000800:00000001:1.0:1788880525.952944:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880525.952947:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880525.952949:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880525.952951:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880525.952952:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880525.952953:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880525.952955:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:1.0:1788880525.952958:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788880525.952961:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880525.952964:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:1.0:1788880525.952967:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880525.952967:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:2.0:1788880525.952973:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880525.952975:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:1.0:1788880525.952980:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880525.952986:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880525.952993:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880525.952994:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880525.952998:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880525.953008:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880525.953016:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880525.953018:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880525.953018:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880525.953023:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880525.953023:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880525.953026:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880525.953028:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880525.953031:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880525.953034:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:2.0:1788880525.953035:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880525.953038:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880525.953041:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000878f51f0. 00000100:00001000:2.0:1788880525.953043:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=53 v=5 (5 5 5 0) 00000400:00000200:0.0:1788880525.953045:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000400:00000010:0.0:1788880525.953049:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00001000:2.0:1788880525.953050:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=53 v=5 (1 1 1 0) 00000100:00000001:0.0:1788880525.953054:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880525.953055:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880525.953058:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880525.953061:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000200:0.0:1788880525.953062:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880525.953065:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880525.953070:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880525.953078:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880525.953080:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880525.953100:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880525.953103:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880525.953104:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880525.953106:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880525.953108:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880525.953111:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880525.953114:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:2.0:1788880525.953116:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880525.953117:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880525.953121:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880525.953126:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880525.953132:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780665344:192.168.204.111@tcp:400: 00000100:00001000:0.0:1788880525.953133:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=53 v=5 (5 5 5 0) 00000100:00001000:0.0:1788880525.953140:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=10 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880525.953142:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880525.953144:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880525.953146:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880525.953147:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880525.953151:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:2.0:1788880525.953153:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780665344/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00080000:0.0:1788880525.953159:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880525.953169:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00010000:00000001:0.0:1788880525.953173:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880525.953178:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:2.0:1788880525.953180:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:0.0:1788880525.953182:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000010:2.0:1788880525.953184:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000087acc4de. 00000100:00000001:0.0:1788880525.953186:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880525.953190:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880525.953190:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880525.953194:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880525.953197:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000040:0.0:1788880525.953198:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:2.0:1788880525.953202:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000800:00000001:3.0:1788880525.953204:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000010:2.0:1788880525.953205:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f92e72a2. 00000100:00000001:0.0:1788880525.953214:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880525.953215:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000800:00000001:3.0:1788880525.953217:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880525.953219:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880525.953220:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000010:2.0:1788880525.953222:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:2.0:1788880525.953225:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880525.953227:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880525.953229:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880525.953230:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880525.953257:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880525.953266:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780665600:192.168.204.111@tcp:400: 00000800:00000001:3.0:1788880525.953271:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880525.953274:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880525.953278:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:3.0:1788880525.953279:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880525.953291:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880525.953292:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780665600/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880525.953303:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880525.953309:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880525.953311:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880525.953317:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880525.953327:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880525.953329:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880525.953332:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000746bfd07. 00000400:00000200:3.0:1788880525.953338:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08280 02000000:00000001:0.0:1788880525.953341:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880525.953344:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880525.953346:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000200:3.0:1788880525.953352:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x95d [1] + 224 00000020:00000001:0.0:1788880525.953353:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880525.953357:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000088542ca3. 00000800:00000001:3.0:1788880525.953364:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:0.0:1788880525.953365:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880525.953369:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880525.953370:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:0.0:1788880525.953375:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000800:00000001:3.0:1788880525.953376:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880525.953377:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880525.953386:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880525.953388:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880525.953391:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880525.953395:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880525.953399:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880525.953400:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880525.953404:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880525.953411:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880525.953417:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880525.953427:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880525.953446:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880525.953461:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880525.953464:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880525.953484:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880525.953488:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880525.953492:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880525.953494:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880525.953495:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a3419a3. 00000400:00000200:1.0:1788880525.953499:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000030b8c48c 00000400:00000010:1.0:1788880525.953500:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000030b8c48c. 00000100:00000001:1.0:1788880525.953502:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880525.953510:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880525.953517:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880525.953522:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880525.953523:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880525.953524:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880525.953526:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880525.953527:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880525.953529:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880525.953530:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880525.953536:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=43 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880525.953540:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=43 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880525.953547:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880525.953549:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880525.953551:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880525.953554:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880525.953557:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880525.953559:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880525.953561:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880525.953562:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880525.953566:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880525.953572:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880525.953575:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880525.953576:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880525.953579:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880525.953587:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780665472:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880525.953592:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880525.953595:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880525.953597:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780665472/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880541 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880525.953608:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880525.953610:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880525.953612:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005578e134. 02000000:00000001:1.0:1788880525.953616:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880525.953617:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880525.953619:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880525.953621:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880525.953623:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000446e4ede. 02000000:00000001:1.0:1788880525.953627:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880525.953628:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880525.953630:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:1.0:1788880525.953633:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880525.953634:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880525.953636:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880525.953641:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880525.953643:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880525.953645:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880525.953647:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880526.014380:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880526.014391:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880526.014397:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880526.014398:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880526.014426:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880526.014429:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880526.014432:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880527.038393:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880527.038402:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880527.038408:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880527.038412:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880527.038414:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880527.038416:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880527.038419:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880527.038421:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880527.038423:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880527.038424:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880527.038483:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880527.038488:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880527.038490:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880528.062362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880528.062384:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880528.062392:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880528.062394:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880528.062422:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880528.062425:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880528.062428:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880529.086316:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880529.086321:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880529.086325:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880529.086327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880529.086357:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880529.086361:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880529.086364:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880530.110341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880530.110348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880530.110354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880530.110355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880530.110378:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880530.110381:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880530.110384:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880531.070406:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306162127, at 4306162176 00000100:00000040:1.0:1788880531.070406:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306162127, at 4306162176 00000100:00000040:2.0:1788880531.070406:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880531.070407:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306162127, at 4306162176 00000100:00000001:2.0:1788880531.070426:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880531.070431:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006738d3fe. 00000100:00000010:0.0:1788880531.070434:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ccadc63b. 00000100:00000010:2.0:1788880531.070436:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000100:00000010:1.0:1788880531.070438:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000076a63e17. 00000020:00000040:2.0:1788880531.070447:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880531.070454:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880531.070459:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880531.070466:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880531.070475:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 00000100:00000001:2.0:1788880531.070496:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880531.070508:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880531.070519:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880531.070522:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880531.070528:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fd6b7ee1] to pc [ptlrpcd_00_02+2] req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.070561:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880531.070564:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880531.070584:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880531.070584:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880531.070590:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000100:00000001:1.0:1788880531.070593:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:2.0:1788880531.070595:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880531.070598:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880531.070600:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880531.070600:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880531.070603:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880531.070606:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880531.070607:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880531.070614:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880531.070616:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:2.0:1788880531.070625:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880531.070636:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880531.070641:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880531.070643:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880531.070652:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880531.070658:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880531.070659:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880531.070663:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880531.070665:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880531.070671:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_00_03+3] req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880531.070674:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780665728:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880531.070691:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880531.070702:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880531.070708:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880531.070711:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880531.070714:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880531.070720:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b2a347c7. 00000100:00000040:2.0:1788880531.070723:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880531.070727:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880531.070730:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000078d5a15d. 00000020:00000040:2.0:1788880531.070733:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880531.070741:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.070742:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880531.070745:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880531.070746:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880531.070751:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880531.070752:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a3419a3. 00000100:00000001:3.0:1788880531.070753:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:2.0:1788880531.070757:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 00000100:00000001:3.0:1788880531.070757:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880531.070765:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880531.070766:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880531.070767:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000030b8c48c. 00000100:00000040:2.0:1788880531.070776:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000200:1.0:1788880531.070783:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780665728, portal 25 00000100:00000001:3.0:1788880531.070789:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880531.070792:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.070795:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880531.070797:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880531.070799:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880531.070799:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880531.070800:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880531.070806:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 02000000:00000001:3.0:1788880531.070807:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880531.070812:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000078d5a15d] to pc [ptlrpcd_00_00+0] req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880531.070815:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780665856:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880531.070824:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880547 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880531.070826:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880531.070833:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880531.070839:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880531.070848:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880531.070858:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008d432156. 00000100:00000001:1.0:1788880531.070862:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880531.070864:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880531.070876:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009c9e5278. 00000100:00000040:1.0:1788880531.070877:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880531.070888:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000400:00000010:3.0:1788880531.070888:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000100:00000200:1.0:1788880531.070901:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780665728, offset 0 00000100:00000001:2.0:1788880531.070904:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880531.070904:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780665856, portal 10 00000100:00000001:3.0:1788880531.070909:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:2.0:1788880531.070911:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880531.070912:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000400:00000200:1.0:1788880531.070922:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880531.070925:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1788880531.070930:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880531.070934:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11499) 00000100:00000001:0.0:1788880531.070939:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880531.070947:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880531.070947:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880531.070948:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880531.070956:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880531.070958:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306162127, at 4306162176 00000100:00000040:3.0:1788880531.070961:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880531.070965:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:3.0:1788880531.070968:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000100:00000010:2.0:1788880531.070970:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f6d3b065. 00000100:00000040:0.0:1788880531.070971:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880531.070974:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780665856, offset 0 00000400:00000200:1.0:1788880531.070979:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880531.070996:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1788880531.070996:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880531.070997:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880531.071007:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880531.071009:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880531.071010:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880531.071012:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880531.071016:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880531.071021:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.071021:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880531.071028:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [275:275:256:4294967295] 192.168.204.111@tcp LPNI seq info [275:275:8:4294967295] 00000100:00100000:0.0:1788880531.071036:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780665984:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880531.071041:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880531.071043:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880531.071046:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880531.071048:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880531.071051:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007b91b305. 02000000:00000001:0.0:1788880531.071055:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880531.071058:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:0.0:1788880531.071060:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000878f51f0. 00000400:00000010:0.0:1788880531.071066:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000800:00000200:1.0:1788880531.071068:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:0.0:1788880531.071072:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780665984, portal 10 00000400:00000200:3.0:1788880531.071075:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880531.071077:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000001:0.0:1788880531.071081:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880531.071085:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938426496 : -126823771125120 : ffff8ca78a2c4a80) 00000400:00000200:3.0:1788880531.071090:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:0.0:1788880531.071098:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880531.071099:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880531.071108:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880531.071111:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880531.071124:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880531.071136:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:1.0:1788880531.071143:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880531.071143:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [276:276:255:4294967295] 192.168.204.111@tcp LPNI seq info [276:276:7:4294967295] 00000100:00000040:0.0:1788880531.071148:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880531.071160:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ee6cf6a3. 00000800:00000200:1.0:1788880531.071160:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000200:0.0:1788880531.071174:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780665984, offset 0 00000400:00000200:3.0:1788880531.071180:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880531.071194:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880531.071194:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880531.071209:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880531.071220:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306167247 00000800:00000010:3.0:1788880531.071221:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000088d6bc1. 00000400:00000200:0.0:1788880531.071227:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880531.071260:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:1.0:1788880531.071261:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000400:00000200:0.0:1788880531.071266:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880531.071271:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:3.0:1788880531.071273:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880531.071280:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306167248 00000400:00000200:0.0:1788880531.071283:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:1.0:1788880531.071287:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000800:00000200:3.0:1788880531.071288:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880531.071295:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000088d6bc1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880531.071296:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880531.071302:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880531.071308:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880531.071309:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [277:277:254:4294967295] 192.168.204.111@tcp LPNI seq info [277:277:6:4294967295] 00000100:00000001:1.0:1788880531.071310:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306167248 00000100:00000040:3.0:1788880531.071316:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 332905 usecs (333 jiffies) 00000100:00000010:1.0:1788880531.071318:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000100:00000001:1.0:1788880531.071328:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880531.071328:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306162510 00000400:00000200:0.0:1788880531.071329:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880531.071344:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306167248 00000100:00000001:3.0:1788880531.071348:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306167247 00000100:00000010:1.0:1788880531.071351:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000800:00000200:2.0:1788880531.071354:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:0.0:1788880531.071355:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880531.071360:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306162510 00000100:00000001:1.0:1788880531.071364:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000010:2.0:1788880531.071366:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000800:00000010:0.0:1788880531.071368:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b016e4f4. 00000100:00000001:3.0:1788880531.071373:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306167248 00000400:00000200:2.0:1788880531.071379:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880531.071380:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880531.071383:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306162510 00000800:00000200:0.0:1788880531.071394:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880531.071400:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880531.071407:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306167248 00000800:00000200:0.0:1788880531.071413:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880531.071413:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880531.071415:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880531.071426:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b016e4f4 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880531.071426:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880531.071428:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306162510 00000400:00000200:2.0:1788880531.071431:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:2.0:1788880531.071438:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:0.0:1788880531.071441:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.071443:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880531.071445:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880531.071450:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 21261 usecs (22 jiffies) 00000100:00000001:1.0:1788880531.071450:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000200:2.0:1788880531.071451:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880547 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880531.071464:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880531.071465:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306162199 00000100:00000001:1.0:1788880531.071470:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880531.071475:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306167247 00000100:00000001:1.0:1788880531.071475:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.071476:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880531.071485:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306162199 00000100:00000040:2.0:1788880531.071485:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880547 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880531.071487:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880531.071497:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306167248 00000100:00000001:0.0:1788880531.071507:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306162199 00000100:00000001:3.0:1788880531.071510:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306167248 00000100:00000001:0.0:1788880531.071518:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306167248 00000100:00000001:0.0:1788880531.071525:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306162199 00000100:00000001:3.0:1788880531.071527:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880531.071530:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306167248 00000100:00000001:2.0:1788880531.071535:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880531.071536:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880531.071538:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880531.071544:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.071546:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880531.071549:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880531.071549:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880531.071564:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880531.071565:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788880531.071569:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880531.071581:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000088d6bc1. 00000100:00000001:0.0:1788880531.071583:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880531.071585:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880531.071586:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880531.071587:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880531.071592:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880531.071593:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880531.071599:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880531.071601:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880531.071606:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880531.071606:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880531.071608:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880531.071614:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880531.071622:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880531.071622:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880531.071623:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000400:00000010:2.0:1788880531.071629:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000001:2.0:1788880531.071638:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880531.071645:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.071665:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880531.071668:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.071674:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.071675:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880531.071711:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880531.071713:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b016e4f4. 00000400:00000200:2.0:1788880531.071717:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880531.071720:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880531.071722:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ee6cf6a3 00000400:00000010:2.0:1788880531.071724:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ee6cf6a3. 00000100:00000001:2.0:1788880531.071726:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880531.071729:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.071735:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880531.071737:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.071742:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.071744:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880531.073266:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880531.073277:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880531.073280:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880531.073282:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880531.073288:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880531.073301:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b08380 00000400:00000200:1.0:1788880531.073308:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x96d [1] + 224 00000800:00000001:1.0:1788880531.073313:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880531.073318:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880531.073320:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880531.073323:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880531.073327:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880531.073331:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880531.073341:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880531.073372:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880531.073380:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880531.073386:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880531.073387:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880531.073389:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880531.073394:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880531.073401:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08480 00000400:00000200:1.0:1788880531.073408:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x97d [1] + 224 00000800:00000001:1.0:1788880531.073411:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880531.073415:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:1.0:1788880531.073416:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880531.073420:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880531.073421:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880531.073426:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:2.0:1788880531.073427:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880531.073430:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880531.073432:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a3419a3. 00000400:00000200:2.0:1788880531.073436:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000030b8c48c 00000400:00000010:2.0:1788880531.073438:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000030b8c48c. 00000100:00000001:2.0:1788880531.073441:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880531.073443:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880531.073447:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880531.073450:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880531.073475:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.073484:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880531.073486:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.073493:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880531.073497:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880531.073501:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880531.073504:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880531.073508:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880531.073511:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880531.073513:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880531.073519:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880531.073523:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880531.073529:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=59 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880531.073536:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=59 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880531.073543:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880531.073545:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880531.073549:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880531.073551:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.073555:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880531.073560:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880531.073563:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880531.073568:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880531.073575:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000001:0.0:1788880531.073578:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880531.073580:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880531.073581:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000010:0.0:1788880531.073584:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000878f51f0. 00000100:00000001:2.0:1788880531.073585:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880531.073594:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000400:00000010:0.0:1788880531.073597:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00000040:2.0:1788880531.073597:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880531.073607:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:2.0:1788880531.073617:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780665728:192.168.204.111@tcp:400: 00000100:00000200:0.0:1788880531.073619:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.073625:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880531.073628:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880531.073630:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000fd6b7ee1 x1875776780665728/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.073660:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:0.0:1788880531.073661:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880531.073665:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880531.073668:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b2a347c7. 02000000:00000001:2.0:1788880531.073677:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880531.073678:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:2.0:1788880531.073681:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880531.073682:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:2.0:1788880531.073685:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880531.073686:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880531.073689:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:2.0:1788880531.073691:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880531.073692:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880531.073699:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880531.073700:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 02000000:00000001:0.0:1788880531.073703:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880531.073705:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00001000:0.0:1788880531.073714:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=59 v=5 (5 5 5 0) 02000000:00000001:2.0:1788880531.073714:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880531.073716:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000fd6b7ee1. 00000100:00000001:2.0:1788880531.073722:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880531.073726:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880531.073727:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=16 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880531.073728:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880531.073736:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880531.073736:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880531.073742:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.073743:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.073753:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880531.073755:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880531.073763:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880531.073769:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880531.073775:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880531.073778:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880531.073780:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880531.073782:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880531.073783:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880531.073786:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880531.073792:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880531.073793:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880531.073795:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880531.073797:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880531.073801:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780665984:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880531.073805:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880531.073813:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880531.073815:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000078d5a15d x1875776780665984/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880531.073819:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880531.073821:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880531.073823:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007b91b305. 02000000:00000001:0.0:1788880531.073826:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880531.073827:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880531.073829:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880531.073831:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880531.073833:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 02000000:00000001:0.0:1788880531.073836:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880531.073837:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880531.073838:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000078d5a15d. 00000100:00000001:0.0:1788880531.073841:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880531.073842:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880531.073843:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880531.073847:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880531.073848:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880531.073850:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880531.073851:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880531.073965:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880531.073970:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880531.073971:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880531.073973:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880531.073977:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880531.073990:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08400 00000400:00000200:3.0:1788880531.073998:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x975 [1] + 224 00000800:00000001:3.0:1788880531.074002:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880531.074008:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880531.074010:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880531.074048:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880531.074052:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880531.074055:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880531.074061:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880531.074071:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880531.074075:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880531.074078:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880531.074098:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880531.074100:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880531.074104:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880531.074105:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880531.074107:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009c9e5278. 00000400:00000200:2.0:1788880531.074110:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00000400:00000010:2.0:1788880531.074112:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000100:00000001:2.0:1788880531.074114:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880531.074117:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880531.074122:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.074132:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880531.074133:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880531.074134:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880531.074135:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880531.074136:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880531.074137:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880531.074138:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880531.074143:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=49 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880531.074145:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=49 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880531.074147:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880531.074148:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880531.074149:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880531.074151:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.074152:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880531.074153:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880531.074154:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880531.074161:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880531.074163:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.074167:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880531.074168:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880531.074169:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880531.074171:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880531.074175:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780665856:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880531.074177:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880531.074178:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880531.074179:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780665856/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880547 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880531.074183:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880531.074184:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880531.074185:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008d432156. 02000000:00000001:2.0:1788880531.074187:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880531.074188:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880531.074189:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880531.074190:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880531.074192:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 02000000:00000001:2.0:1788880531.074193:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880531.074194:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880531.074195:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:2.0:1788880531.074207:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880531.074209:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880531.074210:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880531.074214:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880531.074215:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880531.074217:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880531.074218:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880531.134331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880531.134337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880531.134342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880531.134343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880531.134408:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880531.134412:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880531.134415:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880532.158387:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880532.158395:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880532.158402:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880532.158404:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880532.158438:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880532.158442:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880532.158447:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880533.182371:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880533.182394:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880533.182400:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880533.182401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880533.182466:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880533.182475:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880533.182477:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880534.206329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880534.206337:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880534.206339:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880534.206343:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880534.206347:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880534.206350:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880534.206352:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880534.206354:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880534.206359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880534.206360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880534.206379:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880534.206383:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880534.206385:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880535.230659:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880535.230669:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880535.230676:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880535.230677:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880535.230747:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880535.230752:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880535.230755:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880536.190347:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306167247, at 4306167296 00000100:00000040:1.0:1788880536.190347:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306167248, at 4306167296 00000100:00000040:2.0:1788880536.190348:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306167248, at 4306167296 00000100:00000040:3.0:1788880536.190348:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880536.190358:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880536.190364:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000010:1.0:1788880536.190366:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:3.0:1788880536.190368:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000100:00000010:0.0:1788880536.190371:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000020:00000040:3.0:1788880536.190374:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880536.190379:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880536.190382:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880536.190386:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880536.190395:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000088d6bc1. 00000100:00000001:3.0:1788880536.190402:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880536.190413:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880536.190429:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880536.190432:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880536.190438:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_01+1] req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880536.190456:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880536.190458:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880536.190463:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880536.190466:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1788880536.190468:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:3.0:1788880536.190469:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000100:00000001:2.0:1788880536.190470:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:3.0:1788880536.190472:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000040:2.0:1788880536.190475:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880536.190475:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880536.190476:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880536.190479:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880536.190483:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000010:3.0:1788880536.190483:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000446e4ede. 00000100:00000001:2.0:1788880536.190485:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880536.190486:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880536.190486:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880536.190489:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880536.190492:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880536.190492:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880536.190493:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880536.190499:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880536.190503:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780666112:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880536.190507:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880536.190513:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880536.190513:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880536.190514:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880536.190515:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880536.190518:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880536.190518:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_02+2] req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880536.190522:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005701f3a3. 02000000:00000001:2.0:1788880536.190524:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880536.190531:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000100:00000001:3.0:1788880536.190537:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880536.190538:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000040:3.0:1788880536.190540:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:2.0:1788880536.190544:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780666112, portal 25 00000100:00000001:3.0:1788880536.190544:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880536.190547:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880536.190547:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000010:3.0:1788880536.190547:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000100:00000001:2.0:1788880536.190551:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000020:00000040:3.0:1788880536.190551:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880536.190552:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880536.190555:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880536.190555:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:2.0:1788880536.190557:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880552 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880536.190558:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880536.190558:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880536.190560:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880536.190562:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880536.190564:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880536.190565:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000088542ca3. 00000100:00000001:3.0:1788880536.190571:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880536.190575:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880536.190577:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:2.0:1788880536.190581:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880536.190585:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:2.0:1788880536.190586:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c56cc487. 00000100:00000001:1.0:1788880536.190589:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880536.190589:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000200:2.0:1788880536.190590:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780666112, offset 0 00000100:00000001:3.0:1788880536.190593:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880536.190594:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1788880536.190597:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_03+3] req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880536.190598:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880536.190599:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880536.190605:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880536.190611:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780666240:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880536.190613:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880536.190617:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880536.190620:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880536.190621:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880536.190624:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880536.190630:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880536.190630:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:2.0:1788880536.190633:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880536.190635:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880536.190635:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11504) 02000000:00000010:1.0:1788880536.190639:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000088ba1b50. 00000400:00000200:2.0:1788880536.190639:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788880536.190644:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880536.190644:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:1.0:1788880536.190649:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008ffd4b4e. 00000100:00000040:3.0:1788880536.190649:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306167248, at 4306167296 00000400:00000010:1.0:1788880536.190656:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000002fa6ff6. 00000400:00000200:2.0:1788880536.190656:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:3.0:1788880536.190656:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000400:00000200:2.0:1788880536.190661:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [278:278:256:4294967295] 192.168.204.111@tcp LPNI seq info [278:278:8:4294967295] 00000100:00000200:1.0:1788880536.190663:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780666240, portal 10 00000100:00000001:1.0:1788880536.190666:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880536.190668:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000040:1.0:1788880536.190677:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880536.190679:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880536.190685:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880536.190690:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880536.190693:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880536.190694:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:2.0:1788880536.190694:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880536.190700:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880536.190702:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000011e1ab42. 00000100:00000040:0.0:1788880536.190703:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880536.190705:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000100:00000200:1.0:1788880536.190710:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780666240, offset 0 00000800:00000200:2.0:1788880536.190715:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880536.190721:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880536.190722:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880536.190726:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880536.190729:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:0.0:1788880536.190730:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880536.190733:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880536.190735:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000011e1ab42 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880536.190738:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880536.190744:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880536.190746:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:0.0:1788880536.190751:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780666368:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880536.190751:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880536.190753:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880536.190758:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880536.190759:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880536.190760:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306172367 00000400:00000200:1.0:1788880536.190763:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788880536.190765:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880536.190768:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880536.190768:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f6d3b065. 00000400:00000200:1.0:1788880536.190769:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [279:279:255:4294967295] 192.168.204.111@tcp LPNI seq info [279:279:7:4294967295] 02000000:00000001:0.0:1788880536.190774:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880536.190775:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 02000000:00000010:0.0:1788880536.190779:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000fc6edce. 00000100:00000001:2.0:1788880536.190781:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306172367 02000000:00000001:0.0:1788880536.190784:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880536.190787:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000247083fe. 00000400:00000010:0.0:1788880536.190790:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000878f51f0. 00000400:00000200:1.0:1788880536.190792:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880536.190792:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000010:0.0:1788880536.190797:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000100:00000001:2.0:1788880536.190798:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306172367 00000100:00000200:0.0:1788880536.190803:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780666368, portal 10 00000800:00000200:1.0:1788880536.190804:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880536.190804:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000076127301. 00000100:00000001:0.0:1788880536.190807:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880536.190810:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000800:00000010:1.0:1788880536.190811:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000001:2.0:1788880536.190811:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880536.190818:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306172367 00000100:00000040:0.0:1788880536.190819:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880536.190823:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:2.0:1788880536.190825:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045db2575. 00000100:00000001:2.0:1788880536.190830:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880536.190835:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880536.190836:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:1.0:1788880536.190836:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880536.190842:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880536.190842:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880536.190843:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880536.190848:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f4d84419. 00000100:00000001:2.0:1788880536.190848:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880536.190849:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880536.190853:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880536.190854:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780666368, offset 0 00000100:00000040:1.0:1788880536.190855:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 328692 usecs (329 jiffies) 00000100:00000001:1.0:1788880536.190860:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306167625 00000100:00000001:2.0:1788880536.190863:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880536.190864:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880536.190867:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880536.190871:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880536.190874:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880536.190874:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880536.190877:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306172367 00000100:00000001:2.0:1788880536.190878:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880536.190882:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880536.190883:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306167625 00000100:00000001:2.0:1788880536.190884:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880536.190888:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880536.190888:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306172367 00000400:00000200:0.0:1788880536.190894:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880536.190895:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306167625 00000400:00000200:0.0:1788880536.190900:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880536.190900:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306172367 00000100:00000001:1.0:1788880536.190905:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306167625 00000400:00000200:0.0:1788880536.190908:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [280:280:254:4294967295] 192.168.204.111@tcp LPNI seq info [280:280:6:4294967295] 00000100:00000001:1.0:1788880536.190909:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306172367 00000100:00000001:1.0:1788880536.190915:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880536.190919:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880536.190924:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880536.190927:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880536.190928:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880536.190935:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:0.0:1788880536.190936:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880536.190938:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:0.0:1788880536.190941:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f0a0057b. 00000100:00000001:1.0:1788880536.190942:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880536.190944:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880536.190947:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880536.190950:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880536.190951:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880536.190959:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880536.190968:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f0a0057b type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880536.190971:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880536.190975:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 21023 usecs (22 jiffies) 00000100:00000001:0.0:1788880536.190978:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306167318 00000100:00000001:0.0:1788880536.190980:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306172367 00000100:00000001:0.0:1788880536.190982:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306167318 00000100:00000001:0.0:1788880536.190984:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306172367 00000100:00000001:0.0:1788880536.190986:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306167318 00000100:00000001:0.0:1788880536.190988:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306172367 00000100:00000001:0.0:1788880536.190990:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306167318 00000100:00000001:0.0:1788880536.190992:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306172367 00000100:00000001:0.0:1788880536.190994:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880536.190997:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880536.191000:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880536.191002:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880536.191006:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880536.191007:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880536.191009:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880536.191010:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880536.191011:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880536.191013:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880536.191038:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880536.191049:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000011e1ab42. 00000400:00000200:3.0:1788880536.191053:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880536.191057:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880536.191062:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c56cc487 00000400:00000010:3.0:1788880536.191064:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c56cc487. 00000100:00000001:3.0:1788880536.191068:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880536.191071:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880552 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880536.191077:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880536.191079:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880552 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880536.191083:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880536.191084:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880536.191093:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880536.191094:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:3.0:1788880536.191096:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880536.191097:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880536.191113:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000400:00000010:3.0:1788880536.191114:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00000100:00000001:3.0:1788880536.191117:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880536.191120:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880536.191127:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880536.191128:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880536.191134:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880536.191135:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880536.191192:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880536.191195:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f0a0057b. 00000400:00000200:3.0:1788880536.191198:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880536.191202:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880536.191204:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f4d84419 00000400:00000010:3.0:1788880536.191206:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f4d84419. 00000100:00000001:3.0:1788880536.191214:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880536.191217:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880536.191224:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880536.191226:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880536.191231:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880536.191249:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880536.194191:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880536.194205:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880536.194209:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880536.194211:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880536.194220:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880536.194255:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b08500 00000400:00000200:1.0:1788880536.194265:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x985 [1] + 224 00000800:00000001:1.0:1788880536.194271:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880536.194276:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880536.194279:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880536.194283:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880536.194289:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880536.194296:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880536.194323:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880536.194338:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880536.194345:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880536.194349:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880536.194351:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880536.194354:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880536.194354:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880536.194358:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880536.194363:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:2.0:1788880536.194365:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880536.194368:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880536.194369:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:1.0:1788880536.194374:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08580 00000400:00000200:2.0:1788880536.194375:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880536.194379:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880536.194381:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880536.194386:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880536.194387:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x98d [1] + 224 00000800:00000001:1.0:1788880536.194398:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:2.0:1788880536.194399:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880536.194404:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880536.194405:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880536.194408:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880536.194409:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880536.194410:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880536.194414:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880536.194417:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:1.0:1788880536.194418:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880536.194420:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880536.194423:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880536.194424:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:2.0:1788880536.194428:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=64 v=5 (5 5 5 0) 00000100:00000200:1.0:1788880536.194429:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:2.0:1788880536.194434:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=64 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880536.194439:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880536.194442:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880536.194446:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880536.194449:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880536.194451:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880536.194458:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880536.194465:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880536.194471:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880536.194472:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880536.194473:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880536.194475:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880536.194479:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780666112:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880536.194481:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880536.194487:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880536.194490:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880536.194493:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880536.194493:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:1.0:1788880536.194498:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:2.0:1788880536.194498:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780666112/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880536.194503:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880536.194505:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880536.194509:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880536.194511:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005701f3a3. 00000400:00000200:1.0:1788880536.194514:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880536.194515:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880536.194516:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880536.194520:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000001:2.0:1788880536.194521:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880536.194523:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000200:1.0:1788880536.194527:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08600 00000020:00000001:2.0:1788880536.194527:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000001:3.0:1788880536.194527:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000010:2.0:1788880536.194530:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000088d6bc1. 00000400:00000001:3.0:1788880536.194531:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880536.194535:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008ffd4b4e. 02000000:00000001:2.0:1788880536.194536:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880536.194539:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000200:3.0:1788880536.194539:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000002fa6ff6 00000100:00000010:2.0:1788880536.194540:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000400:00000010:3.0:1788880536.194542:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000002fa6ff6. 00000100:00000001:2.0:1788880536.194545:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880536.194546:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880536.194548:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880536.194552:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880536.194554:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880536.194555:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880536.194556:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880536.194560:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x995 [1] + 224 00000100:00000001:2.0:1788880536.194560:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880536.194564:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880536.194569:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880536.194577:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880536.194578:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880536.194588:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880536.194590:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880536.194593:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880536.194596:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880536.194597:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880536.194600:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880536.194601:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880536.194602:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880536.194603:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880536.194606:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880536.194610:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880536.194611:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=54 v=5 (5 5 5 0) 00000100:00001000:3.0:1788880536.194617:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=54 v=5 (1 1 1 0) 00000100:00000001:3.0:1788880536.194622:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880536.194624:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880536.194627:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880536.194631:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880536.194633:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880536.194634:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880536.194637:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880536.194650:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880536.194653:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880536.194653:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880536.194660:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880536.194664:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880536.194664:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880536.194674:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880536.194677:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880536.194679:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880536.194682:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880536.194683:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880536.194688:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880536.194693:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00100000:3.0:1788880536.194693:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780666240:192.168.204.111@tcp:400: 00000400:00000001:0.0:1788880536.194696:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880536.194699:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000878f51f0. 00000100:00000001:3.0:1788880536.194701:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880536.194705:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1788880536.194706:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000100:00000040:3.0:1788880536.194709:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780666240/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880536.194710:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00000001:0.0:1788880536.194715:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880536.194721:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:0.0:1788880536.194723:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880536.194724:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880536.194727:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000088ba1b50. 02000000:00000001:3.0:1788880536.194732:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880536.194735:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880536.194737:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000200:0.0:1788880536.194739:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880536.194741:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880536.194742:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000446e4ede. 02000000:00000001:3.0:1788880536.194745:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880536.194747:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880536.194749:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:3.0:1788880536.194753:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880536.194756:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880536.194759:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880536.194760:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880536.194764:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880536.194764:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880536.194767:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880536.194769:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880536.194770:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880536.194772:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880536.194774:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880536.194775:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880536.194778:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880536.194781:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880536.194785:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=64 v=5 (5 5 5 0) 00000100:00001000:0.0:1788880536.194788:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=21 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880536.194790:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880536.194791:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880536.194793:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880536.194796:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880536.194802:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880536.194804:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880536.194805:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880536.194807:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880536.194808:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880536.194810:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880536.194815:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880536.194817:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880536.194818:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880536.194821:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880536.194828:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780666368:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880536.194833:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880536.194835:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880536.194837:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780666368/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880552 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880536.194855:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880536.194857:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880536.194859:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000fc6edce. 02000000:00000001:0.0:1788880536.194862:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880536.194863:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880536.194864:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880536.194867:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880536.194868:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000088542ca3. 02000000:00000001:0.0:1788880536.194871:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880536.194872:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880536.194874:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:0.0:1788880536.194876:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880536.194877:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880536.194879:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880536.194883:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880536.194884:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880536.194886:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880536.194888:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880536.254342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880536.254349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880536.254356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880536.254359:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880536.254361:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880536.254364:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880536.254368:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880536.254370:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880536.254372:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880536.254374:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880536.254395:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880536.254398:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880536.254400:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880537.278346:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880537.278353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880537.278359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880537.278360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880537.278421:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880537.278440:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880537.278443:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880538.302349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880538.302355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880538.302359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880538.302360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880538.302375:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880538.302377:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880538.302379:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880539.326365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880539.326385:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880539.326391:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880539.326392:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880539.326459:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880539.326464:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880539.326467:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880540.350327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880540.350335:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880540.350341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880540.350342:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880540.350367:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880540.350370:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880540.350372:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880541.310349:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306172367, at 4306172416 00000100:00000040:1.0:1788880541.310349:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306172367, at 4306172416 00000100:00000040:2.0:1788880541.310349:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306172367, at 4306172416 00000100:00000010:0.0:1788880541.310367:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f6d3b065. 00000100:00000010:1.0:1788880541.310369:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000247083fe. 00000100:00000010:2.0:1788880541.310374:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000076127301. 00000100:00000040:1.0:1788880541.310385:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880541.310391:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880541.310397:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000020:00000040:1.0:1788880541.310401:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880541.310406:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880541.310408:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880541.310413:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880541.310420:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:1.0:1788880541.310427:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880541.310428:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306172367, at 4306172416 00000100:00000040:1.0:1788880541.310437:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000010:3.0:1788880541.310439:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045db2575. 00000100:00000001:1.0:1788880541.310459:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880541.310462:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880541.310468:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca5e4c87] to pc [ptlrpcd_00_00+0] req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880541.310493:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880541.310495:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880541.310499:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880541.310501:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000663051e5. 00000020:00000040:1.0:1788880541.310503:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880541.310505:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880541.310519:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880541.310521:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880541.310523:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880541.310525:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:0.0:1788880541.310526:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880541.310529:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880541.310529:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788880541.310530:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880541.310533:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880541.310537:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880541.310540:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880541.310546:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880541.310548:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880541.310552:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000663051e5] to pc [ptlrpcd_00_01+1] req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880541.310560:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880541.310563:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880541.310565:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880541.310566:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880541.310567:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880541.310570:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880541.310572:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880541.310576:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880541.310580:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000094565713. 00000100:00000001:0.0:1788880541.310582:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880541.310582:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000020:00000040:1.0:1788880541.310584:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880541.310586:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880541.310588:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880541.310591:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880541.310591:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880541.310594:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880541.310595:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880541.310598:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000010:1.0:1788880541.310599:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 02000000:00000001:2.0:1788880541.310602:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880541.310603:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880541.310605:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780666624:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880541.310609:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880541.310610:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880541.310613:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880541.310616:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880541.310619:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880541.310621:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880541.310624:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880541.310624:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000659bd78d. 00000100:00000040:1.0:1788880541.310629:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000094565713] to pc [ptlrpcd_00_02+2] req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880541.310635:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880541.310640:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000400:00000010:2.0:1788880541.310647:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000001:1.0:1788880541.310650:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880541.310653:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780666624, portal 10 00000100:00000040:1.0:1788880541.310654:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880541.310658:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:1.0:1788880541.310659:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880541.310661:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938053248 : -126823771498368 : ffff8ca78a269880) 00000100:00000040:1.0:1788880541.310665:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11510) 00000100:00000001:0.0:1788880541.310669:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880541.310669:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880541.310673:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880541.310676:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880541.310683:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880541.310684:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880541.310691:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880541.310692:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880541.310697:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880541.310697:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000793778c8. 00000100:00000001:0.0:1788880541.310700:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880541.310703:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880541.310703:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780666624, offset 0 02000000:00000001:0.0:1788880541.310707:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880541.310710:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880541.310713:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880541.310716:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:0.0:1788880541.310719:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780666752:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880541.310731:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880541.310737:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880541.310737:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788880541.310741:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1788880541.310744:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880541.310745:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880541.310748:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880541.310750:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000010:0.0:1788880541.310753:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000051bfba0. 00000400:00000200:2.0:1788880541.310753:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788880541.310757:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880541.310758:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [281:281:256:4294967295] 192.168.204.111@tcp LPNI seq info [281:281:8:4294967295] 00000400:00000010:0.0:1788880541.310763:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000878f51f0. 00000400:00000010:0.0:1788880541.310769:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000400:00000200:2.0:1788880541.310769:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:0.0:1788880541.310774:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780666752, portal 10 00000800:00000200:2.0:1788880541.310775:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880541.310779:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880541.310781:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055040 : -126823771496576 : ffff8ca78a269f80) 00000800:00000010:2.0:1788880541.310782:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000781b5970. 00000100:00000040:0.0:1788880541.310788:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880541.310795:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880541.310803:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1788880541.310806:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:0.0:1788880541.310810:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880541.310811:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000781b5970 type 1, nob 320 niov 1 nkiov 1 00000400:00000010:0.0:1788880541.310815:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000dc289a80. 00000100:00000200:0.0:1788880541.310818:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780666752, offset 0 00000100:00000001:2.0:1788880541.310823:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880541.310824:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880541.310828:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 324596 usecs (325 jiffies) 00000400:00000200:0.0:1788880541.310832:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880541.310833:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306172741 00000400:00000200:0.0:1788880541.310839:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:2.0:1788880541.310839:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000076127301. 00000100:00000001:2.0:1788880541.310844:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880541.310845:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880541.310849:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306172741 00000400:00000200:0.0:1788880541.310850:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:2.0:1788880541.310855:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000017cf871b. 00000400:00000200:0.0:1788880541.310856:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880541.310860:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880541.310863:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [282:282:255:4294967295] 192.168.204.111@tcp LPNI seq info [282:282:7:4294967295] 00000100:00000001:2.0:1788880541.310864:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306172741 00000100:00000010:2.0:1788880541.310868:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000668fd8a. 00000100:00000001:2.0:1788880541.310871:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880541.310875:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306172741 00000100:00000010:2.0:1788880541.310879:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005c6a0f3f. 00000400:00000200:0.0:1788880541.310881:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880541.310882:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880541.310885:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880541.310888:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880541.310892:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880541.310892:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:3.0:1788880541.310894:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880541.310898:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000100:00000040:2.0:1788880541.310898:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:3.0:1788880541.310900:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000781b5970. 00000400:00000200:3.0:1788880541.310907:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:0.0:1788880541.310908:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880541.310916:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:3.0:1788880541.310919:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000200:0.0:1788880541.310920:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880541.310920:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880541.310925:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:0.0:1788880541.310926:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880541.310927:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880541.310927:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000793778c8 00000100:00000001:0.0:1788880541.310930:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880541.310932:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880541.310933:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000793778c8. 00000100:00100000:2.0:1788880541.310937:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780666496:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880541.310939:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000040:0.0:1788880541.310940:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 20816 usecs (21 jiffies) 00000100:00000001:2.0:1788880541.310941:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880541.310945:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306172437 00000100:00000200:3.0:1788880541.310946:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880541.310949:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880541.310951:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306172741 02000000:00000001:2.0:1788880541.310952:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880541.310956:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880541.310957:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306172437 02000000:00000010:2.0:1788880541.310960:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005309c375. 00000100:00000001:0.0:1788880541.310964:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306172741 02000000:00000001:2.0:1788880541.310964:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880541.310968:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001f454358. 00000100:00000001:0.0:1788880541.310969:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306172437 00000400:00000010:2.0:1788880541.310973:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000170fdf47. 00000100:00000001:0.0:1788880541.310974:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306172741 00000100:00000200:2.0:1788880541.310976:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780666496, portal 25 00000100:00000001:0.0:1788880541.310978:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306172437 00000100:00000001:2.0:1788880541.310980:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880541.310982:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306172741 00000100:00000001:2.0:1788880541.310983:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000001:0.0:1788880541.310987:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880541.310988:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880557 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880541.310992:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880541.310997:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880541.310997:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880541.311014:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880541.311015:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880541.311015:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880541.311019:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880541.311020:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d208f26. 00000100:00000001:0.0:1788880541.311025:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:2.0:1788880541.311025:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780666496, offset 0 00000100:00000001:0.0:1788880541.311029:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880541.311034:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880541.311034:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880541.311035:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880541.311038:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880541.311040:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880541.311041:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880541.311045:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880541.311046:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880541.311051:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880541.311054:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880541.311056:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880541.311059:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880541.311063:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [283:283:254:4294967295] 192.168.204.111@tcp LPNI seq info [283:283:6:4294967295] 00000400:00000200:2.0:1788880541.311071:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880541.311075:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880541.311078:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c52725cf. 00000800:00000200:2.0:1788880541.311082:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880541.311087:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880541.311089:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c52725cf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880541.311093:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880541.311095:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306177487 00000100:00000001:2.0:1788880541.311100:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306172741 00000100:00000001:2.0:1788880541.311107:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306177487 00000100:00000001:2.0:1788880541.311110:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306172741 00000800:00000200:3.0:1788880541.311111:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880541.311112:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306177487 00000800:00000010:3.0:1788880541.311116:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000100:00000001:2.0:1788880541.311117:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306172741 00000100:00000001:2.0:1788880541.311122:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306177487 00000400:00000200:3.0:1788880541.311123:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880541.311125:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306172741 00000100:00000001:2.0:1788880541.311128:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880541.311131:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880541.311131:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880541.311133:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880541.311135:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000dc289a80 00000100:00000001:2.0:1788880541.311136:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880541.311139:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000dc289a80. 00000100:00000001:2.0:1788880541.311141:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880541.311143:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880541.311144:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880541.311147:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880541.311148:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880541.311150:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880541.311150:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880541.311155:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880541.311162:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880541.311165:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880541.311222:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880541.311224:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880541.311283:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880541.311286:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c52725cf. 00000400:00000200:3.0:1788880541.311288:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880541.311291:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880541.311293:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d208f26 00000400:00000010:3.0:1788880541.311294:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d208f26. 00000100:00000001:3.0:1788880541.311295:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880541.311298:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880557 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880541.311303:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880541.311304:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880557 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880541.311308:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880541.311309:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880541.312825:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880541.312835:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880541.312838:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880541.312840:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880541.312847:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880541.312951:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08700 00000400:00000200:3.0:1788880541.312958:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x99d [1] + 224 00000800:00000001:3.0:1788880541.312964:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880541.312968:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880541.312970:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880541.312974:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880541.312978:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880541.312982:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880541.312991:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880541.313024:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880541.313030:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880541.313034:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880541.313036:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880541.313037:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880541.313041:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880541.313047:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08780 00000100:00000001:2.0:1788880541.313059:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880541.313062:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880541.313087:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x9a5 [1] + 224 00000400:00000001:2.0:1788880541.313093:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788880541.313094:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:2.0:1788880541.313096:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880541.313098:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000800:00000001:3.0:1788880541.313100:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880541.313101:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000200:3.0:1788880541.313103:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:2.0:1788880541.313104:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880541.313109:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:3.0:1788880541.313111:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:2.0:1788880541.313114:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880541.313118:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880541.313124:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880541.313125:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880541.313148:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880541.313150:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880541.313153:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:3.0:1788880541.313153:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880541.313156:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880541.313159:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880541.313167:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880541.313169:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880541.313172:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:2.0:1788880541.313177:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=59 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880541.313182:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=59 v=5 (1 1 1 0) 00000800:00000001:3.0:1788880541.313182:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880541.313187:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:3.0:1788880541.313187:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880541.313189:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880541.313191:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:2.0:1788880541.313193:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:3.0:1788880541.313195:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788880541.313199:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880541.313203:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880541.313205:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880541.313208:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880541.313210:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880541.313212:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880541.313212:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b08680 00000100:00000001:2.0:1788880541.313213:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880541.313217:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880541.313223:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880541.313225:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x9ad [1] + 224 00000100:00000001:2.0:1788880541.313251:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880541.313253:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880541.313255:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880541.313258:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880541.313259:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880541.313260:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880541.313263:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880541.313266:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000878f51f0. 00000800:00000001:3.0:1788880541.313266:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:2.0:1788880541.313269:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780666624:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880541.313270:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880541.313272:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000400:00000010:0.0:1788880541.313276:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000400:00000200:3.0:1788880541.313278:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880541.313279:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880541.313280:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880541.313282:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880541.313284:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:0.0:1788880541.313286:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880541.313290:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000663051e5 x1875776780666624/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880541.313292:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880541.313312:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880541.313322:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880541.313324:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880541.313329:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880541.313332:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000659bd78d. 00000100:00000001:0.0:1788880541.313334:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:2.0:1788880541.313339:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880541.313340:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:2.0:1788880541.313344:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880541.313345:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880541.313346:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000040:2.0:1788880541.313348:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880541.313350:0:201287:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:3.0:1788880541.313351:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788880541.313352:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:0.0:1788880541.313354:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880541.313357:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000010:2.0:1788880541.313357:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 00000100:00000001:0.0:1788880541.313360:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880541.313363:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880541.313364:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880541.313368:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=69 v=5 (5 5 5 0) 00000100:00001000:0.0:1788880541.313373:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=26 v=5 (1 0 0 0) 02000000:00000001:2.0:1788880541.313373:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880541.313376:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000663051e5. 00000100:00000001:0.0:1788880541.313378:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880541.313380:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880541.313381:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00010000:00000001:0.0:1788880541.313382:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880541.313385:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00010000:00080000:0.0:1788880541.313387:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880541.313390:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880541.313395:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:0.0:1788880541.313396:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880541.313397:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880541.313399:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880541.313401:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:2.0:1788880541.313402:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880541.313405:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880541.313406:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001f454358. 00000100:00000001:0.0:1788880541.313408:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880541.313410:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880541.313412:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000170fdf47 00000100:00000040:0.0:1788880541.313414:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880541.313416:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000170fdf47. 00000100:00000001:0.0:1788880541.313422:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880541.313422:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880541.313425:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880541.313426:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880541.313429:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880541.313430:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880541.313456:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780666752:192.168.204.111@tcp:400: 00000100:00000200:2.0:1788880541.313461:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880541.313463:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880541.313467:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880541.313473:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000094565713 x1875776780666752/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880541.313475:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880541.313477:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880541.313482:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880541.313485:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880541.313487:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880541.313487:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:0.0:1788880541.313491:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880541.313491:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880541.313493:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880541.313496:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=69 v=5 (5 5 5 0) 02000000:00000010:0.0:1788880541.313500:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000051bfba0. 00000100:00001000:2.0:1788880541.313501:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=69 v=5 (1 1 1 0) 02000000:00000001:0.0:1788880541.313505:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880541.313506:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000020:00000001:0.0:1788880541.313509:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880541.313509:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880541.313512:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00010000:00000001:2.0:1788880541.313513:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880541.313516:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880541.313517:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880541.313520:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880541.313521:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:0.0:1788880541.313526:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:2.0:1788880541.313527:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880541.313529:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880541.313532:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000094565713. 00000100:00000001:0.0:1788880541.313537:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880541.313539:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880541.313541:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880541.313543:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880541.313545:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880541.313547:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880541.313549:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880541.313552:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880541.313553:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880541.313556:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880541.313559:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880541.313568:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780666496:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880541.313573:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880541.313575:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880541.313578:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780666496/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880557 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880541.313584:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880541.313586:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880541.313588:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005309c375. 02000000:00000001:2.0:1788880541.313590:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880541.313592:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880541.313593:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880541.313595:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880541.313596:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 02000000:00000001:2.0:1788880541.313599:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880541.313600:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880541.313601:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:2.0:1788880541.313604:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880541.313605:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880541.313606:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880541.313610:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880541.313611:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880541.313613:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880541.313615:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880541.374326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880541.374333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880541.374340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880541.374342:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880541.374394:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880541.374398:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880541.374401:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880542.398319:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880542.398325:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880542.398329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880542.398330:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880542.398349:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880542.398352:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880542.398355:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880543.422309:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880543.422314:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880543.422318:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880543.422319:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880543.422347:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880543.422351:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880543.422353:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880544.446321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880544.446328:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880544.446330:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880544.446332:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880544.446335:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880544.446337:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880544.446338:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880544.446339:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880544.446342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880544.446342:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880544.446354:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880544.446356:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880544.446357:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880545.470373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880545.470381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880545.470388:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880545.470389:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880545.470452:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880545.470456:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880545.470459:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880546.430331:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306177487, at 4306177536 00000100:00000040:3.0:1788880546.430331:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306177487, at 4306177536 00000100:00000040:1.0:1788880546.430335:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306177487, at 4306177536 00000100:00000010:0.0:1788880546.430348:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000076127301. 00000100:00000010:1.0:1788880546.430352:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000017cf871b. 00000100:00000010:3.0:1788880546.430355:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005c6a0f3f. 00000100:00000040:2.0:1788880546.430387:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306177487, at 4306177536 00000100:00000040:0.0:1788880546.430388:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880546.430397:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880546.430405:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000668fd8a. 00000100:00000010:0.0:1788880546.430407:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000009ee4c6bf. 00000020:00000040:0.0:1788880546.430414:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880546.430419:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880546.430422:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880546.430428:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880546.430435:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 00000100:00000001:0.0:1788880546.430440:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880546.430448:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880546.430454:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880546.430456:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880546.430460:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000009ee4c6bf] to pc [ptlrpcd_00_03+3] req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880546.430477:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880546.430489:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880546.430491:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880546.430492:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880546.430494:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000451fe31a. 00000020:00000040:0.0:1788880546.430496:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880546.430498:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880546.430499:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880546.430500:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880546.430503:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000004e0f6846. 00000100:00000001:3.0:1788880546.430504:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880546.430508:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880546.430509:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880546.430509:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880546.430512:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880546.430512:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880546.430519:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880546.430523:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880546.430525:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880546.430532:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880546.430533:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880546.430536:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880546.430538:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880546.430539:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000451fe31a] to pc [ptlrpcd_00_00+0] req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880546.430541:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880546.430547:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780666880:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880546.430553:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880546.430557:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880546.430560:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880546.430561:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880546.430564:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000356bd8da. 02000000:00000001:1.0:1788880546.430566:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880546.430571:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b884f446. 00000100:00000001:0.0:1788880546.430574:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880546.430576:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000040:0.0:1788880546.430579:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:1.0:1788880546.430582:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780666880, portal 25 00000100:00000001:0.0:1788880546.430586:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880546.430586:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000010:0.0:1788880546.430591:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000007f91037. 00000100:00000001:1.0:1788880546.430591:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849358720 : -126822860192896 : ffff8ca7c0780380) 00000020:00000040:0.0:1788880546.430596:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880546.430599:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:1.0:1788880546.430600:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880562 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880546.430602:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880546.430605:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880546.430608:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:0.0:1788880546.430610:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c93f60a2. 00000100:00000001:2.0:1788880546.430614:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880546.430614:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880546.430617:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880546.430620:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880546.430621:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880546.430621:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880546.430626:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:1.0:1788880546.430630:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000200:1.0:1788880546.430636:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780666880, offset 0 00000100:00000040:2.0:1788880546.430640:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880546.430646:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:1.0:1788880546.430658:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880546.430680:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880546.430691:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880546.430694:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880546.430697:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880546.430697:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:0.0:1788880546.430704:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000007f91037] to pc [ptlrpcd_00_01+1] req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880546.430705:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880546.430710:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880546.430715:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:2.0:1788880546.430723:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880546.430724:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:2.0:1788880546.430729:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880546.430738:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880546.430747:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [284:284:256:4294967295] 192.168.204.111@tcp LPNI seq info [284:284:8:4294967295] 00000100:00100000:2.0:1788880546.430753:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780667008:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880546.430771:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880546.430777:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1788880546.430781:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788880546.430781:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880546.430787:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880546.430789:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880546.430798:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:2.0:1788880546.430800:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003816d5cd. 00000800:00000200:1.0:1788880546.430808:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880546.430808:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880546.430809:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880546.430818:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:1.0:1788880546.430822:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:3.0:1788880546.430822:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880546.430824:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11515) 00000400:00000010:2.0:1788880546.430824:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001f454358. 00000100:00000001:3.0:1788880546.430826:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:2.0:1788880546.430838:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000170fdf47. 00000100:00000040:3.0:1788880546.430841:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880546.430845:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880546.430848:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780667008, portal 10 00000100:00000001:2.0:1788880546.430856:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:1.0:1788880546.430857:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880546.430861:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849366784 : -126822860184832 : ffff8ca7c0782300) 00000100:00000001:3.0:1788880546.430863:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880546.430867:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880546.430869:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 02000000:00000001:3.0:1788880546.430872:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880546.430877:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880546.430878:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880546.430885:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880546.430888:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880546.430889:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306182607 00000100:00000010:1.0:1788880546.430900:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000100:00100000:3.0:1788880546.430902:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780667136:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880546.430918:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:2.0:1788880546.430918:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880546.430920:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880546.430926:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306182607 00000100:00000040:2.0:1788880546.430926:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880546.430931:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880546.430933:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880546.430934:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000010:1.0:1788880546.430935:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 02000000:00000001:3.0:1788880546.430935:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880546.430941:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000081a10824. 00000100:00000001:1.0:1788880546.430942:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000200:2.0:1788880546.430942:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780667008, offset 0 02000000:00000001:3.0:1788880546.430944:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880546.430945:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306182607 00000400:00000010:3.0:1788880546.430956:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bc3da258. 00000100:00000010:1.0:1788880546.430958:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000400:00000200:2.0:1788880546.430963:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880546.430969:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000010:3.0:1788880546.430972:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cf82a3fa. 00000100:00000001:1.0:1788880546.430976:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306182607 00000100:00000200:3.0:1788880546.430978:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780667136, portal 10 00000400:00000200:2.0:1788880546.430982:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:1.0:1788880546.430983:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000100:00000001:3.0:1788880546.430983:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880546.430988:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849371264 : -126822860180352 : ffff8ca7c0783480) 00000100:00000001:1.0:1788880546.430990:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:2.0:1788880546.430996:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880546.430997:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880546.430999:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:3.0:1788880546.431000:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880546.431005:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880546.431007:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:2.0:1788880546.431008:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880546.431010:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880546.431017:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880546.431020:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880546.431020:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880546.431026:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880546.431028:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880546.431029:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880546.431037:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:3.0:1788880546.431041:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880546.431044:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880546.431045:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [285:285:255:4294967295] 192.168.204.111@tcp LPNI seq info [285:285:7:4294967295] 00000100:00000001:1.0:1788880546.431048:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880546.431049:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 00000100:00000001:1.0:1788880546.431054:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880546.431061:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880546.431061:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780667136, offset 0 00000100:00000001:1.0:1788880546.431066:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880546.431070:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880546.431079:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880546.431088:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880546.431089:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000400:00000200:3.0:1788880546.431094:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880546.431095:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000007cb951ea. 00000400:00000010:0.0:1788880546.431096:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000400:00000200:3.0:1788880546.431100:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880546.431104:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:3.0:1788880546.431105:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:0.0:1788880546.431115:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880562 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880546.431120:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880546.431131:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880546.431137:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000007cb951ea type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880546.431144:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880546.431145:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880546.431152:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880562 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880546.431152:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 320578 usecs (321 jiffies) 00000100:00000001:2.0:1788880546.431164:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306177857 00000100:00000001:0.0:1788880546.431169:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880546.431170:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880546.431172:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306182607 00000100:00000001:0.0:1788880546.431175:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880546.431181:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306177857 00000400:00000200:3.0:1788880546.431182:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880546.431190:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306182607 00000400:00000200:3.0:1788880546.431196:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [286:286:254:4294967295] 192.168.204.111@tcp LPNI seq info [286:286:6:4294967295] 00000100:00000001:2.0:1788880546.431198:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306177857 00000100:00000001:2.0:1788880546.431205:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306182607 00000100:00000001:2.0:1788880546.431209:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306177857 00000100:00000001:2.0:1788880546.431211:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306182607 00000100:00000001:2.0:1788880546.431217:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880546.431226:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880546.431226:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880546.431270:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880546.431277:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:3.0:1788880546.431278:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880546.431284:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:3.0:1788880546.431285:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 00000100:00000001:2.0:1788880546.431286:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880546.431292:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880546.431294:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880546.431297:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880546.431305:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880546.431305:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880546.431315:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:3.0:1788880546.431318:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000010:0.0:1788880546.431320:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000007cb951ea. 00000800:00000200:3.0:1788880546.431325:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880546.431327:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880546.431330:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880546.431334:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:3.0:1788880546.431336:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 20590 usecs (21 jiffies) 00000400:00000200:0.0:1788880546.431339:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000100:00000001:3.0:1788880546.431340:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306177558 00000400:00000010:0.0:1788880546.431344:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:0.0:1788880546.431347:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880546.431350:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306182607 00000100:00000200:0.0:1788880546.431354:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880546.431356:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306177558 00000100:00000001:3.0:1788880546.431360:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306182607 00000100:00000001:0.0:1788880546.431369:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880546.431372:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880546.431390:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880546.431392:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880546.431401:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306177558 00000100:00000001:3.0:1788880546.431403:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306182607 00000100:00000001:3.0:1788880546.431405:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306177558 00000100:00000001:3.0:1788880546.431407:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306182607 00000100:00000001:3.0:1788880546.431410:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880546.431412:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880546.431415:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880546.431421:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880546.431426:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880546.431427:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880546.431431:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:0.0:1788880546.431432:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880546.431433:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000010:0.0:1788880546.431437:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000100:00000001:3.0:1788880546.431437:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880546.431441:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880546.431445:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880546.431449:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880546.431453:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 00000400:00000010:0.0:1788880546.431455:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000100:00000001:0.0:1788880546.431458:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880546.431461:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880546.431467:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880546.431469:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880546.431474:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880546.431476:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880546.432627:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880546.432637:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880546.432771:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880546.432775:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880546.432783:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880546.432794:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08880 00000400:00000200:3.0:1788880546.432801:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x9bd [1] + 224 00000800:00000001:3.0:1788880546.432806:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880546.432809:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880546.432811:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880546.432814:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880546.432818:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880546.432823:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880546.432845:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880546.432861:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880546.432900:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880546.432915:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880546.432916:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880546.432926:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880546.432927:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880546.432931:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:3.0:1788880546.432932:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000010:2.0:1788880546.432936:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001f454358. 00000800:00000001:3.0:1788880546.432938:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880546.432946:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000170fdf47 00000400:00000200:3.0:1788880546.432948:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000010:2.0:1788880546.432951:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000170fdf47. 00000100:00000001:2.0:1788880546.432958:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:3.0:1788880546.432958:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08900 00000100:00000200:2.0:1788880546.432968:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880546.432970:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x9c5 [1] + 224 00000800:00000001:3.0:1788880546.432975:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880546.432981:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880546.432989:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880546.433003:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:2.0:1788880546.433008:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880546.433011:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880546.433019:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880546.433037:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880546.433042:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880546.433043:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880546.433054:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880546.433059:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880546.433061:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880546.433063:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880546.433065:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:2.0:1788880546.433068:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880546.433070:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880546.433075:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880546.433076:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880546.433077:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880546.433079:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=64 v=5 (5 5 5 0) 00000400:00000200:3.0:1788880546.433085:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:2.0:1788880546.433090:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=64 v=5 (1 1 1 0) 00000400:00000200:3.0:1788880546.433097:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b08800 00000100:00000001:2.0:1788880546.433103:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880546.433105:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880546.433107:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880546.433108:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x9b5 [1] + 224 00000100:00000001:0.0:1788880546.433109:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:2.0:1788880546.433117:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000001:0.0:1788880546.433118:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788880546.433119:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880546.433127:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00010000:00000001:2.0:1788880546.433127:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880546.433128:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880546.433132:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000bc3da258. 00000100:00000001:2.0:1788880546.433135:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000200:3.0:1788880546.433141:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:2.0:1788880546.433142:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000200:0.0:1788880546.433143:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cf82a3fa 00000400:00000200:3.0:1788880546.433151:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880546.433152:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000010:0.0:1788880546.433153:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cf82a3fa. 00000100:00000001:2.0:1788880546.433160:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880546.433161:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880546.433162:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880546.433168:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880546.433173:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880546.433177:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880546.433205:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880546.433267:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880546.433269:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880546.433274:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880546.433285:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880546.433286:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880546.433292:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880546.433295:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880546.433298:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880546.433301:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880546.433301:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880546.433304:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880546.433307:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880546.433308:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880546.433310:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880546.433317:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=74 v=5 (5 5 5 0) 00000800:00000001:3.0:1788880546.433317:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880546.433318:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880546.433330:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:0.0:1788880546.433331:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=31 v=5 (1 0 0 0) 00000400:00000001:1.0:1788880546.433337:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880546.433341:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880546.433342:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00100000:2.0:1788880546.433344:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780667008:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880546.433346:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880546.433346:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b884f446. 00010000:00000001:0.0:1788880546.433359:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:1.0:1788880546.433360:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000100:00000001:2.0:1788880546.433360:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880546.433365:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:2.0:1788880546.433367:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00010000:00080000:0.0:1788880546.433371:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880546.433371:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:2.0:1788880546.433377:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000451fe31a x1875776780667008/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880546.433381:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880546.433430:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880546.433437:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000200:1.0:1788880546.433438:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880546.433443:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880546.433449:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880546.433456:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880546.433472:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880546.433503:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880546.433505:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:2.0:1788880546.433510:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880546.433511:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880546.433511:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880546.433519:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000010:2.0:1788880546.433519:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003816d5cd. 00000100:00000001:0.0:1788880546.433520:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880546.433526:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880546.433527:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880546.433530:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:2.0:1788880546.433530:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880546.433533:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880546.433534:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:2.0:1788880546.433536:0:201285:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000001:1.0:1788880546.433538:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880546.433544:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00001000:1.0:1788880546.433545:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=74 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880546.433557:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=74 v=5 (1 1 1 0) 00000100:00100000:0.0:1788880546.433558:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780667136:192.168.204.111@tcp:400: 00000020:00000001:2.0:1788880546.433558:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880546.433566:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000010:2.0:1788880546.433567:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000004e0f6846. 00000100:00000001:1.0:1788880546.433569:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880546.433570:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880546.433576:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880546.433577:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:2.0:1788880546.433577:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00010000:00000001:1.0:1788880546.433578:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880546.433583:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880546.433583:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:0.0:1788880546.433587:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000007f91037 x1875776780667136/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788880546.433590:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000451fe31a. 00000100:00000040:1.0:1788880546.433596:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880546.433600:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880546.433607:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880546.433611:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880546.433616:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880546.433618:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880546.433619:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880546.433623:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000010:0.0:1788880546.433624:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000081a10824. 00000100:00000001:1.0:1788880546.433624:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880546.433626:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880546.433632:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880546.433633:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880546.433635:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880546.433644:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:0.0:1788880546.433647:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880546.433649:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880546.433653:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00100000:1.0:1788880546.433653:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780666880:192.168.204.111@tcp:400: 00000020:00000001:0.0:1788880546.433660:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880546.433664:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c93f60a2. 02000000:00000001:0.0:1788880546.433670:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880546.433671:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880546.433673:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880546.433675:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000010:0.0:1788880546.433677:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000007f91037. 00000100:00000040:1.0:1788880546.433679:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000009ee4c6bf x1875776780666880/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880562 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880546.433682:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880546.433684:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880546.433688:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880546.433690:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880546.433691:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880546.433698:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880546.433700:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880546.433701:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000356bd8da. 00000100:00000001:0.0:1788880546.433703:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880546.433706:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880546.433707:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880546.433710:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880546.433710:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880546.433712:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880546.433712:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 02000000:00000001:1.0:1788880546.433714:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880546.433715:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880546.433716:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000009ee4c6bf. 00000100:00000001:1.0:1788880546.433718:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880546.433719:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880546.433720:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880546.433723:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880546.433724:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880546.433726:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880546.433728:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880546.494310:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880546.494318:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880546.494326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880546.494331:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880546.494334:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880546.494337:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880546.494341:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880546.494343:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880546.494345:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880546.494348:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880546.494387:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880546.494391:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880546.494393:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880547.518360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880547.518383:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880547.518391:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880547.518392:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880547.518459:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880547.518466:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880547.518471:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880548.542361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880548.542367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880548.542373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880548.542374:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880548.542402:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880548.542404:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880548.542406:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880549.566360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880549.566368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880549.566375:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880549.566376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880549.566442:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880549.566446:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880549.566449:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880550.590335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880550.590340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880550.590344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880550.590345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880550.590361:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880550.590363:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880550.590365:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880551.550403:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306182607, at 4306182656 00000100:00000040:2.0:1788880551.550403:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306182607, at 4306182656 00000100:00000010:2.0:1788880551.550431:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000010:1.0:1788880551.550441:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000040:0.0:1788880551.550592:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306182607, at 4306182656 00000100:00000010:0.0:1788880551.550598:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000040:3.0:1788880551.550712:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880551.550720:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880551.550740:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788880551.550749:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880551.550754:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880551.550757:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880551.550763:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880551.550773:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f92e72a2. 00000100:00000001:3.0:1788880551.550781:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880551.550795:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880551.550824:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880551.550827:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880551.550835:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_02+2] req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880551.550875:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880551.550895:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880551.550902:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880551.550907:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880551.550910:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880551.550912:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880551.550914:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880551.550915:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880551.550919:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000446e4ede. 00000100:00000001:3.0:1788880551.550923:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880551.550926:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880551.550934:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880551.550935:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880551.550938:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_03+3] req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880551.550945:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880551.550947:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880551.550950:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880551.550953:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880551.550955:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880551.550957:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880551.550958:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880551.550959:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880551.550962:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000088542ca3. 00000100:00000001:3.0:1788880551.550964:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880551.550966:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880551.550972:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880551.550973:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880551.550975:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_00+0] req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880551.551012:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880551.551015:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880551.551020:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880551.551021:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880551.551029:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880551.551031:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880551.551037:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880551.551039:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11520) 00000100:00000001:2.0:1788880551.551043:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880551.551047:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880551.551048:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880551.551056:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880551.551057:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880551.551064:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880551.551075:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880551.551082:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880551.551084:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880551.551087:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880551.551087:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306182607, at 4306182656 00000100:00000001:1.0:1788880551.551089:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880551.551102:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880551.551104:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000010:3.0:1788880551.551106:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000001:2.0:1788880551.551109:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:1.0:1788880551.551112:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880551.551116:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880551.551123:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880551.551124:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880551.551128:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880551.551135:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880551.551136:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780667392:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880551.551142:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00100000:2.0:1788880551.551144:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780667520:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880551.551147:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880551.551152:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880551.551154:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880551.551160:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880551.551164:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880551.551169:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880551.551169:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00100000:0.0:1788880551.551176:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780667264:192.168.204.111@tcp:400: 02000000:00000001:1.0:1788880551.551179:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880551.551183:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880551.551185:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880551.551185:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880551.551189:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:0.0:1788880551.551191:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880551.551192:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880551.551198:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004d9329ab. 02000000:00000001:0.0:1788880551.551200:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880551.551201:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cd0104b. 02000000:00000001:2.0:1788880551.551204:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880551.551206:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:1.0:1788880551.551207:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880551.551214:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000006c62949. 00000400:00000010:2.0:1788880551.551218:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001f454358. 02000000:00000001:0.0:1788880551.551219:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880551.551219:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b884f446. 00000400:00000010:0.0:1788880551.551231:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000878f51f0. 00000400:00000010:0.0:1788880551.551268:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000400:00000010:2.0:1788880551.551270:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000170fdf47. 00000400:00000010:1.0:1788880551.551278:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000200:2.0:1788880551.551283:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780667520, portal 10 00000100:00000200:0.0:1788880551.551287:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780667264, portal 25 00000100:00000200:1.0:1788880551.551287:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780667392, portal 10 00000100:00000001:2.0:1788880551.551289:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880551.551294:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000100:00000001:1.0:1788880551.551295:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880551.551296:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880551.551302:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000001:1.0:1788880551.551303:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000040:2.0:1788880551.551313:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880551.551317:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880567 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880551.551319:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880551.551370:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880551.551384:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880551.551391:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880551.551392:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:2.0:1788880551.551393:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009c31f261. 00000100:00000040:1.0:1788880551.551403:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788880551.551403:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780667520, offset 0 00000100:00000040:0.0:1788880551.551404:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880551.551414:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000400:00000010:1.0:1788880551.551414:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e249bdf9. 00000400:00000200:2.0:1788880551.551423:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:0.0:1788880551.551445:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780667264, offset 0 00000100:00000200:1.0:1788880551.551446:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780667392, offset 0 00000400:00000200:2.0:1788880551.551458:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880551.551460:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880551.551468:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880551.551475:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880551.551477:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880551.551486:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880551.551489:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880551.551491:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [287:287:256:4294967295] 192.168.204.111@tcp LPNI seq info [287:287:8:4294967295] 00000400:00000200:2.0:1788880551.551502:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880551.551507:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880551.551512:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006aca3ff7. 00000400:00000200:0.0:1788880551.551517:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:2.0:1788880551.551522:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880551.551527:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880551.551533:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880551.551536:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880551.551539:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006aca3ff7 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880551.551546:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880551.551551:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880551.551554:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [288:288:255:4294967295] 192.168.204.111@tcp LPNI seq info [288:288:7:4294967295] 00000100:00000001:2.0:1788880551.551556:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880551.551565:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 20366 usecs (21 jiffies) 00000100:00000001:2.0:1788880551.551571:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306182678 00000400:00000200:0.0:1788880551.551573:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880551.551585:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880551.551588:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000077b36062. 00000800:00000010:0.0:1788880551.551593:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c93f60a2. 00000400:00000200:1.0:1788880551.551593:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880551.551597:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000400:00000200:1.0:1788880551.551604:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880551.551611:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306182678 00000400:00000200:1.0:1788880551.551612:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:2.0:1788880551.551620:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000007e595eb4. 00000800:00000200:0.0:1788880551.551621:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880551.551622:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880551.551631:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000400:00000200:1.0:1788880551.551632:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788880551.551639:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880551.551639:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306182678 00000400:00000200:1.0:1788880551.551645:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [289:289:254:4294967295] 192.168.204.111@tcp LPNI seq info [289:289:6:4294967295] 00000800:00000200:0.0:1788880551.551652:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c93f60a2 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880551.551659:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880551.551661:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a3bf26bf. 00000400:00000200:1.0:1788880551.551665:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880551.551666:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306187728 00000100:00000001:2.0:1788880551.551669:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880551.551683:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306182678 00000100:00000001:2.0:1788880551.551683:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306182678 00000800:00000200:1.0:1788880551.551685:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880551.551695:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000668fd8a. 00000100:00000001:0.0:1788880551.551699:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306187728 00000800:00000010:1.0:1788880551.551699:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:2.0:1788880551.551702:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880551.551712:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306182678 00000100:00000001:2.0:1788880551.551713:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880551.551715:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880551.551721:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306187728 00000100:00000001:2.0:1788880551.551726:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880551.551734:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880551.551736:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880551.551741:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006aca3ff7. 00000100:00000001:2.0:1788880551.551743:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:1.0:1788880551.551747:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880551.551752:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880551.551753:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880551.551758:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880551.551765:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880551.551767:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880551.551772:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880551.551773:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880551.551780:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880551.551782:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 316660 usecs (317 jiffies) 00000100:00000001:2.0:1788880551.551783:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880551.551785:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009c31f261 00000100:00000001:1.0:1788880551.551788:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306182974 00000100:00000001:2.0:1788880551.551790:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880551.551791:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009c31f261. 00000100:00000001:3.0:1788880551.551800:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880551.551802:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306187728 00000100:00000001:0.0:1788880551.551803:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306182678 00000100:00000001:1.0:1788880551.551818:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306182974 00000100:00000200:3.0:1788880551.551819:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880551.551827:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306187728 00000100:00000001:0.0:1788880551.551835:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306187728 00000100:00000001:1.0:1788880551.551842:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306182974 00000100:00000001:0.0:1788880551.551850:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306182678 00000100:00000001:1.0:1788880551.551855:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306187728 00000100:00000001:0.0:1788880551.551860:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880551.551864:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306182974 00000100:00000001:3.0:1788880551.551865:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880551.551866:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880551.551871:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306187728 00000100:00000001:0.0:1788880551.551875:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880551.551878:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880551.551880:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880551.551883:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880551.551891:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880551.551899:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880551.551901:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880551.551906:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880551.551910:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880551.551913:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880551.551915:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880551.551916:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880551.551918:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880551.551922:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880551.551925:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880551.551926:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880551.551930:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880551.551935:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880551.551938:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880551.551940:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880551.551942:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880551.552121:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880551.552125:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c93f60a2. 00000400:00000200:3.0:1788880551.552131:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880551.552135:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880551.552138:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000010:3.0:1788880551.552139:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:3.0:1788880551.552142:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880551.552145:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880567 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880551.552163:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880551.552166:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880567 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880551.552172:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880551.552173:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880551.552211:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880551.552213:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:3.0:1788880551.552217:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880551.552220:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880551.552223:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e249bdf9 00000400:00000010:3.0:1788880551.552224:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e249bdf9. 00000100:00000001:3.0:1788880551.552227:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880551.552230:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880551.552306:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880551.552309:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880551.552314:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880551.552316:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880551.554844:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880551.554857:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880551.554863:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880551.554865:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880551.554878:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880551.554896:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08a80 00000400:00000200:0.0:1788880551.554908:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x9cd [1] + 224 00000800:00000001:0.0:1788880551.554916:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880551.554920:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880551.554923:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880551.554928:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880551.554937:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880551.554945:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880551.554957:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880551.554999:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880551.555008:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880551.555011:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880551.555013:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880551.555031:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880551.555035:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880551.555043:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b08980 00000400:00000200:0.0:1788880551.555055:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x9d1 [1] + 224 00000800:00000001:0.0:1788880551.555061:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880551.555068:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:0.0:1788880551.555069:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880551.555075:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880551.555078:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880551.555081:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880551.555086:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:1.0:1788880551.555091:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:0.0:1788880551.555092:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880551.555096:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880551.555100:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001f454358. 00000100:00000040:0.0:1788880551.555112:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880551.555116:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000170fdf47 00000400:00000010:1.0:1788880551.555125:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000170fdf47. 00000100:00000001:1.0:1788880551.555132:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880551.555141:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880551.555143:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880551.555150:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880551.555158:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880551.555160:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880551.555173:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880551.555175:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880551.555178:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880551.555191:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880551.555194:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880551.555196:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880551.555198:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880551.555207:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=79 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880551.555211:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=36 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880551.555213:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880551.555215:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880551.555217:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880551.555222:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880551.555229:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880551.555281:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880551.555284:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880551.555286:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880551.555287:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880551.555291:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880551.555298:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880551.555300:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880551.555303:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880551.555307:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880551.555315:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780667520:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880551.555322:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880551.555325:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880551.555326:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780667520/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880551.555331:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880551.555333:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880551.555336:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004d9329ab. 02000000:00000001:1.0:1788880551.555342:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880551.555353:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880551.555354:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880551.555358:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880551.555361:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000088542ca3. 02000000:00000001:1.0:1788880551.555369:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880551.555372:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880551.555375:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:1.0:1788880551.555379:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880551.555381:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880551.555384:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880551.555392:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880551.555394:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880551.555396:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880551.555398:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880551.555402:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880551.555404:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880551.555413:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880551.555417:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880551.555420:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000878f51f0. 00000400:00000200:3.0:1788880551.555426:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000400:00000010:3.0:1788880551.555430:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00000001:3.0:1788880551.555433:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880551.555439:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880551.555449:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880551.555495:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880551.555505:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880551.555508:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880551.555511:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880551.555513:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880551.555516:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880551.555519:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880551.555524:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=79 v=5 (5 5 5 0) 00000100:00001000:3.0:1788880551.555528:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=79 v=5 (1 1 1 0) 00000100:00000001:3.0:1788880551.555531:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880551.555533:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880551.555535:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880551.555538:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880551.555540:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880551.555545:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880551.555551:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880551.555554:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880551.555556:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880551.555559:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880551.555566:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780667264:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880551.555571:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880551.555574:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880551.555576:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780667264/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880551.555583:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880551.555585:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880551.555588:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000006c62949. 02000000:00000001:3.0:1788880551.555592:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880551.555593:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880551.555595:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880551.555597:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880551.555599:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f92e72a2. 02000000:00000001:3.0:1788880551.555603:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880551.555611:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880551.555612:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:3.0:1788880551.555616:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880551.555617:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880551.555619:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880551.555626:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880551.555628:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880551.555629:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880551.555632:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880551.555692:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880551.555699:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880551.555701:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880551.555703:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880551.555708:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880551.555715:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08a00 00000400:00000200:0.0:1788880551.555721:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x9d5 [1] + 224 00000800:00000001:0.0:1788880551.555725:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880551.555729:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880551.555730:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880551.555733:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880551.555737:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880551.555740:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880551.555748:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880551.555772:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880551.555780:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880551.555784:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880551.555830:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880551.555835:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880551.555839:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880551.555842:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880551.555843:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b884f446. 00000400:00000200:1.0:1788880551.555848:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:1.0:1788880551.555851:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:1.0:1788880551.555855:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880551.555860:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880551.555871:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880551.555877:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880551.555878:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880551.555880:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880551.555882:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880551.555883:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880551.555884:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880551.555886:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880551.555891:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=69 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880551.555895:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=69 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880551.555897:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880551.555898:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880551.555907:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880551.555911:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880551.555914:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880551.555916:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880551.555917:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880551.555918:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880551.555921:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880551.555926:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880551.555927:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880551.555928:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880551.555930:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880551.555934:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780667392:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880551.555936:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880551.555938:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880551.555939:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780667392/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880567 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880551.555942:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880551.555943:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880551.555945:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cd0104b. 02000000:00000001:1.0:1788880551.555947:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880551.555948:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880551.555949:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880551.555951:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880551.555951:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000446e4ede. 02000000:00000001:1.0:1788880551.555954:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880551.555955:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880551.555955:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:1.0:1788880551.555958:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880551.555959:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880551.555960:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880551.555963:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880551.555967:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880551.555968:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880551.555969:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880551.614313:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880551.614320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880551.614327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880551.614328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880551.614411:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880551.614415:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880551.614417:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880552.638357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880552.638364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880552.638372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880552.638373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880552.638392:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880552.638395:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880552.638398:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880553.662337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880553.662343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880553.662348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880553.662349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880553.662400:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880553.662405:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880553.662408:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880554.686362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880554.686371:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880554.686373:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880554.686376:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880554.686381:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880554.686383:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880554.686385:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880554.686387:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880554.686392:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880554.686393:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880554.686417:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880554.686420:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880554.686422:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880555.710334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880555.710341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880555.710348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880555.710349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880555.710436:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880555.710442:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880555.710445:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880556.670351:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880556.670351:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306187728, at 4306187776 00000100:00000040:2.0:1788880556.670351:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306187728, at 4306187776 00000100:00000040:3.0:1788880556.670351:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306187728, at 4306187776 00000100:00000001:0.0:1788880556.670367:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880556.670372:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000007e595eb4. 00000100:00000010:2.0:1788880556.670373:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a3bf26bf. 00000100:00000010:3.0:1788880556.670375:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000668fd8a. 00000100:00000010:0.0:1788880556.670380:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000007f91037. 00000020:00000040:0.0:1788880556.670387:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880556.670395:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880556.670399:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880556.670405:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880556.670414:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000030024c2c. 00000100:00000001:0.0:1788880556.670423:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880556.670436:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880556.670450:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880556.670454:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880556.670461:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000007f91037] to pc [ptlrpcd_00_01+1] req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880556.670487:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880556.670490:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880556.670493:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880556.670494:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880556.670498:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:0.0:1788880556.670499:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000937cf312. 00000100:00000001:3.0:1788880556.670502:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:0.0:1788880556.670503:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880556.670508:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:3.0:1788880556.670508:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880556.670510:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880556.670514:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880556.670519:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000131c1a55. 00000100:00000001:3.0:1788880556.670521:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880556.670524:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880556.670525:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880556.670527:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880556.670530:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880556.670532:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:3.0:1788880556.670534:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880556.670542:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780667648:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880556.670548:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880556.670553:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880556.670556:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880556.670560:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880556.670565:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880556.670566:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880556.670570:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003720a685. 02000000:00000001:3.0:1788880556.670574:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880556.670575:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000937cf312] to pc [ptlrpcd_00_02+2] req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880556.670581:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bfe5eb98. 00000400:00000010:3.0:1788880556.670589:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 00000100:00000001:1.0:1788880556.670595:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:3.0:1788880556.670596:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780667648, portal 25 00000100:00000001:3.0:1788880556.670600:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880556.670601:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880556.670602:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880556.670602:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849371264 : -126822860180352 : ffff8ca7c0783480) 00000100:00000040:0.0:1788880556.670608:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880556.670611:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880572 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880556.670614:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880556.670619:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000a2475c3c. 00000100:00000001:3.0:1788880556.670620:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000020:00000040:0.0:1788880556.670624:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:3.0:1788880556.670624:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880556.670628:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000999bcd04. 00000100:00000001:0.0:1788880556.670629:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880556.670631:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:3.0:1788880556.670631:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780667648, offset 0 02000000:00000001:0.0:1788880556.670633:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880556.670635:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880556.670637:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880556.670638:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:3.0:1788880556.670642:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880556.670646:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880556.670648:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880556.670649:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000ac392bfe. 00000100:00000001:1.0:1788880556.670652:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880556.670656:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880556.670658:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:0.0:1788880556.670666:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880556.670670:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880556.670673:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880556.670679:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880556.670681:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880556.670684:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a2475c3c] to pc [ptlrpcd_00_03+3] req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880556.670688:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880556.670697:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880556.670700:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880556.670701:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880556.670706:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1788880556.670720:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780667776:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880556.670723:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880556.670727:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880556.670731:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880556.670733:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880556.670735:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880556.670736:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880556.670738:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11525) 00000100:00000001:1.0:1788880556.670740:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:2.0:1788880556.670742:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000020450263. 00000100:00000001:1.0:1788880556.670743:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880556.670747:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880556.670750:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788880556.670753:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880556.670754:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000400:00000200:3.0:1788880556.670757:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:0.0:1788880556.670759:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306187728, at 4306187776 00000400:00000010:2.0:1788880556.670761:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000084794ee4. 00000100:00000010:0.0:1788880556.670769:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000077b36062. 00000100:00000200:2.0:1788880556.670769:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780667776, portal 10 00000100:00000001:1.0:1788880556.670770:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880556.670774:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880556.670775:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880556.670778:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849367680 : -126822860183936 : ffff8ca7c0782680) 02000000:00000001:1.0:1788880556.670780:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880556.670783:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880556.670787:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880556.670789:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880556.670793:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780667904:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880556.670802:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880556.670806:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880556.670808:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880556.670812:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880556.670812:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880556.670816:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:2.0:1788880556.670817:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cc94f399. 02000000:00000010:1.0:1788880556.670820:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000cf77ae66. 00000100:00000200:2.0:1788880556.670822:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780667776, offset 0 02000000:00000001:1.0:1788880556.670825:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880556.670832:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b884f446. 00000400:00000200:2.0:1788880556.670832:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880556.670840:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000200:1.0:1788880556.670845:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780667904, portal 10 00000100:00000001:1.0:1788880556.670847:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880556.670848:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849361408 : -126822860190208 : ffff8ca7c0780e00) 00000100:00000040:1.0:1788880556.670854:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880556.670862:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880556.670865:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880556.670868:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 00000100:00000200:1.0:1788880556.670870:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780667904, offset 0 00000400:00000200:1.0:1788880556.670876:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880556.670879:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880556.670883:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880556.670885:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [290:290:256:4294967295] 192.168.204.111@tcp LPNI seq info [290:290:8:4294967295] 00000400:00000200:3.0:1788880556.670895:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880556.670901:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880556.670905:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:3.0:1788880556.670906:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000400:00000200:2.0:1788880556.670910:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880556.670916:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880556.670917:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880556.670921:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880556.670925:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880556.670928:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [291:291:255:4294967295] 192.168.204.111@tcp LPNI seq info [291:291:7:4294967295] 00000400:00000200:2.0:1788880556.670939:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880556.670945:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880556.670948:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880556.670951:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000800:00000200:3.0:1788880556.670958:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880556.670961:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880556.670962:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:3.0:1788880556.670963:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880556.670969:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880556.670973:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880556.670975:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880556.670976:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880556.670981:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880556.670983:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306192847 00000100:00000010:3.0:1788880556.670987:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fbefa178. 00000400:00000200:1.0:1788880556.670988:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [292:292:254:4294967295] 192.168.204.111@tcp LPNI seq info [292:292:6:4294967295] 00000800:00000200:2.0:1788880556.670989:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880556.670994:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:2.0:1788880556.670997:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880556.671000:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306192847 00000400:00000200:1.0:1788880556.671004:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:2.0:1788880556.671004:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 312873 usecs (313 jiffies) 00000100:00000010:3.0:1788880556.671010:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000945234cf. 00000100:00000001:2.0:1788880556.671011:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306188089 00000100:00000001:3.0:1788880556.671013:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:1.0:1788880556.671017:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880556.671017:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306192847 00000100:00000001:2.0:1788880556.671022:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306188089 00000800:00000010:1.0:1788880556.671023:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:2.0:1788880556.671031:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306192847 00000100:00000001:3.0:1788880556.671031:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306192847 00000800:00000200:1.0:1788880556.671032:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880556.671036:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:3.0:1788880556.671036:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001f846689. 00000100:00000001:2.0:1788880556.671037:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306188089 00000100:00000001:3.0:1788880556.671041:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880556.671048:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306192847 00000100:00000001:3.0:1788880556.671048:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306192847 00000800:00000010:0.0:1788880556.671049:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f92e72a2. 00000800:00000200:1.0:1788880556.671049:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:3.0:1788880556.671055:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000016ab1a15. 00000100:00000001:2.0:1788880556.671056:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306188089 00000800:00000200:1.0:1788880556.671057:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880556.671059:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880556.671060:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880556.671064:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880556.671067:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306192847 00000100:00000001:3.0:1788880556.671067:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880556.671070:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 20175 usecs (21 jiffies) 00000400:00000200:0.0:1788880556.671072:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880556.671074:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880556.671074:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880556.671076:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306187797 00000100:00000001:1.0:1788880556.671082:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306192847 00000100:00000001:2.0:1788880556.671082:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880556.671082:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880556.671083:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000999bcd04 00000100:00000001:2.0:1788880556.671088:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880556.671089:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306187797 00000400:00000010:0.0:1788880556.671090:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000999bcd04. 00000100:00000001:3.0:1788880556.671092:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880556.671093:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880556.671094:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306192847 00000100:00000001:0.0:1788880556.671099:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880556.671099:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306187797 00000100:00000001:1.0:1788880556.671103:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306192847 00000100:00000001:3.0:1788880556.671103:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880556.671104:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880556.671105:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306187797 00000100:00000200:0.0:1788880556.671107:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880572 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880556.671107:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880556.671108:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880556.671109:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306192847 00000100:00000001:3.0:1788880556.671112:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880556.671115:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880556.671115:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880556.671116:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880556.671119:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880556.671120:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880556.671121:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880556.671124:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880556.671125:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880556.671126:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880556.671130:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880556.671131:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880556.671139:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880556.671141:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880556.671142:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880556.671147:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880556.671148:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880572 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880556.671155:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880556.671159:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880556.671161:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880556.671163:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880556.671165:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880556.671185:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880556.671187:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000400:00000200:0.0:1788880556.671191:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880556.671195:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880556.671198:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cc94f399 00000400:00000010:0.0:1788880556.671200:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cc94f399. 00000100:00000001:0.0:1788880556.671203:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880556.671206:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880556.671213:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880556.671216:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880556.671224:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880556.671225:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880556.671294:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880556.671297:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:0.0:1788880556.671309:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880556.671312:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880556.671315:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000400:00000010:0.0:1788880556.671317:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000001:0.0:1788880556.671326:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880556.671329:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880556.671337:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880556.671339:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880556.671345:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880556.671347:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880556.672643:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880556.672652:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880556.672657:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880556.672659:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880556.672667:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880556.672679:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b08b00 00000400:00000200:0.0:1788880556.672688:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x9e5 [1] + 224 00000800:00000001:0.0:1788880556.672695:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880556.672701:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880556.672703:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880556.672707:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880556.672726:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880556.672730:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880556.672741:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880556.672758:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880556.672766:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880556.672770:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880556.672859:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880556.672863:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880556.672870:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880556.672872:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880556.672874:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000bfe5eb98. 00000800:00000001:0.0:1788880556.672877:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000200:3.0:1788880556.672878:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000400:00000010:3.0:1788880556.672882:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000800:00000001:0.0:1788880556.672885:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880556.672886:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:0.0:1788880556.672890:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1788880556.672894:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880556.672896:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880556.672906:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880556.672914:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08b80 00000400:00000200:0.0:1788880556.672927:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x9ed [1] + 224 00000800:00000001:0.0:1788880556.672931:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880556.672936:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880556.672938:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880556.672941:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880556.672944:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880556.672947:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880556.672955:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880556.672971:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880556.672976:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1788880556.672999:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880556.673005:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880556.673008:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880556.673010:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880556.673010:0:201286:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:0.0:1788880556.673015:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880556.673015:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880556.673019:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:0.0:1788880556.673020:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788880556.673021:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000001:2.0:1788880556.673022:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880556.673025:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000800:00000001:0.0:1788880556.673026:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880556.673030:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000084794ee4 00000400:00000010:2.0:1788880556.673035:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000084794ee4. 00000400:00000200:0.0:1788880556.673037:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880556.673039:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880556.673046:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880556.673048:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08c00 00000400:00000200:0.0:1788880556.673059:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x9f5 [1] + 224 00000800:00000001:0.0:1788880556.673066:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:2.0:1788880556.673067:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880556.673073:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880556.673078:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880556.673086:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:0.0:1788880556.673087:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880556.673088:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880556.673090:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880556.673093:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880556.673095:0:201280:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1788880556.673096:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880556.673098:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000200:0.0:1788880556.673101:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880556.673102:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00001000:3.0:1788880556.673104:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=84 v=5 (5 5 5 0) 00000100:00000001:2.0:1788880556.673106:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880556.673109:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880556.673112:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=84 v=5 (1 1 1 0) 00000100:00000001:3.0:1788880556.673118:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880556.673121:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880556.673123:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880556.673124:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880556.673124:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880556.673128:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880556.673132:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=74 v=5 (5 5 5 0) 00000100:00000001:3.0:1788880556.673132:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880556.673140:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=74 v=5 (1 1 1 0) 00000100:00000040:3.0:1788880556.673140:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880556.673145:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880556.673147:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880556.673151:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880556.673152:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:2.0:1788880556.673157:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880556.673159:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880556.673159:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880556.673163:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000800:00000001:0.0:1788880556.673165:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880556.673166:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000040:2.0:1788880556.673167:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880556.673171:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880556.673171:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880556.673175:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880556.673176:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880556.673180:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880556.673180:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880556.673184:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880556.673185:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880556.673188:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880556.673190:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b884f446. 00000400:00000200:1.0:1788880556.673195:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:1.0:1788880556.673198:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:1.0:1788880556.673203:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:3.0:1788880556.673204:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780667648:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880556.673209:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880556.673210:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880556.673212:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880556.673216:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880556.673217:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880556.673221:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880556.673223:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000007f91037 x1875776780667648/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880556.673227:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880556.673229:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880556.673257:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880556.673260:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880556.673263:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003720a685. 02000000:00000001:3.0:1788880556.673274:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880556.673277:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880556.673280:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880556.673285:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:3.0:1788880556.673286:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00100000:2.0:1788880556.673288:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780667776:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880556.673290:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1788880556.673291:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000030024c2c. 00000100:00000001:1.0:1788880556.673296:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880556.673297:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880556.673299:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880556.673300:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880556.673301:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:3.0:1788880556.673303:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880556.673304:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:2.0:1788880556.673307:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000937cf312 x1875776780667776/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:3.0:1788880556.673307:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000007f91037. 00000100:00000001:1.0:1788880556.673308:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880556.673314:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880556.673321:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=84 v=5 (5 5 5 0) 00000100:00000001:3.0:1788880556.673321:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880556.673322:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880556.673325:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880556.673326:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00001000:1.0:1788880556.673328:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=41 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880556.673330:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880556.673335:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000010:2.0:1788880556.673336:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000020450263. 00000100:00000001:1.0:1788880556.673339:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880556.673339:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788880556.673343:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880556.673343:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880556.673344:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:2.0:1788880556.673347:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880556.673348:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880556.673351:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00010000:00080000:1.0:1788880556.673352:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880556.673353:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880556.673357:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880556.673361:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000131c1a55. 00010000:00000001:1.0:1788880556.673363:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880556.673366:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880556.673367:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:2.0:1788880556.673369:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:1.0:1788880556.673370:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000010:2.0:1788880556.673372:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000937cf312. 00000100:00000001:1.0:1788880556.673374:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880556.673377:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880556.673377:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880556.673380:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880556.673384:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880556.673385:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880556.673389:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880556.673390:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880556.673394:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880556.673397:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880556.673405:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880556.673408:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880556.673410:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880556.673414:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880556.673421:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780667904:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880556.673424:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880556.673425:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880556.673427:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000a2475c3c x1875776780667904/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880572 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880556.673431:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880556.673432:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880556.673433:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000cf77ae66. 02000000:00000001:1.0:1788880556.673435:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880556.673436:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880556.673436:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880556.673438:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880556.673439:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000ac392bfe. 02000000:00000001:1.0:1788880556.673440:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880556.673441:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880556.673442:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000a2475c3c. 00000100:00000001:1.0:1788880556.673444:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880556.673445:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880556.673446:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880556.673449:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880556.673450:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880556.673451:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880556.673452:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880556.734374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880556.734384:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880556.734399:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880556.734403:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880556.734435:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880556.734438:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880556.734443:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880556.734445:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880556.734446:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880556.734448:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880556.734467:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880556.734469:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880556.734471:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880557.758351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880557.758360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880557.758366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880557.758367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880557.758726:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880557.758730:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880557.758733:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880558.782347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880558.782354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880558.782360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880558.782361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880558.782380:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788880558.782383:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880558.782386:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880559.806330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880559.806338:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880559.806344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880559.806346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880559.806405:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880559.806409:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880559.806411:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880560.830309:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880560.830316:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880560.830321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880560.830322:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880560.830335:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880560.830338:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880560.830340:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880561.790328:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306192847, at 4306192896 00000100:00000040:2.0:1788880561.790328:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880561.790328:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306192847, at 4306192896 00000100:00000040:0.0:1788880561.790335:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306192847, at 4306192896 00000100:00000010:1.0:1788880561.790342:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000945234cf. 00000100:00000001:2.0:1788880561.790342:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880561.790344:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000016ab1a15. 00000100:00000010:0.0:1788880561.790346:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fbefa178. 00000100:00000010:2.0:1788880561.790351:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880561.790356:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880561.790359:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880561.790362:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880561.790365:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880561.790371:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003139f6ba. 00000100:00000001:2.0:1788880561.790377:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880561.790386:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880561.790393:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880561.790394:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880561.790398:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_00_00+0] req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880561.790414:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880561.790427:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880561.790429:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880561.790430:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880561.790433:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000100:00000001:1.0:1788880561.790434:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:2.0:1788880561.790435:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880561.790437:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880561.790437:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880561.790439:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880561.790441:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880561.790442:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880561.790443:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880561.790445:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880561.790447:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5d1e8ed. 00000100:00000001:2.0:1788880561.790452:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880561.790458:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880561.790460:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880561.790467:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880561.790467:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880561.790468:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880561.790472:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880561.790472:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fd6b7ee1] to pc [ptlrpcd_00_01+1] req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880561.790476:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880561.790480:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880561.790486:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780668032:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880561.790490:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880561.790490:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880561.790493:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880561.790494:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880561.790496:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1788880561.790497:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880561.790500:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:2.0:1788880561.790500:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000020:00000040:2.0:1788880561.790503:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880561.790504:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880561.790505:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000abce1aea. 00000100:00000001:2.0:1788880561.790506:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880561.790507:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880561.790509:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880561.790510:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880561.790511:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880561.790512:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880561.790516:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000040:3.0:1788880561.790517:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880561.790518:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b884f446. 00000100:00000001:2.0:1788880561.790519:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880561.790525:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000040:2.0:1788880561.790527:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880561.790532:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000200:1.0:1788880561.790534:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780668032, portal 25 00000100:00000001:3.0:1788880561.790535:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880561.790539:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:3.0:1788880561.790540:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880561.790542:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 02000000:00000001:3.0:1788880561.790544:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880561.790547:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880561.790549:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880561.790551:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880577 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880561.790551:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880561.790556:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780668160:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880561.790561:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d838cab4] to pc [ptlrpcd_00_02+2] req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880561.790566:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880561.790571:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880561.790571:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880561.790575:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880561.790578:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880561.790580:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:1.0:1788880561.790581:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000066e8f89d. 00000100:00000200:1.0:1788880561.790584:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780668032, offset 0 02000000:00000010:3.0:1788880561.790586:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003720a685. 00000400:00000200:1.0:1788880561.790591:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880561.790592:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880561.790601:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bfe5eb98. 00000400:00000200:1.0:1788880561.790605:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:3.0:1788880561.790622:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 00000400:00000200:1.0:1788880561.790625:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:3.0:1788880561.790626:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780668160, portal 10 00000100:00000001:2.0:1788880561.790630:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880561.790631:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880561.790632:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880561.790634:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000100:00000040:2.0:1788880561.790636:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880561.790642:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:1.0:1788880561.790643:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880561.790646:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880561.790651:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:2.0:1788880561.790651:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11530) 00000100:00000001:0.0:1788880561.790653:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880561.790657:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880561.790660:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788880561.790663:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [293:293:256:4294967295] 192.168.204.111@tcp LPNI seq info [293:293:8:4294967295] 00000100:00000001:3.0:1788880561.790664:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880561.790668:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880561.790671:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306192847, at 4306192896 00000100:00000040:3.0:1788880561.790671:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880561.790676:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000035d99bc2. 00000100:00000010:2.0:1788880561.790681:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001f846689. 00000100:00000200:3.0:1788880561.790682:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780668160, offset 0 00000100:00000001:0.0:1788880561.790692:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:3.0:1788880561.790694:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880561.790695:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880561.790696:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880561.790702:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880561.790703:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880561.790706:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880561.790707:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880561.790710:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880561.790711:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880561.790711:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880561.790718:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:1.0:1788880561.790719:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000400:00000200:3.0:1788880561.790724:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00100000:0.0:1788880561.790728:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780668288:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880561.790729:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:1.0:1788880561.790737:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880561.790738:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1788880561.790738:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788880561.790744:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:3.0:1788880561.790746:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [294:294:255:4294967295] 192.168.204.111@tcp LPNI seq info [294:294:7:4294967295] 02000000:00000001:0.0:1788880561.790748:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880561.790753:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:0.0:1788880561.790754:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880561.790760:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d2602e97. 00000800:00000200:1.0:1788880561.790760:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880561.790763:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880561.790765:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880561.790772:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000edb03a4b. 00000800:00000200:3.0:1788880561.790776:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880561.790778:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000096cf0c4a. 00000100:00000001:1.0:1788880561.790779:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880561.790782:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000088d6bc1. 00000100:00000200:0.0:1788880561.790783:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780668288, portal 10 00000100:00000001:1.0:1788880561.790786:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306197967 00000100:00000001:0.0:1788880561.790788:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880561.790792:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000800:00000200:3.0:1788880561.790792:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:1.0:1788880561.790793:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000100:00000001:1.0:1788880561.790800:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000040:0.0:1788880561.790804:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880561.790804:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880561.790813:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306197967 00000100:00000001:0.0:1788880561.790817:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:3.0:1788880561.790819:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000088d6bc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000010:1.0:1788880561.790820:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000100:00000040:0.0:1788880561.790823:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880561.790825:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880561.790829:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880561.790830:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ff62fd67. 00000100:00000001:1.0:1788880561.790831:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306197967 00000100:00000200:0.0:1788880561.790837:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780668288, offset 0 00000100:00000040:3.0:1788880561.790837:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 309142 usecs (310 jiffies) 00000100:00000010:1.0:1788880561.790838:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000800:00000200:2.0:1788880561.790838:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880561.790843:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306193206 00000800:00000010:2.0:1788880561.790844:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000100:00000001:1.0:1788880561.790848:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880561.790849:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306197967 00000400:00000200:0.0:1788880561.790851:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880561.790851:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880561.790853:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306193206 00000100:00000001:1.0:1788880561.790855:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306197967 00000100:00000001:3.0:1788880561.790858:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306197967 00000400:00000200:0.0:1788880561.790861:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:1.0:1788880561.790862:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000400:00000200:2.0:1788880561.790863:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880561.790869:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880561.790869:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880561.790871:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306193206 00000100:00000001:3.0:1788880561.790874:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306197967 00000100:00000001:1.0:1788880561.790875:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880561.790876:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306193206 00000100:00000001:3.0:1788880561.790881:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306197967 00000400:00000200:0.0:1788880561.790882:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880561.790882:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880561.790884:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880561.790888:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880561.790889:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880561.790889:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880561.790894:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880561.790896:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880561.790896:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880561.790899:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [295:295:254:4294967295] 192.168.204.111@tcp LPNI seq info [295:295:6:4294967295] 00000100:00000001:3.0:1788880561.790901:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880561.790907:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880561.790907:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880561.790911:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880561.790911:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880561.790911:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880561.790918:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880561.790918:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880561.790921:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:0.0:1788880561.790922:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880561.790923:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880561.790923:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000066e8f89d 00000100:00000001:3.0:1788880561.790923:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880561.790927:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880561.790929:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000066e8f89d. 00000100:00000001:3.0:1788880561.790929:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880561.790930:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ac392bfe. 00000100:00000001:1.0:1788880561.790930:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880561.790939:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:0.0:1788880561.790940:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880561.790947:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880577 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880561.790948:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880561.790952:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ac392bfe type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880561.790959:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880561.790965:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 19962 usecs (20 jiffies) 00000100:00000001:2.0:1788880561.790966:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880561.790969:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306192916 00000100:00000040:2.0:1788880561.790972:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880577 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880561.790974:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306197967 00000100:00000001:0.0:1788880561.790978:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306192916 00000100:00000001:0.0:1788880561.790983:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306197967 00000100:00000001:0.0:1788880561.790987:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306192916 00000100:00000001:2.0:1788880561.790989:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880561.790991:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306197967 00000100:00000001:2.0:1788880561.790993:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880561.790994:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306192916 00000100:00000001:0.0:1788880561.790998:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306197967 00000100:00000001:0.0:1788880561.791000:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880561.791002:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880561.791005:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880561.791006:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788880561.791010:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880561.791011:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880561.791015:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:2.0:1788880561.791015:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000088d6bc1. 00000100:00000001:0.0:1788880561.791021:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880561.791022:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880561.791023:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880561.791025:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880561.791029:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880561.791033:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000035d99bc2 00000100:00000001:0.0:1788880561.791035:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880561.791037:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000035d99bc2. 00000100:00000001:2.0:1788880561.791041:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880561.791044:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880561.791061:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880561.791063:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880561.791069:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880561.791070:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880561.791124:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880561.791126:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ac392bfe. 00000400:00000200:2.0:1788880561.791130:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880561.791133:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880561.791136:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ff62fd67 00000400:00000010:2.0:1788880561.791137:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ff62fd67. 00000100:00000001:2.0:1788880561.791139:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880561.791142:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880561.791148:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880561.791150:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880561.791156:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880561.791157:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880561.792659:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880561.792669:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880561.792672:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880561.792675:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880561.792683:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880561.792694:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b08c80 00000400:00000200:0.0:1788880561.792701:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x9fd [1] + 224 00000800:00000001:0.0:1788880561.792706:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880561.792711:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880561.792713:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880561.792716:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880561.792721:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880561.792725:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880561.792734:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880561.792750:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880561.792757:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880561.792762:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880561.792764:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880561.792766:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880561.792770:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880561.792774:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08d00 00000400:00000200:0.0:1788880561.792778:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa05 [1] + 224 00000800:00000001:0.0:1788880561.792781:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880561.792783:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:0.0:1788880561.792785:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880561.792788:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880561.792789:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880561.792794:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:1.0:1788880561.792795:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880561.792797:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880561.792799:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b884f446. 00000400:00000200:1.0:1788880561.792802:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:1.0:1788880561.792803:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:1.0:1788880561.792810:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880561.792811:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880561.792814:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880561.792819:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880561.792829:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880561.792837:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880561.792839:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880561.792845:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880561.792848:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880561.792852:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:0.0:1788880561.792855:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880561.792855:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880561.792861:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880561.792870:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=89 v=5 (5 5 5 0) 00000100:00000001:0.0:1788880561.792875:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:1.0:1788880561.792875:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=89 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880561.792880:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880561.792881:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880561.792883:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:1.0:1788880561.792884:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880561.792887:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880561.792889:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880561.792890:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880561.792895:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788880561.792898:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880561.792899:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880561.792903:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880561.792906:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880561.792910:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:0.0:1788880561.792913:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:3.0:1788880561.792913:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880561.792916:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788880561.792917:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000bfe5eb98. 00000400:00000200:3.0:1788880561.792921:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000100:00000001:1.0:1788880561.792922:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000010:3.0:1788880561.792924:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000400:00000200:0.0:1788880561.792925:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08d80 00000100:00000001:1.0:1788880561.792927:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880561.792928:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880561.792933:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880561.792937:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880561.792939:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa0d [1] + 224 00000800:00000001:0.0:1788880561.792950:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880561.792956:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880561.792958:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880561.792961:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00100000:1.0:1788880561.792962:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780668032:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880561.792972:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880561.792973:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880561.792979:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880561.792980:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880561.792981:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880561.792983:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880561.792985:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780668032/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880561.792987:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000200:0.0:1788880561.792989:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880561.792991:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880561.792994:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880561.792997:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880561.793002:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880561.793008:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00001000:3.0:1788880561.793009:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=79 v=5 (5 5 5 0) 00000100:00000040:0.0:1788880561.793012:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880561.793019:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880561.793024:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000abce1aea. 00000100:00001000:3.0:1788880561.793024:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=79 v=5 (1 1 1 0) 02000000:00000001:1.0:1788880561.793031:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880561.793031:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880561.793032:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880561.793035:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00010000:00000001:3.0:1788880561.793036:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000040:1.0:1788880561.793038:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00010000:00000001:3.0:1788880561.793040:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880561.793042:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880561.793045:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:1.0:1788880561.793047:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003139f6ba. 00000100:00000001:3.0:1788880561.793047:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880561.793051:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:0.0:1788880561.793052:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:1.0:1788880561.793053:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880561.793055:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000001:1.0:1788880561.793057:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000800:00000001:0.0:1788880561.793058:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880561.793059:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880561.793060:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:1.0:1788880561.793066:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:3.0:1788880561.793067:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880561.793070:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880561.793075:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880561.793082:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880561.793082:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880561.793085:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880561.793086:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880561.793090:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880561.793090:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880561.793094:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880561.793095:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880561.793105:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780668160:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880561.793110:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880561.793112:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880561.793115:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000fd6b7ee1 x1875776780668160/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880561.793121:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880561.793123:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880561.793124:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880561.793125:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003720a685. 00000100:00000001:2.0:1788880561.793129:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880561.793130:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880561.793133:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000001:2.0:1788880561.793134:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000040:3.0:1788880561.793135:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000001:2.0:1788880561.793137:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000020:00000001:3.0:1788880561.793139:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000010:2.0:1788880561.793140:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000edb03a4b. 02000000:00000010:3.0:1788880561.793143:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5d1e8ed. 00000400:00000200:2.0:1788880561.793146:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000096cf0c4a 02000000:00000001:3.0:1788880561.793148:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000010:2.0:1788880561.793150:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000096cf0c4a. 02000000:00000001:3.0:1788880561.793151:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880561.793153:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000fd6b7ee1. 00000100:00000001:2.0:1788880561.793156:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880561.793158:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000200:2.0:1788880561.793162:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880561.793165:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880561.793169:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880561.793174:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880561.793176:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880561.793179:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880561.793181:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880561.793182:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880561.793190:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880561.793198:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880561.793200:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880561.793201:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880561.793203:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880561.793204:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880561.793206:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880561.793210:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=89 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880561.793212:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=46 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880561.793215:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880561.793216:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880561.793217:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880561.793221:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880561.793226:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880561.793229:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880561.793230:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880561.793231:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880561.793253:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880561.793257:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880561.793263:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880561.793265:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880561.793266:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880561.793267:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880561.793272:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780668288:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880561.793274:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880561.793275:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880561.793276:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d838cab4 x1875776780668288/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880577 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880561.793279:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880561.793280:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880561.793286:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d2602e97. 02000000:00000001:2.0:1788880561.793288:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880561.793289:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880561.793290:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880561.793291:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880561.793292:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 02000000:00000001:2.0:1788880561.793294:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880561.793295:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880561.793296:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d838cab4. 00000100:00000001:2.0:1788880561.793298:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880561.793299:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880561.793301:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880561.793304:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880561.793305:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880561.793306:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880561.793308:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880561.854355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880561.854363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880561.854369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880561.854370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880561.854428:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880561.854433:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880561.854435:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880562.878423:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880562.878431:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880562.878438:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880562.878439:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880562.878743:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880562.878750:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880562.878755:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880563.902308:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880563.902316:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880563.902322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880563.902323:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880563.902383:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880563.902387:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880563.902390:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880564.926317:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880564.926324:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880564.926327:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880564.926330:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880564.926334:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880564.926337:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880564.926339:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880564.926342:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880564.926348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880564.926362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880564.926398:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880564.926401:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880564.926402:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880565.950329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880565.950336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880565.950340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880565.950341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880565.950398:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880565.950402:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880565.950405:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880566.910354:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306197967, at 4306198016 00000100:00000040:3.0:1788880566.910354:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306197967, at 4306198016 00000100:00000040:0.0:1788880566.910364:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306197967, at 4306198016 00000100:00000040:2.0:1788880566.910374:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306197967, at 4306198016 00000100:00000010:3.0:1788880566.910374:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:1.0:1788880566.910375:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:0.0:1788880566.910377:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000010:2.0:1788880566.910385:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000040:2.0:1788880566.910420:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880566.910426:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880566.910435:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000020:00000040:2.0:1788880566.910440:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880566.910445:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880566.910448:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880566.910453:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880566.910461:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000001:2.0:1788880566.910469:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880566.910480:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880566.910527:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880566.910530:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880566.910537:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d838cab4] to pc [ptlrpcd_00_03+3] req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880566.910561:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880566.910564:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880566.910568:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880566.910571:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000020:00000040:2.0:1788880566.910574:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880566.910576:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880566.910577:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880566.910579:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880566.910582:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f9812869. 00000100:00000001:2.0:1788880566.910586:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880566.910590:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880566.910597:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880566.910598:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880566.910601:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fd6b7ee1] to pc [ptlrpcd_00_00+0] req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880566.910621:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880566.910623:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880566.910627:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880566.910630:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880566.910633:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880566.910636:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880566.910637:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880566.910639:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880566.910641:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880566.910643:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008eccda10. 00000100:00000001:3.0:1788880566.910645:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880566.910648:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880566.910654:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880566.910654:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880566.910655:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788880566.910660:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000040:2.0:1788880566.910665:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880566.910666:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880566.910667:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880566.910669:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880566.910673:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880566.910676:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:3.0:1788880566.910677:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880566.910678:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880566.910679:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880566.910682:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880566.910683:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880566.910684:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880566.910688:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780668544:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880566.910695:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880566.910695:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880566.910699:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880566.910702:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_00_01+1] req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880566.910703:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880566.910707:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880566.910708:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880566.910713:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880566.910719:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008f22742c. 00000100:00000001:2.0:1788880566.910725:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880566.910725:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880566.910729:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880566.910731:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880566.910734:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880566.910735:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bfe5eb98. 00000100:00000040:2.0:1788880566.910736:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:0.0:1788880566.910738:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880566.910741:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11535) 02000000:00000001:0.0:1788880566.910742:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788880566.910746:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 02000000:00000001:0.0:1788880566.910747:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880566.910751:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780668544, portal 10 00000100:00100000:0.0:1788880566.910752:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780668416:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880566.910754:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880566.910756:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000100:00000001:0.0:1788880566.910760:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880566.910764:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880566.910765:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880566.910766:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880566.910768:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880566.910769:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880566.910772:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880566.910775:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880566.910780:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880566.910780:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:0.0:1788880566.910781:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d5a90811. 02000000:00000001:0.0:1788880566.910786:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880566.910787:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880566.910792:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005b101ac6. 00000400:00000010:0.0:1788880566.910794:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000c66bed7. 00000100:00000200:3.0:1788880566.910796:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780668544, offset 0 00000100:00000001:2.0:1788880566.910798:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880566.910801:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880566.910802:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002aa6ba25. 02000000:00000001:2.0:1788880566.910805:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:3.0:1788880566.910805:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880566.910809:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880566.910813:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880566.910816:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780668672:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880566.910819:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880566.910823:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1788880566.910826:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:2.0:1788880566.910827:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880566.910831:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880566.910831:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:0.0:1788880566.910834:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780668416, portal 25 02000000:00000001:2.0:1788880566.910835:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788880566.910835:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880566.910840:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788880566.910840:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000010:2.0:1788880566.910841:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000065b4c69f. 00000100:00000001:0.0:1788880566.910844:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 02000000:00000001:2.0:1788880566.910845:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880566.910846:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [296:296:256:4294967295] 192.168.204.111@tcp LPNI seq info [296:296:8:4294967295] 00000100:00000001:1.0:1788880566.910848:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880566.910853:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880566.910853:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000100:00000040:0.0:1788880566.910855:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880582 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880566.910863:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:2.0:1788880566.910869:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000084794ee4. 00000100:00000001:0.0:1788880566.910870:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:2.0:1788880566.910874:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780668672, portal 10 00000100:00000040:0.0:1788880566.910877:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880566.910881:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880566.910882:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000001:2.0:1788880566.910884:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880566.910887:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000100:00000200:0.0:1788880566.910888:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780668416, offset 0 00000800:00000010:3.0:1788880566.910888:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000088d6bc1. 00000100:00000040:2.0:1788880566.910895:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880566.910900:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880566.910906:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880566.910912:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:3.0:1788880566.910913:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880566.910918:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:2.0:1788880566.910919:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880566.910919:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000088d6bc1 type 1, nob 320 niov 1 nkiov 1 00000400:00000010:2.0:1788880566.910925:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000032b7316b. 00000400:00000200:0.0:1788880566.910927:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:2.0:1788880566.910931:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780668672, offset 0 00000400:00000200:0.0:1788880566.910933:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880566.910937:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880566.910938:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880566.910941:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880566.910943:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 305498 usecs (306 jiffies) 00000400:00000200:0.0:1788880566.910945:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880566.910947:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306198322 00000100:00000010:3.0:1788880566.910951:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000016ab1a15. 00000100:00000001:3.0:1788880566.910955:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880566.910958:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [297:297:255:4294967295] 192.168.204.111@tcp LPNI seq info [297:297:7:4294967295] 00000400:00000200:0.0:1788880566.910970:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880566.910976:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880566.910978:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306198322 00000800:00000010:0.0:1788880566.910980:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000400:00000200:2.0:1788880566.910980:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:3.0:1788880566.910985:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000029868be. 00000400:00000200:2.0:1788880566.910988:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1788880566.910992:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880566.910995:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880566.910997:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000400:00000200:2.0:1788880566.911001:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788880566.911002:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880566.911004:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306198322 00000800:00000200:0.0:1788880566.911009:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880566.911009:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:3.0:1788880566.911011:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c451d2b9. 00000400:00000200:2.0:1788880566.911016:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [298:298:254:4294967295] 192.168.204.111@tcp LPNI seq info [298:298:6:4294967295] 00000100:00000001:0.0:1788880566.911017:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880566.911017:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880566.911022:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306198322 00000100:00000001:0.0:1788880566.911024:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306203087 00000100:00000010:3.0:1788880566.911028:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000dc3cc638. 00000100:00000001:0.0:1788880566.911033:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306198322 00000100:00000001:3.0:1788880566.911034:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000400:00000200:2.0:1788880566.911035:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880566.911040:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880566.911042:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306203087 00000800:00000200:2.0:1788880566.911046:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880566.911046:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880566.911048:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306198322 00000100:00000001:3.0:1788880566.911053:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880566.911054:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306203087 00000800:00000010:2.0:1788880566.911054:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5d1e8ed. 00000100:00000001:3.0:1788880566.911057:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880566.911062:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306198322 00000800:00000200:2.0:1788880566.911062:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880566.911064:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:1.0:1788880566.911066:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880566.911068:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880566.911070:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306203087 00000800:00000010:1.0:1788880566.911072:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000088d6bc1. 00000100:00000001:3.0:1788880566.911074:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:2.0:1788880566.911075:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880566.911077:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306198322 00000100:00000001:3.0:1788880566.911078:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880566.911080:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:2.0:1788880566.911081:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5d1e8ed type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880566.911082:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880566.911084:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880566.911089:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880566.911090:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880566.911090:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880566.911091:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880566.911097:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880566.911097:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 19756 usecs (20 jiffies) 00000400:00000200:1.0:1788880566.911100:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005b101ac6 00000100:00000001:2.0:1788880566.911103:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306198036 00000400:00000010:1.0:1788880566.911104:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005b101ac6. 00000100:00000001:0.0:1788880566.911105:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880566.911111:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306203087 00000100:00000001:1.0:1788880566.911113:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880566.911116:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306198036 00000100:00000200:1.0:1788880566.911120:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880566.911121:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306203087 00000100:00000001:2.0:1788880566.911127:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306198036 00000100:00000001:2.0:1788880566.911132:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306203087 00000100:00000001:1.0:1788880566.911137:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880566.911142:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306198036 00000100:00000001:2.0:1788880566.911144:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306203087 00000100:00000001:2.0:1788880566.911146:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880566.911149:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880566.911150:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880566.911155:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880566.911161:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880566.911167:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880566.911167:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880566.911171:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880566.911171:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880566.911176:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880566.911178:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880566.911180:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880566.911183:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880566.911191:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880566.911194:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000400:00000200:1.0:1788880566.911198:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880566.911202:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880566.911204:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:1.0:1788880566.911206:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:1.0:1788880566.911208:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880566.911211:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880582 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880566.911218:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880566.911221:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880582 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880566.911254:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880566.911256:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880566.911305:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880566.911308:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5d1e8ed. 00000400:00000200:1.0:1788880566.911311:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880566.911315:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880566.911317:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000032b7316b 00000400:00000010:1.0:1788880566.911319:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000032b7316b. 00000100:00000001:1.0:1788880566.911321:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880566.911324:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880566.911331:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880566.911333:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880566.911340:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880566.911341:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880566.913181:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880566.913195:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880566.913199:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880566.913201:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880566.913209:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880566.913221:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08e80 00000400:00000200:2.0:1788880566.913265:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa15 [1] + 224 00000800:00000001:2.0:1788880566.913271:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880566.913276:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880566.913278:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880566.913282:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880566.913287:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880566.913291:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880566.913300:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880566.913313:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880566.913319:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880566.913322:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880566.913323:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880566.913335:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880566.913340:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880566.913345:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880566.913346:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b08e00 00000100:00000001:3.0:1788880566.913350:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880566.913355:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xa1d [1] + 224 00000800:00000001:2.0:1788880566.913359:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1788880566.913360:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880566.913362:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880566.913363:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788880566.913366:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:3.0:1788880566.913368:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000bfe5eb98. 00000400:00000200:2.0:1788880566.913373:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880566.913374:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000400:00000010:3.0:1788880566.913377:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:3.0:1788880566.913380:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880566.913384:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880566.913385:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880566.913392:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880566.913402:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880566.913409:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880566.913416:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880566.913419:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880566.913423:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880566.913425:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880566.913427:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880566.913430:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880566.913436:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880566.913436:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880566.913441:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880566.913441:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880566.913444:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880566.913446:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788880566.913450:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=84 v=5 (5 5 5 0) 00000400:00000200:2.0:1788880566.913451:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:3.0:1788880566.913458:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=84 v=5 (1 1 1 0) 00000400:00000200:2.0:1788880566.913462:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b08f00 00000100:00000001:3.0:1788880566.913464:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880566.913466:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880566.913468:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880566.913469:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:3.0:1788880566.913471:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880566.913474:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880566.913478:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000200:2.0:1788880566.913482:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa21 [1] + 224 00000100:00000040:3.0:1788880566.913482:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880566.913486:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880566.913489:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880566.913491:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880566.913491:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:3.0:1788880566.913495:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880566.913498:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880566.913503:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:0.0:1788880566.913504:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880566.913506:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880566.913509:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000c66bed7. 00000100:00000001:3.0:1788880566.913510:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:2.0:1788880566.913512:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880566.913527:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002aa6ba25 00000100:00000001:3.0:1788880566.913528:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880566.913532:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002aa6ba25. 00000100:00000040:3.0:1788880566.913535:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880566.913538:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880566.913539:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880566.913546:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880566.913547:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880566.913554:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780668544:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880566.913566:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880566.913570:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880566.913575:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000fd6b7ee1 x1875776780668544/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880566.913580:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880566.913598:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880566.913603:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880566.913606:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008f22742c. 00000100:00000200:0.0:1788880566.913610:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880566.913611:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880566.913614:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880566.913617:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880566.913622:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880566.913625:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f9812869. 00000100:00000001:0.0:1788880566.913630:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:3.0:1788880566.913632:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880566.913634:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880566.913639:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880566.913641:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:0.0:1788880566.913643:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000010:3.0:1788880566.913644:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000fd6b7ee1. 00000100:00000001:0.0:1788880566.913647:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880566.913649:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880566.913650:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880566.913652:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880566.913654:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880566.913656:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880566.913657:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880566.913662:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000001:2.0:1788880566.913664:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880566.913665:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=94 v=5 (5 5 5 0) 00000100:00000001:3.0:1788880566.913665:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880566.913670:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880566.913670:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:0.0:1788880566.913671:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=94 v=5 (1 1 1 0) 00000100:00000001:3.0:1788880566.913674:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880566.913678:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880566.913684:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880566.913686:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880566.913688:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880566.913690:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880566.913696:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880566.913704:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880566.913707:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880566.913708:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880566.913711:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880566.913712:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880566.913716:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880566.913721:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880566.913724:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00100000:0.0:1788880566.913727:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780668416:192.168.204.111@tcp:400: 00000400:00000010:1.0:1788880566.913728:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00000400:00000200:1.0:1788880566.913735:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000084794ee4 00000100:00000001:0.0:1788880566.913736:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880566.913738:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000084794ee4. 00000100:00000001:0.0:1788880566.913739:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880566.913743:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:0.0:1788880566.913744:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d838cab4 x1875776780668416/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880566.913750:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880566.913759:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880566.913762:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880566.913766:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d5a90811. 00000100:00000200:1.0:1788880566.913766:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880566.913774:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880566.913777:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880566.913778:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880566.913779:0:201286:0:(client.c:1580:after_reply()) Process entered 00000020:00000001:0.0:1788880566.913782:0:201287:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000001:1.0:1788880566.913783:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000040:0.0:1788880566.913784:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880566.913786:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000001:0.0:1788880566.913788:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880566.913790:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 02000000:00000001:0.0:1788880566.913792:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880566.913795:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880566.913796:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880566.913797:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d838cab4. 02000000:00000001:1.0:1788880566.913800:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880566.913802:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880566.913805:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880566.913807:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880566.913807:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=94 v=5 (5 5 5 0) 00000100:00000001:0.0:1788880566.913813:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00001000:1.0:1788880566.913813:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=51 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880566.913816:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880566.913818:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880566.913820:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880566.913821:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880566.913824:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880566.913829:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880566.913834:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880566.913841:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880566.913844:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880566.913845:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880566.913846:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880566.913847:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880566.913850:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880566.913857:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880566.913858:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880566.913859:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880566.913862:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880566.913868:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780668672:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880566.913872:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880566.913874:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880566.913875:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780668672/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880582 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880566.913881:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880566.913882:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880566.913884:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000065b4c69f. 02000000:00000001:1.0:1788880566.913887:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880566.913888:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880566.913895:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880566.913897:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880566.913899:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008eccda10. 02000000:00000001:1.0:1788880566.913901:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880566.913902:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880566.913903:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:1.0:1788880566.913906:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880566.913907:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880566.913909:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880566.913913:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880566.913914:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880566.913916:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880566.913917:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880566.974526:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880566.974534:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880566.974541:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880566.974544:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880566.974546:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880566.974549:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880566.974553:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880566.974555:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880566.974557:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880566.974560:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880566.974679:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880566.974683:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880566.974685:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880567.998383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880567.998391:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880567.998397:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880567.998398:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880567.998550:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880567.998554:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880567.998556:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880569.022342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880569.022348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880569.022353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880569.022354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880569.022410:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880569.022415:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880569.022417:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880570.046392:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880570.046400:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880570.046408:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880570.046409:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880570.046528:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880570.046531:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880570.046534:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880571.070358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880571.070366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880571.070373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880571.070395:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880571.070475:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880571.070479:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880571.070481:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880572.030404:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306203087, at 4306203136 00000100:00000040:1.0:1788880572.030404:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306203087, at 4306203136 00000100:00000040:2.0:1788880572.030404:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306203087, at 4306203136 00000100:00000040:3.0:1788880572.030426:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:2.0:1788880572.030434:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c451d2b9. 00000100:00000010:1.0:1788880572.030436:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000029868be. 00000100:00000001:3.0:1788880572.030436:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880572.030438:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000016ab1a15. 00000100:00000010:3.0:1788880572.030458:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788880572.030467:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880572.030472:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880572.030477:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880572.030482:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880572.030492:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b899fe8d. 00000100:00000001:3.0:1788880572.030499:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.030510:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880572.030522:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880572.030524:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.030530:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_02+2] req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880572.030555:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.030556:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880572.030559:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880572.030561:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000020:00000040:3.0:1788880572.030575:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880572.030576:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880572.030577:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880572.030579:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880572.030582:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f92e72a2. 00000100:00000001:3.0:1788880572.030584:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.030590:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880572.030595:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880572.030596:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.030599:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_03+3] req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880572.030604:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.030606:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880572.030608:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880572.030609:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880572.030609:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880572.030617:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880572.030617:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:3.0:1788880572.030622:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000100:00000001:1.0:1788880572.030624:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880572.030625:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:3.0:1788880572.030629:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880572.030632:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880572.030633:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880572.030634:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880572.030638:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000446e4ede. 00000100:00000040:0.0:1788880572.030641:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880572.030641:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880572.030642:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.030652:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880572.030711:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880572.030712:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880572.030713:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880572.030721:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880572.030722:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880572.030722:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880572.030737:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880572.030737:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880572.030747:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880572.030747:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880572.030747:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_00+0] req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880572.030763:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880572.030764:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880572.030784:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780668800:192.168.204.111@tcp:400: 00000100:00100000:1.0:1788880572.030784:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780668928:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880572.030802:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880572.030804:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880572.030805:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880572.030807:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880572.030814:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880572.030822:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.030822:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880572.030823:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880572.030826:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880572.030827:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880572.030829:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880572.030830:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.030830:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11540) 02000000:00000001:0.0:1788880572.030834:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880572.030836:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880572.030840:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880572.030842:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002f5492f5. 02000000:00000001:0.0:1788880572.030848:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.030852:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306203087, at 4306203136 00000400:00000010:0.0:1788880572.030854:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000c66bed7. 02000000:00000010:1.0:1788880572.030854:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000cf77ae66. 00000100:00000001:2.0:1788880572.030854:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880572.030860:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880572.030863:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000dc3cc638. 02000000:00000001:1.0:1788880572.030865:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880572.030867:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:0.0:1788880572.030870:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002aa6ba25. 02000000:00000001:2.0:1788880572.030873:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880572.030876:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880572.030878:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780668800, portal 25 00000100:00100000:2.0:1788880572.030879:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780669056:192.168.204.111@tcp:400: 00000400:00000010:1.0:1788880572.030880:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b884f446. 00000100:00000001:0.0:1788880572.030883:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880572.030891:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000001:2.0:1788880572.030892:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:1.0:1788880572.030895:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 02000000:00000001:2.0:1788880572.030896:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880572.030900:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880588 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880572.030900:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880572.030902:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880572.030905:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880572.030907:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780668928, portal 10 02000000:00000001:2.0:1788880572.030908:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880572.030916:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880572.030919:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:2.0:1788880572.030919:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000790fa4ee. 00000100:00000001:1.0:1788880572.030920:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000040:0.0:1788880572.030929:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880572.030930:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880572.030934:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e73139ca. 00000400:00000010:2.0:1788880572.030937:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ded9c590. 00000100:00000040:1.0:1788880572.030938:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880572.030941:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780668800, offset 0 00000400:00000010:2.0:1788880572.030946:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c0c07443. 00000400:00000200:0.0:1788880572.030954:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788880572.030965:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780669056, portal 10 00000100:00000001:1.0:1788880572.030967:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880572.030971:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880572.030974:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000400:00000200:0.0:1788880572.030977:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:1.0:1788880572.030981:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880572.030986:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:1.0:1788880572.030987:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 00000100:00000040:2.0:1788880572.030989:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880572.030996:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:1.0:1788880572.030998:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780668928, offset 0 00000400:00000200:0.0:1788880572.031007:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880572.031014:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880572.031016:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880572.031029:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [299:299:256:4294967295] 192.168.204.111@tcp LPNI seq info [299:299:8:4294967295] 00000100:00000001:2.0:1788880572.031031:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880572.031036:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880572.031040:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c013629f. 00000100:00000200:2.0:1788880572.031045:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780669056, offset 0 00000400:00000200:0.0:1788880572.031049:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:2.0:1788880572.031054:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880572.031060:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880572.031065:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788880572.031067:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c93f60a2. 00000800:00000200:0.0:1788880572.031076:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880572.031091:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1788880572.031095:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880572.031101:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788880572.031104:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c93f60a2 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880572.031108:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880572.031116:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880572.031120:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [300:300:255:4294967295] 192.168.204.111@tcp LPNI seq info [300:300:7:4294967295] 00000100:00000001:0.0:1788880572.031124:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880572.031129:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306208207 00000100:00000010:0.0:1788880572.031137:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000a4f85af. 00000400:00000200:1.0:1788880572.031139:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880572.031150:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:1.0:1788880572.031160:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880572.031160:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880572.031164:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306208207 00000400:00000200:2.0:1788880572.031167:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880572.031171:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:0.0:1788880572.031173:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c0bfe00. 00000800:00000010:1.0:1788880572.031174:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000400:00000200:2.0:1788880572.031175:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880572.031182:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:2.0:1788880572.031182:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880572.031191:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880572.031192:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306208207 00000400:00000200:2.0:1788880572.031193:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [301:301:254:4294967295] 192.168.204.111@tcp LPNI seq info [301:301:6:4294967295] 00000100:00000010:0.0:1788880572.031205:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005b8bebe2. 00000100:00000001:0.0:1788880572.031210:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880572.031216:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306208207 00000800:00000200:1.0:1788880572.031218:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:0.0:1788880572.031223:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009ac8d70b. 00000400:00000200:2.0:1788880572.031226:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880572.031229:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880572.031230:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:3.0:1788880572.031267:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880572.031272:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880572.031274:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:3.0:1788880572.031275:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c93f60a2. 00000800:00000200:2.0:1788880572.031278:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880572.031284:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880572.031284:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 301946 usecs (302 jiffies) 00000800:00000010:2.0:1788880572.031286:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003139f6ba. 00000400:00000200:3.0:1788880572.031288:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880572.031294:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306203439 00000100:00000001:0.0:1788880572.031296:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880572.031304:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306208207 00000800:00000200:2.0:1788880572.031304:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880572.031304:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880572.031305:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880572.031317:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306203439 00000100:00000001:0.0:1788880572.031321:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:3.0:1788880572.031321:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e73139ca 00000800:00000200:2.0:1788880572.031325:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880572.031328:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306208207 00000100:00000001:0.0:1788880572.031329:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:2.0:1788880572.031330:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003139f6ba type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880572.031340:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880572.031340:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306203439 00000100:00000001:0.0:1788880572.031345:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880572.031347:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306208207 00000100:00000001:0.0:1788880572.031349:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880572.031349:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880572.031357:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 19562 usecs (20 jiffies) 00000100:00000001:0.0:1788880572.031359:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880572.031359:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306203439 00000100:00000001:2.0:1788880572.031363:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306203157 00000100:00000001:1.0:1788880572.031372:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306208207 00000100:00000001:2.0:1788880572.031372:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306208207 00000400:00000010:3.0:1788880572.031385:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e73139ca. 00000100:00000001:1.0:1788880572.031386:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880572.031387:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306203157 00000100:00000001:3.0:1788880572.031391:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880572.031396:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306208207 00000100:00000001:1.0:1788880572.031397:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880572.031399:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880588 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880572.031400:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306203157 00000100:00000001:2.0:1788880572.031408:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306208207 00000100:00000001:1.0:1788880572.031409:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880572.031413:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306203157 00000100:00000001:2.0:1788880572.031427:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306208207 00000100:00000001:1.0:1788880572.031433:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880572.031435:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880572.031437:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880572.031443:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.031443:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880588 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880572.031447:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880572.031449:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880572.031453:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880572.031454:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880572.031466:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880572.031466:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880572.031467:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880572.031468:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880572.031470:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880572.031473:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880572.031474:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880572.031480:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880572.031481:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880572.031483:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880572.031486:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880572.031486:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880572.031488:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:1.0:1788880572.031499:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880572.031499:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880572.031513:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880572.031516:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:3.0:1788880572.031517:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:3.0:1788880572.031519:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880572.031521:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880572.031526:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880572.031530:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880572.031543:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880572.031544:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880572.031569:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880572.031571:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003139f6ba. 00000400:00000200:3.0:1788880572.031574:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880572.031578:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880572.031580:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c013629f 00000400:00000010:3.0:1788880572.031582:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c013629f. 00000100:00000001:3.0:1788880572.031584:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880572.031587:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880572.031594:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880572.031596:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880572.031602:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880572.031603:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880572.032992:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880572.033007:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880572.033010:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880572.033012:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880572.033019:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880572.033030:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09000 00000400:00000200:1.0:1788880572.033037:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa31 [1] + 224 00000800:00000001:1.0:1788880572.033042:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880572.033045:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880572.033047:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880572.033050:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880572.033055:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880572.033059:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880572.033066:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880572.033083:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880572.033090:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880572.033092:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880572.033094:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880572.033096:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880572.033100:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880572.033106:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b08f80 00000400:00000200:1.0:1788880572.033110:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xa2d [1] + 224 00000800:00000001:1.0:1788880572.033113:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880572.033130:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880572.033135:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:1.0:1788880572.033142:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880572.033142:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:1.0:1788880572.033146:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:2.0:1788880572.033147:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:1.0:1788880572.033151:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:2.0:1788880572.033153:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b884f446. 00000400:00000200:2.0:1788880572.033160:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:2.0:1788880572.033162:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:1.0:1788880572.033165:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880572.033166:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880572.033175:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880572.033182:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880572.033199:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880572.033261:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880572.033266:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880572.033270:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880572.033272:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880572.033273:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880572.033281:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000200:2.0:1788880572.033288:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880572.033289:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09080 00000400:00000200:1.0:1788880572.033303:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa39 [1] + 224 00000100:00000001:0.0:1788880572.033308:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880572.033314:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880572.033317:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880572.033322:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880572.033323:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880572.033325:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880572.033328:0:201288:0:(client.c:1580:after_reply()) Process entered 00000400:00000001:0.0:1788880572.033329:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:2.0:1788880572.033332:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:1.0:1788880572.033334:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880572.033336:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000c66bed7. 00000100:00000001:2.0:1788880572.033340:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:1.0:1788880572.033341:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880572.033345:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002aa6ba25 00000100:00000001:2.0:1788880572.033350:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880572.033353:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002aa6ba25. 00000400:00000200:1.0:1788880572.033353:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:2.0:1788880572.033359:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880572.033368:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880572.033368:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880572.033372:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:2.0:1788880572.033377:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=90 v=5 (5 5 5 0) 00000100:00000200:0.0:1788880572.033379:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:2.0:1788880572.033398:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=90 v=5 (1 1 1 0) 00000100:00000040:1.0:1788880572.033424:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880572.033425:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880572.033430:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880572.033437:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880572.033443:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880572.033450:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880572.033452:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788880572.033466:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880572.033468:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:2.0:1788880572.033470:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:1.0:1788880572.033474:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880572.033476:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880572.033476:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000800:00000001:1.0:1788880572.033479:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880572.033480:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880572.033481:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880572.033487:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880572.033490:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880572.033493:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880572.033493:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880572.033498:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880572.033501:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:0.0:1788880572.033505:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=100 v=5 (5 5 5 0) 00000100:00000001:3.0:1788880572.033505:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880572.033508:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000001:3.0:1788880572.033510:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00001000:0.0:1788880572.033511:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=100 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880572.033513:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000001:3.0:1788880572.033513:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880572.033517:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880572.033518:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000010:3.0:1788880572.033518:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ded9c590. 00000100:00000001:0.0:1788880572.033522:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880572.033524:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c0c07443 00010000:00000001:0.0:1788880572.033525:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:2.0:1788880572.033527:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880572.033528:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c0c07443. 00010000:00000001:0.0:1788880572.033529:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880572.033533:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880572.033536:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880572.033539:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880572.033545:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880572.033550:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780668928:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880572.033560:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880572.033564:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880572.033569:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780668928/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880572.033574:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:3.0:1788880572.033578:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880572.033580:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880572.033583:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880572.033586:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880572.033590:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:0.0:1788880572.033593:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880572.033594:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000cf77ae66. 00000100:00000001:3.0:1788880572.033597:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:2.0:1788880572.033600:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880572.033601:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788880572.033604:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880572.033607:0:201285:0:(client.c:1580:after_reply()) Process entered 00000020:00000040:2.0:1788880572.033608:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:3.0:1788880572.033610:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00100000:0.0:1788880572.033613:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780668800:192.168.204.111@tcp:400: 00000020:00000001:2.0:1788880572.033613:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880572.033620:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000010:2.0:1788880572.033621:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f92e72a2. 00000100:00000001:0.0:1788880572.033622:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880572.033623:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880572.033627:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:3.0:1788880572.033629:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880572.033631:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:0.0:1788880572.033633:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780668800/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880572.033634:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00001000:3.0:1788880572.033636:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=100 v=5 (5 5 5 0) 00000100:00000010:2.0:1788880572.033637:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:2.0:1788880572.033642:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00001000:3.0:1788880572.033645:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=57 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880572.033646:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880572.033649:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880572.033651:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880572.033653:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880572.033654:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000010:0.0:1788880572.033656:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002f5492f5. 00000100:00000001:3.0:1788880572.033657:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880572.033658:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880572.033662:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880572.033662:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:0.0:1788880572.033663:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880572.033667:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880572.033668:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00080000:3.0:1788880572.033669:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1788880572.033670:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880572.033671:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880572.033675:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880572.033679:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b899fe8d. 00010000:00000001:3.0:1788880572.033684:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880572.033686:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880572.033686:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880572.033688:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880572.033689:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880572.033693:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880572.033698:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880572.033698:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880572.033700:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880572.033700:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880572.033702:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880572.033707:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00100000:3.0:1788880572.033709:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780669056:192.168.204.111@tcp:400: 00000100:00000010:0.0:1788880572.033711:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:3.0:1788880572.033713:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880572.033715:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880572.033716:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:3.0:1788880572.033717:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780669056/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880588 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880572.033720:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880572.033724:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880572.033727:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880572.033729:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880572.033730:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000010:3.0:1788880572.033731:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000790fa4ee. 00000100:00000001:0.0:1788880572.033733:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880572.033735:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880572.033737:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880572.033738:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880572.033742:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880572.033743:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880572.033745:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880572.033746:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000446e4ede. 02000000:00000001:3.0:1788880572.033747:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880572.033749:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880572.033750:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:3.0:1788880572.033752:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880572.033754:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880572.033755:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880572.033759:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880572.033761:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880572.033762:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880572.033764:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880572.094374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880572.094395:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880572.094405:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880572.094406:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880572.094487:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880572.094493:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880572.094497:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880573.118359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880573.118367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880573.118374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880573.118375:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880573.118490:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880573.118494:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880573.118497:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880574.142366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880574.142374:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880574.142390:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880574.142391:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880574.142452:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880574.142457:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880574.142459:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880575.166367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880575.166375:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880575.166378:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880575.166381:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880575.166386:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880575.166399:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880575.166401:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880575.166404:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880575.166409:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880575.166411:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880575.166473:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880575.166476:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880575.166479:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880576.190349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880576.190356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880576.190362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880576.190364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880576.190423:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880576.190427:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880576.190430:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880577.150392:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306208207, at 4306208256 00000100:00000040:3.0:1788880577.150392:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306208207, at 4306208256 00000100:00000010:3.0:1788880577.150414:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009ac8d70b. 00000100:00000040:1.0:1788880577.150480:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880577.150499:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880577.150511:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c09231ec. 00000020:00000040:1.0:1788880577.150520:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880577.150527:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880577.150531:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880577.150539:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880577.150549:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:1.0:1788880577.150560:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880577.150576:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:0.0:1788880577.150584:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306208207, at 4306208256 00000100:00000001:1.0:1788880577.150591:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880577.150596:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880577.150597:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000a4f85af. 00000100:00000040:1.0:1788880577.150613:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c09231ec] to pc [ptlrpcd_00_01+1] req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788880577.150651:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005b8bebe2. 00000100:00000001:1.0:1788880577.150661:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880577.150687:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880577.150690:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880577.150694:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000003b122924. 00000020:00000040:1.0:1788880577.150697:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880577.150701:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880577.150703:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880577.150706:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880577.150711:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:1.0:1788880577.150715:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880577.150718:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880577.150722:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788880577.150726:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:1.0:1788880577.150740:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880577.150741:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880577.150744:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880577.150751:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003b122924] to pc [ptlrpcd_00_02+2] req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880577.150754:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880577.150761:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880577.150773:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880577.150776:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880577.150779:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880577.150790:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880577.150794:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880577.150796:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880577.150813:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880577.150816:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880577.150821:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880577.150825:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880577.150830:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880577.150835:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780669184:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880577.150840:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880577.150843:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880577.150848:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880577.150853:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880577.150854:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880577.150860:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880577.150861:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880577.150861:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002dcbd5d0. 00000100:00100000:3.0:1788880577.150865:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000020:00000040:1.0:1788880577.150868:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880577.150869:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880577.150875:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880577.150878:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880577.150879:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880577.150880:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880577.150888:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880577.150890:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880577.150890:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880577.150893:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880577.150896:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880577.150906:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007f19b4dd. 02000000:00000010:1.0:1788880577.150906:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:3.0:1788880577.150906:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880577.150913:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880577.150916:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880577.150917:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880577.150924:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880577.150934:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:0.0:1788880577.150941:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000c66bed7. 00000100:00000001:1.0:1788880577.150956:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000400:00000010:0.0:1788880577.150957:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002aa6ba25. 00000100:00000001:1.0:1788880577.150963:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880577.150971:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780669184, portal 25 00000100:00000040:1.0:1788880577.150974:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002dcbd5d0] to pc [ptlrpcd_00_03+3] req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880577.150982:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880577.150983:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880577.150988:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938056832 : -126823771494784 : ffff8ca78a26a680) 00000100:00000001:3.0:1788880577.150990:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880577.151003:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:0.0:1788880577.151004:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880593 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880577.151008:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880577.151017:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880577.151032:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780669312:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880577.151052:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880577.151061:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880577.151067:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880577.151074:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880577.151075:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880577.151079:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000057e92cbc. 02000000:00000001:3.0:1788880577.151082:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880577.151087:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880577.151089:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bfe5eb98. 00000400:00000010:0.0:1788880577.151093:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000fbbca8a. 00000400:00000010:3.0:1788880577.151095:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 00000100:00000200:3.0:1788880577.151106:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780669312, portal 10 00000100:00000200:0.0:1788880577.151107:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780669184, offset 0 00000100:00000001:3.0:1788880577.151111:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880577.151115:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938062208 : -126823771489408 : ffff8ca78a26bb80) 00000400:00000200:0.0:1788880577.151121:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880577.151144:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880577.151150:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880577.151153:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880577.151158:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:1.0:1788880577.151160:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880577.151167:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:0.0:1788880577.151169:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880577.151181:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788880577.151182:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11545) 00000400:00000200:0.0:1788880577.151192:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [302:302:256:4294967295] 192.168.204.111@tcp LPNI seq info [302:302:8:4294967295] 00000100:00000001:2.0:1788880577.151200:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:2.0:1788880577.151206:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000040:1.0:1788880577.151209:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306208207, at 4306208256 00000100:00000001:2.0:1788880577.151212:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:1.0:1788880577.151220:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c0bfe00. 00000100:00000001:2.0:1788880577.151220:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880577.151222:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880577.151224:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880577.151264:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880577.151265:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880577.151267:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880577.151273:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880577.151284:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880577.151286:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880577.151290:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880577.151296:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880577.151304:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000da26fd2f. 00000100:00000001:2.0:1788880577.151308:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880577.151311:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880577.151315:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:3.0:1788880577.151317:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ec90f18d. 02000000:00000001:2.0:1788880577.151330:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:3.0:1788880577.151335:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780669312, offset 0 00000800:00000200:0.0:1788880577.151336:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:2.0:1788880577.151339:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880577.151348:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780669440:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880577.151350:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880577.151363:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880577.151363:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1788880577.151371:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:2.0:1788880577.151374:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880577.151380:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880577.151381:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000da26fd2f type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880577.151383:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788880577.151388:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:2.0:1788880577.151390:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004ffd965d. 02000000:00000001:2.0:1788880577.151395:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880577.151398:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:2.0:1788880577.151403:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000702dddb0. 00000400:00000200:3.0:1788880577.151404:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880577.151409:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:2.0:1788880577.151410:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000044b1d7fc. 00000400:00000200:3.0:1788880577.151414:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [303:303:255:4294967295] 192.168.204.111@tcp LPNI seq info [303:303:7:4294967295] 00000100:00000001:0.0:1788880577.151418:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880577.151418:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780669440, portal 10 00000100:00000001:2.0:1788880577.151427:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880577.151432:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938047872 : -126823771503744 : ffff8ca78a268380) 00000100:00000001:0.0:1788880577.151433:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306213328 00000400:00000200:3.0:1788880577.151436:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:2.0:1788880577.151445:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880577.151446:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000a4f85af. 00000100:00000001:2.0:1788880577.151463:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880577.151464:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:3.0:1788880577.151464:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880577.151472:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 00000100:00000040:2.0:1788880577.151482:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880577.151484:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306213328 00000400:00000010:2.0:1788880577.151488:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009543d7d4. 00000100:00000010:0.0:1788880577.151492:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000029f028ab. 00000800:00000200:3.0:1788880577.151493:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880577.151495:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780669440, offset 0 00000100:00000001:0.0:1788880577.151505:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:1.0:1788880577.151508:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880577.151515:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306213328 00000400:00000200:2.0:1788880577.151517:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880577.151518:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000010:1.0:1788880577.151519:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000da26fd2f. 00000100:00000010:0.0:1788880577.151533:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014321469. 00000800:00000200:3.0:1788880577.151534:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880577.151535:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880577.151540:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880577.151545:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880577.151550:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880577.151558:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880577.151562:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306213328 00000100:00000040:3.0:1788880577.151562:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 298475 usecs (299 jiffies) 00000400:00000200:1.0:1788880577.151563:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880577.151571:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:0.0:1788880577.151572:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000085563e6f. 00000400:00000200:2.0:1788880577.151577:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880577.151581:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:2.0:1788880577.151585:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880577.151588:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880577.151594:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [304:304:254:4294967295] 192.168.204.111@tcp LPNI seq info [304:304:6:4294967295] 00000100:00000001:0.0:1788880577.151597:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880577.151606:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880577.151616:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306208556 00000100:00000001:0.0:1788880577.151620:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880577.151624:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880577.151625:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306213328 00000100:00000001:3.0:1788880577.151631:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306208556 00000100:00000001:3.0:1788880577.151634:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306213328 00000100:00000001:3.0:1788880577.151639:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306208556 00000100:00000001:3.0:1788880577.151642:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306213328 00000100:00000001:3.0:1788880577.151645:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306208556 00000400:00000200:1.0:1788880577.151648:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000fbbca8a 00000800:00000200:2.0:1788880577.151649:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880577.151649:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306213328 00000400:00000010:1.0:1788880577.151653:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000fbbca8a. 00000100:00000001:3.0:1788880577.151662:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880577.151670:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880577.151670:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880577.151671:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000100:00000001:3.0:1788880577.151677:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880577.151679:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880593 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880577.151688:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880577.151689:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880577.151701:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880577.151704:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880577.151711:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880577.151714:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880577.151724:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880577.151725:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880577.151729:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880577.151737:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880577.151739:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880593 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880577.151747:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 19368 usecs (20 jiffies) 00000100:00000001:1.0:1788880577.151750:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880577.151755:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880577.151755:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306208277 00000100:00000001:2.0:1788880577.151762:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306213328 00000800:00000200:1.0:1788880577.151777:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880577.151777:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306208277 00000100:00000001:2.0:1788880577.151782:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306213328 00000800:00000010:1.0:1788880577.151783:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000100:00000001:2.0:1788880577.151785:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306208277 00000100:00000001:2.0:1788880577.151793:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306213328 00000400:00000200:1.0:1788880577.151796:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880577.151797:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306208277 00000100:00000001:2.0:1788880577.151802:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306213328 00000100:00000001:2.0:1788880577.151808:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880577.151813:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880577.151814:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880577.151820:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ec90f18d 00000400:00000010:1.0:1788880577.151828:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ec90f18d. 00000100:00000001:2.0:1788880577.151830:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880577.151834:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880577.151837:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880577.151841:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880577.151850:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880577.151858:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880577.151863:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880577.151865:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880577.151892:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880577.151894:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000400:00000200:1.0:1788880577.151896:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880577.151898:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880577.151900:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009543d7d4 00000400:00000010:1.0:1788880577.151901:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009543d7d4. 00000100:00000001:1.0:1788880577.151903:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880577.151905:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880577.151910:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880577.151911:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880577.151915:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880577.151916:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880577.153607:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880577.153622:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880577.153626:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880577.153628:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880577.153639:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880577.153652:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b09100 00000400:00000200:3.0:1788880577.153806:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xa45 [1] + 224 00000800:00000001:3.0:1788880577.153824:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880577.153832:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880577.153835:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880577.153839:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880577.153845:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880577.153851:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880577.153861:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880577.153929:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880577.153934:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880577.153950:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000001:0.0:1788880577.153968:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788880577.153971:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880577.153972:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880577.153977:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000c66bed7. 00000400:00000200:0.0:1788880577.153982:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002aa6ba25 00000800:00000001:3.0:1788880577.153982:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880577.153985:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002aa6ba25. 00000800:00000001:3.0:1788880577.153987:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880577.153990:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:3.0:1788880577.153991:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880577.153997:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880577.154000:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880577.154010:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09200 00000100:00000200:0.0:1788880577.154014:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880577.154022:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880577.154024:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880577.154026:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880577.154028:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880577.154030:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880577.154032:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880577.154035:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880577.154041:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=105 v=5 (5 5 5 0) 00000100:00001000:0.0:1788880577.154045:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=105 v=5 (1 1 1 0) 00000100:00000001:0.0:1788880577.154054:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880577.154055:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880577.154058:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880577.154060:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880577.154063:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880577.154067:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa55 [1] + 224 00000100:00000040:0.0:1788880577.154070:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880577.154076:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880577.154081:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880577.154085:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000800:00000001:3.0:1788880577.154085:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880577.154088:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880577.154090:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:0.0:1788880577.154095:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880577.154103:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00100000:0.0:1788880577.154110:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780669184:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880577.154110:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880577.154117:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880577.154118:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880577.154122:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880577.154128:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c09231ec x1875776780669184/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880577.154135:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880577.154142:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880577.154146:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880577.154151:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007f19b4dd. 02000000:00000001:0.0:1788880577.154156:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880577.154158:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880577.154161:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880577.154162:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:0.0:1788880577.154164:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880577.154168:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 00000800:00000001:3.0:1788880577.154170:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:0.0:1788880577.154173:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000800:00000001:3.0:1788880577.154174:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880577.154176:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880577.154180:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c09231ec. 00000100:00000001:0.0:1788880577.154185:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880577.154187:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880577.154189:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880577.154199:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880577.154202:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880577.154205:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880577.154213:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880577.154221:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880577.154228:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880577.154276:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880577.154279:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880577.154281:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000702dddb0. 00000400:00000200:2.0:1788880577.154288:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000044b1d7fc 00000400:00000010:2.0:1788880577.154290:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000044b1d7fc. 00000100:00000001:2.0:1788880577.154295:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880577.154303:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880577.154313:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880577.154333:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880577.154335:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880577.154338:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880577.154341:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880577.154343:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880577.154345:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880577.154348:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880577.154358:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=105 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880577.154361:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=62 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880577.154364:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880577.154365:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880577.154368:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880577.154373:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880577.154380:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880577.154383:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880577.154385:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880577.154387:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880577.154388:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880577.154391:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880577.154415:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880577.154418:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880577.154419:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880577.154423:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880577.154430:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780669440:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880577.154435:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880577.154437:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880577.154439:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002dcbd5d0 x1875776780669440/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880577.154446:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880577.154448:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880577.154450:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004ffd965d. 02000000:00000001:2.0:1788880577.154455:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880577.154457:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880577.154458:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880577.154461:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880577.154462:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:2.0:1788880577.154466:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880577.154467:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880577.154469:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002dcbd5d0. 00000100:00000001:2.0:1788880577.154472:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880577.154473:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880577.154475:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880577.154480:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880577.154482:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880577.154484:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880577.154486:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880577.154794:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880577.154802:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880577.154804:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880577.154806:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880577.154812:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880577.154834:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09180 00000400:00000200:1.0:1788880577.154841:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa49 [1] + 224 00000800:00000001:1.0:1788880577.154846:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880577.154851:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880577.154853:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880577.154856:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880577.154860:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880577.154864:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880577.154874:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880577.154895:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880577.154901:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880577.154904:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880577.155356:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880577.155360:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880577.155371:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880577.155373:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880577.155375:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000bfe5eb98. 00000400:00000200:3.0:1788880577.155378:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000400:00000010:3.0:1788880577.155380:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:3.0:1788880577.155383:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880577.155387:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880577.155395:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880577.155400:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880577.155401:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880577.155404:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880577.155405:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880577.155407:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880577.155409:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880577.155411:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880577.155421:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=95 v=5 (5 5 5 0) 00000100:00001000:3.0:1788880577.155425:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=95 v=5 (1 1 1 0) 00000100:00000001:3.0:1788880577.155428:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880577.155429:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880577.155431:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880577.155433:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880577.155436:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880577.155438:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880577.155439:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880577.155441:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880577.155444:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880577.155451:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880577.155453:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880577.155454:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880577.155457:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880577.155478:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780669312:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880577.155483:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880577.155485:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880577.155487:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000003b122924 x1875776780669312/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880593 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880577.155492:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880577.155494:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880577.155496:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000057e92cbc. 02000000:00000001:3.0:1788880577.155499:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880577.155501:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880577.155502:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880577.155504:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880577.155506:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:3.0:1788880577.155508:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880577.155510:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880577.155511:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000003b122924. 00000100:00000001:3.0:1788880577.155514:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880577.155515:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880577.155517:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880577.155523:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880577.155525:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880577.155526:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880577.155527:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880577.214357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880577.214362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880577.214367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880577.214370:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880577.214372:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880577.214374:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880577.214377:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880577.214379:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880577.214380:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880577.214381:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880577.214434:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880577.214437:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880577.214440:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880578.238361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880578.238368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880578.238375:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880578.238376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880578.238432:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880578.238435:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880578.238437:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880579.262359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880579.262365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880579.262371:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880579.262373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880579.262437:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880579.262441:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880579.262444:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880580.286357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880580.286364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880580.286370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880580.286371:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880580.286417:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880580.286420:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880580.286422:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880581.310334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880581.310341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880581.310347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880581.310348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880581.310396:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880581.310399:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880581.310401:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880582.270305:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306213328, at 4306213376 00000100:00000010:1.0:1788880582.270319:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000029f028ab. 00000100:00000040:3.0:1788880582.270326:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306213328, at 4306213376 00000100:00000040:2.0:1788880582.270327:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306213328, at 4306213376 00000100:00000040:1.0:1788880582.270334:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:3.0:1788880582.270335:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000085563e6f. 00000100:00000010:2.0:1788880582.270337:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014321469. 00000100:00000001:1.0:1788880582.270344:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880582.270350:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880582.270361:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880582.270365:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880582.270368:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880582.270373:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880582.270380:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:1.0:1788880582.270387:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880582.270397:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880582.270408:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880582.270411:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880582.270416:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000008b25f66] to pc [ptlrpcd_00_00+0] req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880582.270443:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880582.270446:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880582.270447:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880582.270451:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880582.270452:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880582.270455:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000070db75b2. 00000020:00000040:1.0:1788880582.270459:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880582.270459:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880582.270463:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880582.270465:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:3.0:1788880582.270465:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880582.270469:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880582.270474:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:2.0:1788880582.270475:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880582.270476:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880582.270478:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880582.270479:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880582.270486:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880582.270487:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880582.270488:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:3.0:1788880582.270489:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880582.270496:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880582.270498:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880582.270501:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000070db75b2] to pc [ptlrpcd_00_01+1] req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880582.270511:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880582.270513:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880582.270521:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880582.270524:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001cf64abe. 00000100:00000001:2.0:1788880582.270524:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:1.0:1788880582.270528:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880582.270528:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880582.270531:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880582.270532:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880582.270534:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880582.270536:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880582.270537:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880582.270539:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 02000000:00000001:3.0:1788880582.270543:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880582.270545:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880582.270552:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780669568:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880582.270553:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880582.270556:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880582.270559:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880582.270562:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880582.270564:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880582.270569:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880582.270570:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880582.270570:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880582.270574:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880582.270574:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880582.270576:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001cf64abe] to pc [ptlrpcd_00_02+2] req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880582.270581:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880582.270581:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880582.270587:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000036ec89fa. 00000100:00100000:2.0:1788880582.270590:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780669696:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880582.270592:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880582.270596:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880582.270598:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880582.270599:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880582.270600:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bfe5eb98. 02000000:00000001:2.0:1788880582.270602:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:1.0:1788880582.270604:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:2.0:1788880582.270607:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880582.270610:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:3.0:1788880582.270610:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 00000100:00000040:1.0:1788880582.270611:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11551) 02000000:00000010:2.0:1788880582.270617:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000099f3ac40. 00000100:00000200:3.0:1788880582.270618:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780669568, portal 25 02000000:00000001:2.0:1788880582.270621:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880582.270623:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000010:2.0:1788880582.270626:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000702dddb0. 00000100:00000001:3.0:1788880582.270626:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000001:1.0:1788880582.270633:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:2.0:1788880582.270633:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000044b1d7fc. 00000100:00000040:3.0:1788880582.270636:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880598 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880582.270638:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:2.0:1788880582.270640:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780669696, portal 10 00000100:00000001:1.0:1788880582.270642:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880582.270647:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:1.0:1788880582.270650:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880582.270651:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938046976 : -126823771504640 : ffff8ca78a268000) 00000100:00000001:3.0:1788880582.270654:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880582.270662:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880582.270667:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000082116f14. 00000100:00000001:1.0:1788880582.270668:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880582.270671:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880582.270672:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880582.270673:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780669568, offset 0 02000000:00000001:1.0:1788880582.270675:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880582.270678:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880582.270683:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880582.270688:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788880582.270690:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780669824:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880582.270695:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880582.270695:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880582.270701:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880582.270705:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:3.0:1788880582.270705:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:2.0:1788880582.270707:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e1dae22. 02000000:00000001:1.0:1788880582.270709:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880582.270714:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:2.0:1788880582.270714:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780669696, offset 0 00000400:00000200:3.0:1788880582.270715:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:1.0:1788880582.270720:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e7b82f46. 00000400:00000200:3.0:1788880582.270722:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788880582.270726:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880582.270727:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880582.270730:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:1.0:1788880582.270733:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b75bbc07. 00000400:00000200:3.0:1788880582.270734:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880582.270739:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [305:305:256:4294967295] 192.168.204.111@tcp LPNI seq info [305:305:8:4294967295] 00000400:00000010:1.0:1788880582.270741:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000100:00000200:1.0:1788880582.270745:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780669824, portal 10 00000100:00000001:1.0:1788880582.270751:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880582.270754:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938054144 : -126823771497472 : ffff8ca78a269c00) 00000400:00000200:3.0:1788880582.270762:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:1.0:1788880582.270763:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880582.270773:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880582.270774:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:3.0:1788880582.270779:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000088542ca3. 00000400:00000200:2.0:1788880582.270784:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880582.270785:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:3.0:1788880582.270789:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:1.0:1788880582.270791:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880582.270797:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d71e0867. 00000800:00000200:3.0:1788880582.270799:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:1.0:1788880582.270803:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780669824, offset 0 00000400:00000200:2.0:1788880582.270803:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880582.270805:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000088542ca3 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880582.270815:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880582.270815:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880582.270816:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880582.270823:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880582.270824:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306218447 00000400:00000200:2.0:1788880582.270831:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [306:306:255:4294967295] 192.168.204.111@tcp LPNI seq info [306:306:7:4294967295] 00000100:00000001:3.0:1788880582.270831:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306213328 00000100:00000001:3.0:1788880582.270837:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306218447 00000100:00000010:3.0:1788880582.270841:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000015631aee. 00000100:00000001:3.0:1788880582.270847:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:2.0:1788880582.270851:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880582.270852:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306218447 00000100:00000010:3.0:1788880582.270858:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014c076ca. 00000800:00000200:2.0:1788880582.270862:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880582.270863:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:1.0:1788880582.270864:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880582.270870:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006aca3ff7. 00000400:00000200:1.0:1788880582.270872:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880582.270872:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306218447 00000400:00000200:1.0:1788880582.270878:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:3.0:1788880582.270879:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000625b9f8b. 00000800:00000200:2.0:1788880582.270882:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880582.270884:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880582.270885:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788880582.270892:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880582.270893:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880582.270894:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880582.270899:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880582.270900:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006aca3ff7 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880582.270903:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [307:307:254:4294967295] 192.168.204.111@tcp LPNI seq info [307:307:6:4294967295] 00000100:00000001:3.0:1788880582.270906:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880582.270908:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880582.270910:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880582.270914:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 295110 usecs (296 jiffies) 00000100:00000001:3.0:1788880582.270918:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880582.270921:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306213672 00000100:00000001:3.0:1788880582.270921:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:1.0:1788880582.270924:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880582.270925:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880582.270927:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306218447 00000100:00000001:3.0:1788880582.270928:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:0.0:1788880582.270930:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880582.270931:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880582.270933:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306213672 00000800:00000010:0.0:1788880582.270935:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000088542ca3. 00000800:00000200:1.0:1788880582.270936:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880582.270937:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880582.270939:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306218447 00000400:00000200:0.0:1788880582.270942:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000010:1.0:1788880582.270944:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000001:2.0:1788880582.270944:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306213672 00000100:00000001:3.0:1788880582.270948:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880582.270950:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880582.270951:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880582.270952:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880582.270958:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306218447 00000400:00000200:0.0:1788880582.270960:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000082116f14 00000800:00000200:1.0:1788880582.270963:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:0.0:1788880582.270965:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000082116f14. 00000100:00000001:2.0:1788880582.270967:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306213672 00000800:00000200:1.0:1788880582.270968:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880582.270972:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880582.270972:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306218447 00000100:00000001:1.0:1788880582.270975:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880582.270977:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880582.270979:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880598 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880582.270981:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 19186 usecs (20 jiffies) 00000100:00000001:2.0:1788880582.270982:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880582.270987:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306213396 00000100:00000001:2.0:1788880582.270988:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880582.270992:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880582.270998:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880582.270998:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306218447 00000100:00000001:2.0:1788880582.270998:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880582.271002:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880582.271003:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306213396 00000100:00000040:0.0:1788880582.271004:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880598 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880582.271007:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306218447 00000100:00000001:2.0:1788880582.271008:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880582.271012:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306213396 00000100:00000001:2.0:1788880582.271012:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880582.271016:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880582.271018:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306218447 00000100:00000001:0.0:1788880582.271019:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880582.271020:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880582.271022:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306213396 00000100:00000001:0.0:1788880582.271023:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880582.271026:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306218447 00000100:00000001:1.0:1788880582.271028:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880582.271031:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880582.271033:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880582.271035:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880582.271036:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880582.271038:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:0.0:1788880582.271039:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006aca3ff7. 00000100:00000001:1.0:1788880582.271040:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880582.271042:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880582.271043:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880582.271046:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880582.271049:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880582.271050:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880582.271054:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880582.271055:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e1dae22 00000400:00000010:0.0:1788880582.271057:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e1dae22. 00000100:00000001:0.0:1788880582.271060:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880582.271062:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880582.271067:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880582.271073:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880582.271077:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880582.271078:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880582.271107:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880582.271110:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:0.0:1788880582.271113:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880582.271117:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880582.271119:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d71e0867 00000400:00000010:0.0:1788880582.271121:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d71e0867. 00000100:00000001:0.0:1788880582.271124:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880582.271127:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880582.271133:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880582.271135:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880582.271141:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880582.271142:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880582.272796:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880582.272808:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880582.272811:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880582.272814:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880582.272821:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880582.272833:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09300 00000400:00000200:0.0:1788880582.272843:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa61 [1] + 224 00000800:00000001:0.0:1788880582.272849:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880582.272853:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880582.272855:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880582.272859:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880582.272863:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880582.272869:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880582.272879:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880582.272904:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880582.272912:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880582.272916:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880582.272918:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880582.272920:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880582.272925:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880582.272931:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09380 00000400:00000200:0.0:1788880582.272936:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa6d [1] + 224 00000800:00000001:0.0:1788880582.272939:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880582.272942:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880582.272944:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880582.272948:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880582.272951:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880582.272954:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880582.272962:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880582.272976:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880582.272980:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880582.272984:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880582.272985:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880582.272987:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880582.272991:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880582.272996:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b09280 00000400:00000200:0.0:1788880582.273000:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xa5d [1] + 224 00000800:00000001:0.0:1788880582.273004:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880582.273011:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880582.273012:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880582.273013:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880582.273017:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880582.273020:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:1.0:1788880582.273025:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880582.273028:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880582.273030:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b75bbc07. 00000400:00000200:1.0:1788880582.273034:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000400:00000010:1.0:1788880582.273036:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:1.0:1788880582.273039:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880582.273044:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880582.273055:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880582.273055:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880582.273063:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880582.273064:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880582.273067:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880582.273070:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880582.273078:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880582.273079:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880582.273081:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880582.273084:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880582.273087:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:1.0:1788880582.273092:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=110 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880582.273098:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=67 v=5 (1 0 0 0) 00000100:00000200:0.0:1788880582.273100:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880582.273102:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880582.273105:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880582.273110:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:0.0:1788880582.273114:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00080000:1.0:1788880582.273118:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880582.273133:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880582.273139:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:1.0:1788880582.273141:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880582.273143:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880582.273146:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788880582.273153:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880582.273154:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880582.273157:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880582.273159:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000001:0.0:1788880582.273161:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880582.273162:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880582.273164:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880582.273168:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000bfe5eb98. 00000100:00000040:1.0:1788880582.273171:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880582.273172:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000400:00000010:0.0:1788880582.273175:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:0.0:1788880582.273179:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880582.273185:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880582.273185:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880582.273191:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880582.273194:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880582.273202:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880582.273202:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880582.273253:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880582.273255:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880582.273259:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880582.273262:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00100000:1.0:1788880582.273263:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780669824:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880582.273265:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880582.273267:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880582.273271:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880582.273272:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880582.273274:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=110 v=5 (5 5 5 0) 00000100:00000001:1.0:1788880582.273276:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00001000:0.0:1788880582.273280:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=110 v=5 (1 1 1 0) 00000100:00000040:1.0:1788880582.273283:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000001cf64abe x1875776780669824/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880582.273285:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880582.273288:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880582.273291:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880582.273294:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880582.273295:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880582.273298:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880582.273299:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880582.273301:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e7b82f46. 02000000:00000001:1.0:1788880582.273305:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:0.0:1788880582.273306:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788880582.273313:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880582.273314:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880582.273316:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880582.273318:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 02000000:00000001:1.0:1788880582.273322:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880582.273325:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880582.273327:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000001cf64abe. 00000100:00000001:0.0:1788880582.273330:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880582.273331:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880582.273333:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880582.273336:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880582.273336:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880582.273340:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880582.273343:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880582.273344:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880582.273346:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880582.273350:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880582.273352:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880582.273359:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780669568:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880582.273365:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880582.273367:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880582.273369:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780669568/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880582.273374:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880582.273376:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880582.273378:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000036ec89fa. 02000000:00000001:0.0:1788880582.273380:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880582.273381:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880582.273382:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880582.273384:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880582.273385:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:0.0:1788880582.273387:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880582.273388:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880582.273389:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:0.0:1788880582.273392:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880582.273393:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880582.273394:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880582.273398:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880582.273399:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880582.273401:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880582.273402:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880582.273456:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880582.273458:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880582.273464:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000702dddb0. 00000400:00000200:3.0:1788880582.273467:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000044b1d7fc 00000400:00000010:3.0:1788880582.273470:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000044b1d7fc. 00000100:00000001:3.0:1788880582.273473:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880582.273478:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880582.273486:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880582.273492:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880582.273493:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880582.273495:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880582.273497:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880582.273498:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880582.273499:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880582.273501:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880582.273504:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=100 v=5 (5 5 5 0) 00000100:00001000:3.0:1788880582.273507:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=100 v=5 (1 1 1 0) 00000100:00000001:3.0:1788880582.273509:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880582.273511:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880582.273512:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880582.273515:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880582.273517:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880582.273519:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880582.273521:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880582.273522:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880582.273524:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880582.273530:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880582.273532:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880582.273533:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880582.273536:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880582.273553:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780669696:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880582.273557:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880582.273558:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880582.273561:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000070db75b2 x1875776780669696/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880598 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880582.273566:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880582.273568:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880582.273569:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000099f3ac40. 02000000:00000001:3.0:1788880582.273572:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880582.273573:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880582.273573:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880582.273575:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880582.273576:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:3.0:1788880582.273578:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880582.273579:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880582.273580:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000070db75b2. 00000100:00000001:3.0:1788880582.273582:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880582.273583:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880582.273585:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880582.273588:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880582.273589:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880582.273591:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880582.273592:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880582.334335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880582.334341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880582.334345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880582.334346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880582.334395:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880582.334398:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880582.334401:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880583.358402:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880583.358408:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880583.358414:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880583.358415:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880583.358482:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880583.358486:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880583.358488:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880584.382328:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880584.382335:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880584.382337:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880584.382339:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880584.382343:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880584.382344:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880584.382346:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880584.382347:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880584.382350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880584.382350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880584.382400:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880584.382403:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880584.382405:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880585.406353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880585.406360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880585.406365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880585.406366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880585.406449:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880585.406455:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880585.406476:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880586.430362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880586.430371:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880586.430400:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880586.430401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880586.430465:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880586.430469:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880586.430472:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880587.390354:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306218447, at 4306218496 00000100:00000040:0.0:1788880587.390355:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306218447, at 4306218496 00000100:00000040:3.0:1788880587.390355:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306218447, at 4306218496 00000100:00000010:2.0:1788880587.390375:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014c076ca. 00000100:00000010:0.0:1788880587.390377:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000a4f85af. 00000100:00000010:3.0:1788880587.390383:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000625b9f8b. 00000100:00000040:1.0:1788880587.390435:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306218447, at 4306218496 00000100:00000010:1.0:1788880587.390443:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000015631aee. 00000100:00000040:3.0:1788880587.390452:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880587.390458:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880587.390464:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880587.390469:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880587.390473:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880587.390476:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880587.390480:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880587.390489:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000088542ca3. 00000100:00000001:3.0:1788880587.390496:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880587.390506:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880587.390517:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880587.390520:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880587.390525:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_03+3] req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880587.390548:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880587.390550:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880587.390553:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880587.390556:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000020:00000040:3.0:1788880587.390559:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880587.390561:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880587.390563:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880587.390565:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880587.390568:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000446e4ede. 00000100:00000001:0.0:1788880587.390570:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880587.390571:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880587.390574:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880587.390575:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880587.390576:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880587.390579:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880587.390581:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880587.390584:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880587.390585:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.390586:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880587.390592:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_00+0] req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880587.390593:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880587.390595:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880587.390599:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880587.390602:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880587.390604:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880587.390606:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780669952:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880587.390610:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880587.390615:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880587.390617:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.390621:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880587.390623:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880587.390626:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880587.390627:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000010:0.0:1788880587.390630:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007f19b4dd. 00000100:00000010:3.0:1788880587.390630:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 02000000:00000001:0.0:1788880587.390632:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880587.390633:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880587.390635:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880587.390635:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880587.390639:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880587.390640:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880587.390642:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880587.390643:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000c66bed7. 00000100:00000001:1.0:1788880587.390643:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:3.0:1788880587.390647:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b899fe8d. 00000400:00000010:0.0:1788880587.390650:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002aa6ba25. 00000100:00000040:1.0:1788880587.390652:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880587.390652:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880587.390657:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780669952, portal 25 00000100:00000040:3.0:1788880587.390660:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880587.390664:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880587.390665:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880587.390669:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000001:2.0:1788880587.390669:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880587.390677:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880603 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880587.390677:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880587.390680:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880587.390681:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.390682:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880587.390688:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_01+1] req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880587.390689:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880587.390689:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880587.390694:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880587.390699:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880587.390700:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880587.390702:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000100:00000200:0.0:1788880587.390706:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780669952, offset 0 00000100:00100000:1.0:1788880587.390707:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780670080:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880587.390710:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880587.390714:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880587.390715:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:0.0:1788880587.390716:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880587.390720:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880587.390720:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:1.0:1788880587.390723:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880587.390726:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11556) 02000000:00000001:1.0:1788880587.390727:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880587.390731:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000010:1.0:1788880587.390734:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a6d90e67. 02000000:00000001:1.0:1788880587.390738:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880587.390744:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b75bbc07. 00000400:00000200:0.0:1788880587.390745:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:1.0:1788880587.390749:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000400:00000200:0.0:1788880587.390752:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:1.0:1788880587.390755:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780670080, portal 10 00000400:00000200:0.0:1788880587.390757:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880587.390760:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880587.390763:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880587.390765:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000100:00000001:2.0:1788880587.390765:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880587.390769:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880587.390771:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [308:308:256:4294967295] 192.168.204.111@tcp LPNI seq info [308:308:8:4294967295] 00000100:00000001:2.0:1788880587.390773:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880587.390777:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880587.390781:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880587.390792:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880587.390800:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880587.390808:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880587.390808:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880587.390810:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880587.390814:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000010:0.0:1788880587.390816:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000004e0f6846. 02000000:00000001:2.0:1788880587.390816:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880587.390819:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880587.390821:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880587.390824:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000062e8e6f4. 00000100:00100000:2.0:1788880587.390826:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780670208:192.168.204.111@tcp:400: 00000800:00000200:0.0:1788880587.390828:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880587.390830:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880587.390834:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000200:1.0:1788880587.390836:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780670080, offset 0 02000000:00000001:2.0:1788880587.390836:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880587.390842:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:2.0:1788880587.390842:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880587.390847:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000010b36294. 00000800:00000200:0.0:1788880587.390848:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000004e0f6846 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880587.390848:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880587.390852:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880587.390856:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001b82b6de. 00000400:00000200:1.0:1788880587.390860:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:2.0:1788880587.390861:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000044b1d7fc. 00000100:00000001:0.0:1788880587.390866:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880587.390867:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:2.0:1788880587.390867:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780670208, portal 10 00000100:00000001:2.0:1788880587.390873:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880587.390874:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306223567 00000100:00000001:2.0:1788880587.390875:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000010:0.0:1788880587.390881:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000a4f85af. 00000400:00000200:1.0:1788880587.390885:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:2.0:1788880587.390886:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880587.390887:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:1.0:1788880587.390892:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880587.390893:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306223567 00000100:00000010:0.0:1788880587.390900:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001a2ff840. 00000400:00000200:1.0:1788880587.390900:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880587.390906:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880587.390907:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:1.0:1788880587.390908:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [309:309:255:4294967295] 192.168.204.111@tcp LPNI seq info [309:309:7:4294967295] 00000100:00000001:0.0:1788880587.390913:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306223567 00000100:00000040:2.0:1788880587.390916:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880587.390918:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000bc13d1e4. 00000400:00000010:2.0:1788880587.390922:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e1dae22. 00000100:00000001:0.0:1788880587.390924:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:1.0:1788880587.390928:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:2.0:1788880587.390929:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780670208, offset 0 00000100:00000001:0.0:1788880587.390930:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306223567 00000100:00000010:0.0:1788880587.390937:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009f22a9a0. 00000100:00000001:0.0:1788880587.390941:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880587.390945:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880587.390945:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880587.390946:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880587.390952:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880587.390953:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000001:0.0:1788880587.390957:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880587.390962:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880587.390962:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880587.390963:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880587.390972:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880587.390973:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:1.0:1788880587.390977:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880587.390978:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:3.0:1788880587.390978:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:2.0:1788880587.390980:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880587.390984:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000800:00000010:3.0:1788880587.390985:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000004e0f6846. 00000400:00000200:2.0:1788880587.390987:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880587.390988:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880587.390991:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880587.390991:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880587.390992:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880587.390993:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880587.390998:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880587.390999:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880587.391000:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 291798 usecs (292 jiffies) 00000100:00000001:1.0:1788880587.391004:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306218788 00000100:00000001:1.0:1788880587.391007:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306223567 00000100:00000001:1.0:1788880587.391009:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306218788 00000100:00000001:1.0:1788880587.391012:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306223567 00000100:00000001:1.0:1788880587.391014:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306218788 00000100:00000001:1.0:1788880587.391016:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306223567 00000100:00000001:1.0:1788880587.391017:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306218788 00000100:00000001:1.0:1788880587.391019:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306223567 00000100:00000001:1.0:1788880587.391021:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880587.391023:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880587.391025:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880587.391027:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880587.391037:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880587.391038:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880587.391040:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880587.391041:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880587.391042:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880587.391044:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880587.391353:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880587.391356:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [310:310:254:4294967295] 192.168.204.111@tcp LPNI seq info [310:310:6:4294967295] 00000400:00000200:2.0:1788880587.391366:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880587.391371:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880587.391374:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000800:00000010:2.0:1788880587.391375:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000007cb951ea. 00000400:00000010:3.0:1788880587.391379:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000800:00000200:2.0:1788880587.391384:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880587.391386:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:2.0:1788880587.391390:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:3.0:1788880587.391393:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880603 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880587.391396:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000007cb951ea type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880587.391402:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880587.391407:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 18997 usecs (19 jiffies) 00000100:00000001:2.0:1788880587.391408:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306218516 00000100:00000001:2.0:1788880587.391411:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306223567 00000100:00000001:2.0:1788880587.391412:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306218516 00000100:00000001:2.0:1788880587.391413:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306223567 00000100:00000001:2.0:1788880587.391414:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306218516 00000100:00000001:2.0:1788880587.391417:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306223567 00000100:00000001:2.0:1788880587.391421:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306218516 00000100:00000001:2.0:1788880587.391425:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306223567 00000100:00000001:2.0:1788880587.391429:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880587.391433:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880587.391436:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880587.391437:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880587.391438:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880587.391440:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880587.391443:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:3.0:1788880587.391444:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880603 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880587.391455:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880587.391455:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.391458:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880587.391459:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880587.391462:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880587.391464:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880587.391485:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880587.391488:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:3.0:1788880587.391491:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880587.391494:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880587.391496:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000062e8e6f4 00000400:00000010:3.0:1788880587.391498:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000062e8e6f4. 00000100:00000001:3.0:1788880587.391500:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880587.391502:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880587.391509:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880587.391511:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880587.391516:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.391517:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880587.391545:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880587.391547:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000007cb951ea. 00000400:00000200:3.0:1788880587.391550:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880587.391552:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880587.391554:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e1dae22 00000400:00000010:3.0:1788880587.391555:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e1dae22. 00000100:00000001:3.0:1788880587.391557:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880587.391559:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880587.391565:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880587.391567:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880587.391582:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.391583:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880587.392320:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880587.392331:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880587.392334:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880587.392336:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880587.392342:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880587.392351:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b09400 00000400:00000200:0.0:1788880587.392358:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xa75 [1] + 224 00000800:00000001:0.0:1788880587.392364:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880587.392369:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880587.392371:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880587.392374:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880587.392378:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880587.392381:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880587.392391:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880587.392406:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880587.392412:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880587.392415:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880587.392475:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880587.392478:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880587.392483:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880587.392485:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880587.392487:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000c66bed7. 00000400:00000200:3.0:1788880587.392490:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002aa6ba25 00000400:00000010:3.0:1788880587.392491:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002aa6ba25. 00000100:00000001:3.0:1788880587.392494:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880587.392498:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880587.392506:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880587.392511:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880587.392513:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880587.392515:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880587.392518:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880587.392520:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880587.392522:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880587.392524:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880587.392530:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=115 v=5 (5 5 5 0) 00000100:00001000:3.0:1788880587.392534:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=115 v=5 (1 1 1 0) 00000100:00000001:3.0:1788880587.392536:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880587.392537:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880587.392540:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880587.392542:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.392544:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880587.392547:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880587.392553:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880587.392556:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880587.392557:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880587.392560:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880587.392573:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780669952:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880587.392578:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880587.392580:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880587.392582:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780669952/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880587.392589:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880587.392600:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880587.392602:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007f19b4dd. 02000000:00000001:3.0:1788880587.392605:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880587.392607:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880587.392608:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880587.392610:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880587.392612:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000088542ca3. 02000000:00000001:3.0:1788880587.392616:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880587.392618:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880587.392619:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:3.0:1788880587.392622:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880587.392623:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880587.392625:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880587.392629:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880587.392630:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.392632:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880587.392634:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880587.393030:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880587.393057:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880587.393059:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880587.393062:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880587.393067:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880587.393087:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09480 00000400:00000200:0.0:1788880587.393094:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa7d [1] + 224 00000800:00000001:0.0:1788880587.393098:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880587.393103:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880587.393105:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880587.393112:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880587.393115:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880587.393118:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880587.393136:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880587.393162:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880587.393168:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880587.393171:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880587.393173:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880587.393175:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880587.393178:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880587.393184:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09500 00000400:00000200:0.0:1788880587.393188:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa85 [1] + 224 00000800:00000001:0.0:1788880587.393191:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880587.393195:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880587.393197:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880587.393199:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880587.393202:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880587.393204:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880587.393211:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880587.393230:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880587.393250:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880587.393253:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880587.393256:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1788880587.393258:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:0.0:1788880587.393261:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880587.393261:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880587.393263:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001b82b6de. 00000400:00000200:3.0:1788880587.393268:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000044b1d7fc 00000400:00000010:3.0:1788880587.393270:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000044b1d7fc. 00000100:00000001:3.0:1788880587.393274:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880587.393279:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880587.393285:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880587.393289:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:3.0:1788880587.393290:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788880587.393297:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880587.393300:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880587.393303:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b75bbc07. 00000400:00000200:2.0:1788880587.393309:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000100:00000001:3.0:1788880587.393309:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880587.393312:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880587.393313:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:3.0:1788880587.393316:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880587.393319:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880587.393321:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880587.393322:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880587.393325:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880587.393329:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880587.393329:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880587.393335:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=115 v=5 (5 5 5 0) 00000100:00001000:3.0:1788880587.393342:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=72 v=5 (1 0 0 0) 00000100:00000200:2.0:1788880587.393345:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880587.393348:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880587.393351:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880587.393355:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880587.393356:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880587.393358:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880587.393362:0:201285:0:(client.c:1580:after_reply()) Process entered 00010000:00080000:3.0:1788880587.393362:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880587.393366:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880587.393368:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880587.393371:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880587.393375:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880587.393376:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.393380:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00001000:2.0:1788880587.393381:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=105 v=5 (5 5 5 0) 00000100:00000040:3.0:1788880587.393382:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880587.393386:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00001000:2.0:1788880587.393387:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=105 v=5 (1 1 1 0) 00000100:00000001:3.0:1788880587.393388:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880587.393394:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:3.0:1788880587.393395:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880587.393397:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880587.393401:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880587.393404:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.393406:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880587.393407:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880587.393409:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880587.393412:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880587.393412:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880587.393415:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.393415:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880587.393419:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880587.393422:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880587.393432:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00100000:3.0:1788880587.393437:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780670208:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880587.393442:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880587.393444:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880587.393445:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880587.393450:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880587.393452:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880587.393457:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780670208/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880587.393462:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780670080:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880587.393468:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880587.393472:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880587.393476:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780670080/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880603 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880587.393483:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880587.393485:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880587.393487:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a6d90e67. 02000000:00000001:2.0:1788880587.393490:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880587.393491:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880587.393492:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880587.393494:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880587.393495:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000446e4ede. 02000000:00000001:2.0:1788880587.393498:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880587.393499:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880587.393500:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:2.0:1788880587.393503:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880587.393504:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880587.393505:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880587.393509:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880587.393510:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880587.393512:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880587.393514:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.393535:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880587.393536:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880587.393538:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000010b36294. 02000000:00000001:3.0:1788880587.393542:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880587.393543:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880587.393544:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880587.393546:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880587.393547:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b899fe8d. 02000000:00000001:3.0:1788880587.393549:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880587.393551:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880587.393552:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:3.0:1788880587.393555:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880587.393556:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880587.393558:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880587.393561:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880587.393562:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880587.393564:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880587.393573:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880587.454342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880587.454349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880587.454355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880587.454359:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880587.454361:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880587.454364:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880587.454367:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880587.454370:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880587.454372:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880587.454373:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880587.454431:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880587.454436:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880587.454439:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880588.478934:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880588.478941:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880588.478947:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880588.478948:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880588.479004:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880588.479007:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880588.479009:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880589.502339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880589.502345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880589.502350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880589.502351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880589.502399:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880589.502402:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880589.502417:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880590.526321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880590.526328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880590.526334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880590.526335:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880590.526380:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880590.526385:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880590.526388:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880591.550352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880591.550358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880591.550362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880591.550363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880591.550415:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880591.550417:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880591.550419:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880592.510341:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306223567, at 4306223616 00000100:00000040:1.0:1788880592.510341:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306223567, at 4306223616 00000100:00000010:1.0:1788880592.510359:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001a2ff840. 00000100:00000010:0.0:1788880592.510361:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000a4f85af. 00000100:00000040:3.0:1788880592.510458:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306223567, at 4306223616 00000100:00000010:3.0:1788880592.510465:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009f22a9a0. 00000100:00000040:2.0:1788880592.510568:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306223567, at 4306223616 00000100:00000010:2.0:1788880592.510584:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000bc13d1e4. 00000100:00000040:2.0:1788880592.510602:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880592.510607:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880592.510618:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000078d5a15d. 00000020:00000040:2.0:1788880592.510624:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880592.510649:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880592.510652:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880592.510659:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880592.510668:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000100:00000001:2.0:1788880592.510676:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880592.510690:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880592.510703:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880592.510706:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880592.510713:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000078d5a15d] to pc [ptlrpcd_00_02+2] req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880592.510747:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880592.510750:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880592.510754:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880592.510757:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000020:00000040:2.0:1788880592.510759:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880592.510762:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880592.510765:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880592.510766:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880592.510770:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 00000100:00000001:2.0:1788880592.510773:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880592.510778:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880592.510781:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880592.510786:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880592.510788:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880592.510790:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880592.510794:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880592.510795:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880592.510796:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880592.510803:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880592.510803:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_03+3] req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880592.510823:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880592.510827:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880592.510833:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880592.510836:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880592.510837:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880592.510841:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880592.510842:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788880592.510849:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780670336:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880592.510849:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880592.510856:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000010:2.0:1788880592.510860:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 02000000:00000001:1.0:1788880592.510865:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000020:00000040:2.0:1788880592.510865:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880592.510868:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880592.510869:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880592.510871:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880592.510876:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880592.510877:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880592.510878:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880592.510880:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880592.510883:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:1.0:1788880592.510886:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e7b82f46. 02000000:00000010:2.0:1788880592.510888:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 02000000:00000001:1.0:1788880592.510891:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880592.510895:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880592.510898:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880592.510903:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000744a548a. 00000100:00000040:2.0:1788880592.510905:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:1.0:1788880592.510917:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000100:00000001:2.0:1788880592.510931:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880592.510933:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880592.510934:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780670336, portal 25 00000100:00000001:1.0:1788880592.510942:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:2.0:1788880592.510945:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_00+0] req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880592.510948:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938426496 : -126823771125120 : ffff8ca78a2c4a80) 00000100:00000040:1.0:1788880592.510963:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880608 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880592.510969:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880592.510988:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880592.510994:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880592.510997:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880592.511000:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000061cfcb81. 00000100:00000200:1.0:1788880592.511006:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780670336, offset 0 02000000:00000001:3.0:1788880592.511006:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880592.511018:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880592.511019:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880592.511019:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880592.511041:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880592.511041:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880592.511052:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880592.511053:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880592.511059:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00100000:3.0:1788880592.511060:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780670464:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880592.511065:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11561) 00000100:00000001:0.0:1788880592.511070:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880592.511072:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880592.511076:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880592.511076:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788880592.511082:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:0.0:1788880592.511087:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880592.511087:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880592.511089:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880592.511093:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880592.511095:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880592.511096:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880592.511105:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880592.511113:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880592.511121:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [311:311:256:4294967295] 192.168.204.111@tcp LPNI seq info [311:311:8:4294967295] 02000000:00000010:3.0:1788880592.511124:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004080df20. 00000100:00000001:0.0:1788880592.511126:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880592.511131:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880592.511135:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880592.511139:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880592.511142:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880592.511146:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880592.511146:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000d609d5db. 00000100:00100000:0.0:1788880592.511156:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780670592:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880592.511159:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:3.0:1788880592.511159:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f73021f1. 02000000:00000001:0.0:1788880592.511165:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1788880592.511166:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880592.511169:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880592.511172:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:3.0:1788880592.511172:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780670464, portal 10 02000000:00000010:0.0:1788880592.511176:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002f5492f5. 00000100:00000001:3.0:1788880592.511181:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:0.0:1788880592.511183:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880592.511186:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 00000400:00000010:0.0:1788880592.511190:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f435c455. 00000400:00000010:0.0:1788880592.511195:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000115221d8. 00000100:00000040:3.0:1788880592.511202:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880592.511204:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:0.0:1788880592.511205:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780670592, portal 10 00000100:00000001:0.0:1788880592.511218:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880592.511220:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000800:00000010:1.0:1788880592.511224:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000040:0.0:1788880592.511258:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880592.511268:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880592.511270:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:1.0:1788880592.511274:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:0.0:1788880592.511275:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880592.511283:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880592.511285:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000400:00000010:3.0:1788880592.511288:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005f2b805f. 00000100:00000200:3.0:1788880592.511292:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780670464, offset 0 00000800:00000200:1.0:1788880592.511295:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:0.0:1788880592.511305:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780670592, offset 0 00000800:00000200:1.0:1788880592.511306:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880592.511314:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880592.511325:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880592.511330:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880592.511337:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880592.511339:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306228688 00000100:00000010:1.0:1788880592.511350:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000400:00000200:3.0:1788880592.511352:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880592.511362:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:3.0:1788880592.511363:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880592.511372:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306228688 00000400:00000200:3.0:1788880592.511372:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880592.511379:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:1.0:1788880592.511380:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000100:00000001:1.0:1788880592.511387:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:3.0:1788880592.511391:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [312:312:255:4294967295] 192.168.204.111@tcp LPNI seq info [312:312:7:4294967295] 00000100:00000001:1.0:1788880592.511397:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306228688 00000100:00000010:1.0:1788880592.511404:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000100:00000001:1.0:1788880592.511410:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:3.0:1788880592.511416:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880592.511418:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306228688 00000100:00000010:1.0:1788880592.511425:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000800:00000200:3.0:1788880592.511428:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880592.511429:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880592.511433:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:0.0:1788880592.511435:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:3.0:1788880592.511438:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b899fe8d. 00000400:00000200:0.0:1788880592.511443:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880592.511444:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880592.511448:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880592.511456:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880592.511458:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788880592.511460:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880592.511465:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880592.511465:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000010:2.0:1788880592.511467:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:0.0:1788880592.511472:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [313:313:254:4294967295] 192.168.204.111@tcp LPNI seq info [313:313:6:4294967295] 00000400:00000200:2.0:1788880592.511477:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880592.511478:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:3.0:1788880592.511489:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880592.511492:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880592.511496:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:3.0:1788880592.511498:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b899fe8d type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880592.511500:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880592.511501:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880592.511509:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880592.511512:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880592.511516:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880592.511520:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880592.511525:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000061cfcb81 00000800:00000200:0.0:1788880592.511526:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880592.511527:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880592.511530:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000061cfcb81. 00000800:00000010:0.0:1788880592.511534:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000100:00000040:3.0:1788880592.511535:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 288558 usecs (289 jiffies) 00000100:00000001:2.0:1788880592.511536:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:0.0:1788880592.511542:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880592.511543:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880608 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880592.511543:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306223906 00000100:00000001:3.0:1788880592.511550:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306228688 00000800:00000200:0.0:1788880592.511552:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880592.511557:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880592.511563:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880592.511565:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880592.511567:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 18813 usecs (19 jiffies) 00000100:00000001:0.0:1788880592.511570:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306223636 00000100:00000040:2.0:1788880592.511571:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880608 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880592.511591:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880592.511592:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880592.511607:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880592.511609:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b899fe8d. 00000400:00000200:2.0:1788880592.511614:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880592.511617:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880592.511620:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005f2b805f 00000400:00000010:2.0:1788880592.511621:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005f2b805f. 00000100:00000001:2.0:1788880592.511624:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880592.511626:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880592.511633:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880592.511635:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880592.511641:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880592.511643:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880592.511647:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306223906 00000100:00000001:3.0:1788880592.511650:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306228688 00000100:00000001:0.0:1788880592.511652:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306228688 00000100:00000001:3.0:1788880592.511653:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306223906 00000800:00000200:2.0:1788880592.511654:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880592.511658:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000100:00000001:0.0:1788880592.511661:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306223636 00000400:00000200:2.0:1788880592.511662:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880592.511665:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306228688 00000100:00000001:0.0:1788880592.511667:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306228688 00000400:00000200:2.0:1788880592.511671:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880592.511671:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306223906 00000100:00000001:0.0:1788880592.511673:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306223636 00000400:00000200:2.0:1788880592.511677:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000100:00000001:3.0:1788880592.511677:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306228688 00000100:00000001:0.0:1788880592.511678:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306228688 00000400:00000010:2.0:1788880592.511682:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00000100:00000001:3.0:1788880592.511682:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880592.511683:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306223636 00000100:00000001:2.0:1788880592.511686:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880592.511689:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306228688 00000100:00000001:3.0:1788880592.511689:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880592.511692:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880592.511695:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880592.511695:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880592.511701:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880592.511705:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880592.511707:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880592.511711:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880592.511716:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880592.511719:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880592.511720:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880592.511731:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880592.511733:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880592.511736:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880592.511738:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880592.511740:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880592.511742:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880592.511764:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880592.511770:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880592.511771:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880592.511773:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880592.511774:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880592.511776:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880592.511778:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880592.513783:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880592.513796:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880592.513800:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880592.513803:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880592.513811:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880592.513836:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b09580 00000400:00000200:0.0:1788880592.513845:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xa8d [1] + 224 00000800:00000001:0.0:1788880592.513851:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880592.513855:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880592.513858:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880592.513862:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880592.513867:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880592.513872:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880592.513883:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880592.513899:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880592.513906:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880592.513909:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880592.513910:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880592.513912:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880592.513916:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880592.513921:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09680 00000400:00000200:0.0:1788880592.513928:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa99 [1] + 224 00000800:00000001:0.0:1788880592.513932:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880592.513937:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880592.513939:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880592.513953:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880592.513958:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880592.513964:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880592.513966:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880592.513974:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880592.513976:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880592.513990:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880592.513994:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880592.514000:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000010:1.0:1788880592.514014:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000744a548a. 00000400:00000200:1.0:1788880592.514023:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000400:00000010:1.0:1788880592.514040:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:1.0:1788880592.514046:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:0.0:1788880592.514054:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788880592.514057:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880592.514059:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:0.0:1788880592.514062:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880592.514066:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880592.514075:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880592.514084:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880592.514085:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880592.514087:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000200:1.0:1788880592.514092:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880592.514094:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000f435c455. 00000400:00000200:0.0:1788880592.514100:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880592.514105:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000115221d8 00000400:00000010:2.0:1788880592.514113:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000115221d8. 00000100:00000001:2.0:1788880592.514124:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:0.0:1788880592.514134:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09600 00000100:00000001:1.0:1788880592.514136:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:2.0:1788880592.514137:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880592.514142:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880592.514152:0:201287:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:0.0:1788880592.514153:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xa95 [1] + 224 02000000:00000001:1.0:1788880592.514161:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:0.0:1788880592.514163:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:2.0:1788880592.514163:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880592.514166:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880592.514174:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880592.514176:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880592.514177:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880592.514184:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1788880592.514185:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880592.514186:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880592.514196:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880592.514198:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788880592.514201:0:201278:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:2.0:1788880592.514202:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00001000:1.0:1788880592.514204:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=120 v=5 (5 5 5 0) 00000100:00000001:2.0:1788880592.514206:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880592.514209:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880592.514211:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880592.514216:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880592.514217:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=120 v=5 (1 1 1 0) 00000100:00001000:2.0:1788880592.514274:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=120 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880592.514278:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=77 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880592.514283:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:0.0:1788880592.514285:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880592.514286:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880592.514290:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00010000:00000001:2.0:1788880592.514291:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880592.514296:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:2.0:1788880592.514301:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880592.514305:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880592.514306:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:2.0:1788880592.514308:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880592.514313:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880592.514316:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000800:00000001:0.0:1788880592.514321:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880592.514321:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880592.514322:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:0.0:1788880592.514329:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880592.514330:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880592.514336:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880592.514337:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880592.514344:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880592.514348:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880592.514350:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880592.514358:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880592.514360:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000001:3.0:1788880592.514362:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880592.514366:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000d609d5db. 00000100:00000001:2.0:1788880592.514367:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:3.0:1788880592.514373:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f73021f1 00000400:00000010:3.0:1788880592.514378:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f73021f1. 00000100:00000001:2.0:1788880592.514380:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880592.514384:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:2.0:1788880592.514389:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880592.514394:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880592.514402:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780670592:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880592.514420:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880592.514432:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:3.0:1788880592.514436:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880592.514442:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780670592/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880592.514454:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880592.514462:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880592.514466:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880592.514481:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880592.514483:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880592.514491:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880592.514500:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880592.514506:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880592.514509:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880592.514510:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880592.514515:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880592.514516:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00100000:1.0:1788880592.514524:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780670336:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880592.514525:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880592.514526:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002f5492f5. 02000000:00000001:2.0:1788880592.514538:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880592.514540:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788880592.514542:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00001000:3.0:1788880592.514543:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=110 v=5 (5 5 5 0) 00000100:00000001:1.0:1788880592.514546:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000020:00000040:2.0:1788880592.514551:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00001000:3.0:1788880592.514552:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=110 v=5 (1 1 1 0) 00000100:00000040:1.0:1788880592.514557:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000078d5a15d x1875776780670336/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:2.0:1788880592.514562:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880592.514562:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880592.514567:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880592.514569:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 00010000:00000001:3.0:1788880592.514574:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880592.514581:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880592.514584:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880592.514590:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:2.0:1788880592.514591:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880592.514592:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000010:2.0:1788880592.514600:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000040:3.0:1788880592.514600:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:1.0:1788880592.514601:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880592.514607:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e7b82f46. 00000100:00000001:3.0:1788880592.514608:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880592.514611:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880592.514613:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880592.514614:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880592.514616:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880592.514622:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880592.514626:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:1.0:1788880592.514627:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000040:3.0:1788880592.514629:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788880592.514636:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880592.514639:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000010:1.0:1788880592.514645:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 00000100:00000001:2.0:1788880592.514645:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880592.514646:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880592.514655:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880592.514656:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880592.514657:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880592.514660:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880592.514662:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880592.514663:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880592.514664:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000078d5a15d. 00000100:00000001:1.0:1788880592.514676:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:3.0:1788880592.514676:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880592.514680:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880592.514685:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1788880592.514689:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780670464:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880592.514694:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880592.514695:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880592.514697:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780670464/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880608 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880592.514714:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880592.514715:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880592.514718:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004080df20. 02000000:00000001:3.0:1788880592.514720:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880592.514722:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880592.514723:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880592.514725:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880592.514727:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 02000000:00000001:3.0:1788880592.514730:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880592.514731:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880592.514732:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000001:3.0:1788880592.514735:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880592.514736:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880592.514737:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880592.514744:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880592.514745:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880592.514747:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880592.514750:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880592.514774:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880592.514776:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880592.514779:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880592.514781:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880592.574352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880592.574361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880592.574369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880592.574370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880592.574443:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880592.574446:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880592.574449:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880593.599084:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880593.599090:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880593.599096:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880593.599097:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880593.599160:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880593.599164:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880593.599166:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880594.622369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880594.622379:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880594.622381:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880594.622384:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880594.622388:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880594.622390:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880594.622392:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880594.622394:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880594.622397:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880594.622399:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880594.622430:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880594.622435:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880594.622437:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880595.646341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880595.646362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880595.646368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880595.646369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880595.646425:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880595.646429:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880595.646440:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880596.670343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880596.670350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880596.670355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880596.670356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880596.670380:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880596.670385:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880596.670387:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880597.630426:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306228688, at 4306228736 00000100:00000040:1.0:1788880597.630426:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306228688, at 4306228736 00000100:00000040:2.0:1788880597.630427:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306228688, at 4306228736 00000100:00000010:1.0:1788880597.630443:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:2.0:1788880597.630444:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000010:0.0:1788880597.630445:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000040:3.0:1788880597.630544:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880597.630554:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880597.630576:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788880597.630583:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880597.630590:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880597.630595:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880597.630602:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880597.630611:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000088542ca3. 00000100:00000001:3.0:1788880597.630623:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880597.630644:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880597.630660:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880597.630663:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880597.630683:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_01+1] req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880597.630721:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880597.630724:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880597.630728:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880597.630732:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880597.630735:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880597.630739:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880597.630740:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880597.630743:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880597.630746:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f92e72a2. 00000100:00000001:2.0:1788880597.630748:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880597.630751:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880597.630756:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880597.630762:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880597.630765:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880597.630768:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880597.630775:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880597.630779:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880597.630783:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_02+2] req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880597.630786:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880597.630786:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880597.630825:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880597.630826:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880597.630832:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880597.630835:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880597.630838:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880597.630842:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780670720:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880597.630844:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880597.630847:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880597.630850:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880597.630855:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880597.630856:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:2.0:1788880597.630861:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880597.630862:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 02000000:00000001:2.0:1788880597.630867:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880597.630869:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:3.0:1788880597.630869:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880597.630874:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880597.630874:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000010b36294. 00000100:00100000:0.0:1788880597.630877:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 02000000:00000001:2.0:1788880597.630880:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880597.630881:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880597.630881:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880597.630882:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880597.630885:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000010:2.0:1788880597.630888:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bcbb0958. 02000000:00000001:3.0:1788880597.630891:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880597.630896:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:2.0:1788880597.630898:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009543d7d4. 02000000:00000010:3.0:1788880597.630899:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000088d6bc1. 00000100:00000001:0.0:1788880597.630900:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880597.630907:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880597.630908:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880597.630908:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780670720, portal 25 00000100:00000001:0.0:1788880597.630913:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880597.630917:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880597.630918:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:3.0:1788880597.630922:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880597.630924:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000040:0.0:1788880597.630939:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880597.630947:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880613 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880597.630996:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880597.631005:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880597.631007:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880597.631013:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880597.631014:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880597.631020:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880597.631022:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c0c07443. 02000000:00000001:0.0:1788880597.631028:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880597.631029:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780670720, offset 0 00000100:00000001:3.0:1788880597.631029:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880597.631036:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880597.631042:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780670848:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880597.631048:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880597.631049:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_03+3] req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880597.631058:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880597.631064:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1788880597.631068:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880597.631069:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880597.631073:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880597.631075:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:0.0:1788880597.631077:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000edcbc44b. 00000100:00000001:3.0:1788880597.631077:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880597.631086:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880597.631086:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788880597.631086:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880597.631097:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880597.631100:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880597.631102:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00100000:1.0:1788880597.631111:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000040:3.0:1788880597.631111:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11566) 00000400:00000010:0.0:1788880597.631112:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dfa9de1d. 00000400:00000200:2.0:1788880597.631116:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880597.631119:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880597.631124:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000400:00000200:2.0:1788880597.631126:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [314:314:256:4294967295] 192.168.204.111@tcp LPNI seq info [314:314:8:4294967295] 00000100:00000001:1.0:1788880597.631128:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:0.0:1788880597.631130:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780670848, portal 10 00000100:00000001:1.0:1788880597.631131:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880597.631136:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880597.631137:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880597.631141:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000001:1.0:1788880597.631141:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880597.631146:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880597.631150:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880597.631154:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880597.631155:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880597.631163:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306228688, at 4306228736 00000800:00000200:2.0:1788880597.631165:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880597.631179:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000011e1ab42. 00000100:00000010:3.0:1788880597.631180:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000001:1.0:1788880597.631183:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880597.631189:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880597.631191:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:1.0:1788880597.631195:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880597.631199:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880597.631204:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880597.631206:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1788880597.631210:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00100000:1.0:1788880597.631211:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780670976:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880597.631216:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880597.631217:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000011e1ab42 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880597.631218:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:0.0:1788880597.631220:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a09430ba. 02000000:00000001:1.0:1788880597.631222:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000200:0.0:1788880597.631225:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780670848, offset 0 02000000:00000001:1.0:1788880597.631226:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880597.631230:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880597.631259:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880597.631264:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880597.631266:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306233808 00000100:00000010:2.0:1788880597.631271:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c99cc67. 00000400:00000200:0.0:1788880597.631273:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000010:1.0:1788880597.631273:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003720a685. 00000100:00000001:2.0:1788880597.631277:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 02000000:00000001:1.0:1788880597.631278:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880597.631281:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:1.0:1788880597.631284:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000744a548a. 00000100:00000001:2.0:1788880597.631284:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306233808 00000400:00000200:0.0:1788880597.631286:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:2.0:1788880597.631291:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ea9971ca. 00000400:00000010:1.0:1788880597.631292:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000100:00000001:2.0:1788880597.631296:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880597.631297:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880597.631300:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306233808 00000400:00000200:0.0:1788880597.631304:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:1.0:1788880597.631304:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780670976, portal 10 00000100:00000010:2.0:1788880597.631307:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c61ceeab. 00000100:00000001:1.0:1788880597.631309:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880597.631313:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [315:315:255:4294967295] 192.168.204.111@tcp LPNI seq info [315:315:7:4294967295] 00000100:00000001:1.0:1788880597.631313:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000001:2.0:1788880597.631313:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880597.631318:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306233808 00000100:00000010:2.0:1788880597.631323:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000043ac658c. 00000100:00000040:1.0:1788880597.631325:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880597.631329:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:0.0:1788880597.631334:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880597.631337:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880597.631343:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880597.631343:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880597.631350:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880597.631351:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f0a0057b. 00000100:00000001:1.0:1788880597.631351:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880597.631353:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880597.631359:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:1.0:1788880597.631359:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880597.631360:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880597.631364:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000010:1.0:1788880597.631365:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000038c5ac58. 00000800:00000200:0.0:1788880597.631369:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880597.631371:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:0.0:1788880597.631372:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f0a0057b type 1, nob 320 niov 1 nkiov 1 00000100:00000200:1.0:1788880597.631372:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780670976, offset 0 00000100:00000001:2.0:1788880597.631374:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880597.631377:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880597.631379:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880597.631381:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880597.631383:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 285403 usecs (286 jiffies) 00000100:00000001:0.0:1788880597.631386:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306229023 00000100:00000001:0.0:1788880597.631389:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306233808 00000100:00000001:0.0:1788880597.631392:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306229023 00000100:00000001:0.0:1788880597.631393:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306233808 00000100:00000001:0.0:1788880597.631394:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306229023 00000100:00000001:0.0:1788880597.631395:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306233808 00000100:00000001:0.0:1788880597.631397:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306229023 00000100:00000001:0.0:1788880597.631398:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306233808 00000100:00000001:0.0:1788880597.631406:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880597.631408:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880597.631412:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880597.631414:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880597.631416:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880597.631417:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880597.631421:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880597.631421:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880597.631429:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880597.631431:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000011e1ab42. 00000400:00000200:1.0:1788880597.631443:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880597.631444:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880597.631453:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880597.631461:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880597.631464:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880597.631471:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880597.631474:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880597.631476:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880597.631479:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [316:316:254:4294967295] 192.168.204.111@tcp LPNI seq info [316:316:6:4294967295] 00000400:00000200:1.0:1788880597.631486:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880597.631490:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880597.631493:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c0c07443 00000800:00000010:1.0:1788880597.631494:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000400:00000010:3.0:1788880597.631501:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c0c07443. 00000800:00000200:1.0:1788880597.631508:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880597.631514:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:1.0:1788880597.631518:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880597.631523:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000200:3.0:1788880597.631524:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880613 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880597.631533:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880597.631544:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 18643 usecs (19 jiffies) 00000100:00000001:1.0:1788880597.631555:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306228756 00000100:00000001:3.0:1788880597.631558:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880597.631559:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306233808 00000100:00000001:1.0:1788880597.631566:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306228756 00000100:00000040:3.0:1788880597.631570:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880613 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880597.631572:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306233808 00000100:00000001:1.0:1788880597.631579:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306228756 00000100:00000001:3.0:1788880597.631586:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880597.631587:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306233808 00000100:00000001:3.0:1788880597.631591:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880597.631594:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306228756 00000100:00000001:1.0:1788880597.631599:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306233808 00000100:00000001:1.0:1788880597.631601:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880597.631604:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880597.631607:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880597.631611:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880597.631614:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880597.631618:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f0a0057b. 00000400:00000200:3.0:1788880597.631623:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880597.631626:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880597.631629:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a09430ba 00000400:00000010:3.0:1788880597.631637:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a09430ba. 00000100:00000001:3.0:1788880597.631640:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880597.631643:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880597.631651:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880597.631653:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880597.631659:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880597.631660:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880597.631742:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880597.631744:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:3.0:1788880597.631748:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880597.631752:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880597.631754:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000038c5ac58 00000400:00000010:3.0:1788880597.631756:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000038c5ac58. 00000100:00000001:3.0:1788880597.631758:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880597.631761:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880597.631768:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880597.631770:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880597.631776:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880597.631777:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880597.633788:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880597.633908:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880597.633912:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880597.633915:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880597.633925:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880597.633950:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b09700 00000400:00000200:2.0:1788880597.633958:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xaa5 [1] + 224 00000800:00000001:2.0:1788880597.633964:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880597.633968:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880597.633970:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880597.633974:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880597.633981:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880597.633986:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880597.633996:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880597.634017:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880597.634024:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880597.634027:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880597.634029:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880597.634031:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880597.634034:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880597.634039:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09780 00000400:00000200:2.0:1788880597.634044:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xaad [1] + 224 00000800:00000001:2.0:1788880597.634047:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880597.634050:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880597.634051:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880597.634054:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880597.634057:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880597.634061:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880597.634070:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880597.634070:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880597.634075:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880597.634081:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880597.634084:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880597.634087:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000bcbb0958. 00000100:00000001:2.0:1788880597.634107:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:3.0:1788880597.634184:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009543d7d4 00000400:00000010:3.0:1788880597.634187:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009543d7d4. 00000100:00000001:3.0:1788880597.634191:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880597.634195:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880597.634205:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880597.634218:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880597.634220:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880597.634223:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880597.634226:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880597.634228:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880597.634229:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880597.634249:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880597.634272:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880597.634277:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880597.634284:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=125 v=5 (5 5 5 0) 00000100:00001000:3.0:1788880597.634288:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=125 v=5 (1 1 1 0) 00000800:00000001:2.0:1788880597.634292:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880597.634292:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880597.634294:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880597.634298:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788880597.634299:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880597.634301:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880597.634303:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880597.634305:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880597.634307:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880597.634312:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880597.634316:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880597.634326:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09800 00000100:00000001:3.0:1788880597.634326:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880597.634331:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880597.634334:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880597.634337:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xab5 [1] + 224 00000100:00000040:3.0:1788880597.634341:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880597.634344:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880597.634352:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1788880597.634352:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780670720:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880597.634356:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880597.634360:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880597.634363:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880597.634365:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880597.634369:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1788880597.634371:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780670720/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880597.634376:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880597.634387:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880597.634391:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:2.0:1788880597.634394:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880597.634395:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000010b36294. 02000000:00000001:3.0:1788880597.634401:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880597.634404:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880597.634407:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880597.634412:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:3.0:1788880597.634417:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000800:00000001:2.0:1788880597.634418:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000010:3.0:1788880597.634421:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000088542ca3. 00000800:00000001:2.0:1788880597.634423:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880597.634426:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880597.634429:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880597.634431:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:3.0:1788880597.634434:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880597.634435:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880597.634438:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880597.634451:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880597.634453:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880597.634455:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880597.634457:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880597.634520:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880597.634523:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880597.634525:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dfa9de1d. 00000400:00000200:0.0:1788880597.634529:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880597.634531:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788880597.634534:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880597.634538:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880597.634546:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880597.634552:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880597.634553:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880597.634556:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880597.634558:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880597.634559:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880597.634561:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880597.634563:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880597.634569:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=115 v=5 (5 5 5 0) 00000100:00001000:0.0:1788880597.634573:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=115 v=5 (1 1 1 0) 00000100:00000001:0.0:1788880597.634575:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880597.634577:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880597.634579:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:0.0:1788880597.634580:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880597.634587:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:0.0:1788880597.634588:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880597.634595:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000001:1.0:1788880597.634597:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:0.0:1788880597.634598:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:1.0:1788880597.634602:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880597.634604:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000010:1.0:1788880597.634608:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000744a548a. 00000100:00000001:0.0:1788880597.634616:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880597.634618:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000400:00000010:1.0:1788880597.634622:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000040:0.0:1788880597.634623:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880597.634631:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880597.634638:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788880597.634643:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880597.634644:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880597.634646:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880597.634661:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880597.634679:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880597.634686:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780670848:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880597.634691:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880597.634693:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880597.634696:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780670848/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880597.634729:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880597.634730:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880597.634736:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880597.634737:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880597.634741:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000010:0.0:1788880597.634742:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000edcbc44b. 02000000:00000001:1.0:1788880597.634747:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:0.0:1788880597.634748:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880597.634752:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000001:0.0:1788880597.634753:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880597.634757:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880597.634758:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880597.634765:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880597.634766:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880597.634772:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f92e72a2. 02000000:00000001:0.0:1788880597.634776:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00001000:1.0:1788880597.634778:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=125 v=5 (5 5 5 0) 02000000:00000001:0.0:1788880597.634779:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880597.634782:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00001000:1.0:1788880597.634787:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=82 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880597.634789:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880597.634793:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880597.634796:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880597.634798:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880597.634800:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880597.634805:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880597.634806:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880597.634809:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880597.634812:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00080000:1.0:1788880597.634814:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880597.634816:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880597.634846:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880597.634851:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880597.634852:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880597.634855:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880597.634856:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880597.634860:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880597.634867:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880597.634870:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880597.634872:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880597.634875:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880597.634883:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780670976:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880597.634888:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880597.634891:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880597.634893:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780670976/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880613 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880597.634900:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880597.634902:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880597.634904:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003720a685. 02000000:00000001:1.0:1788880597.634908:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880597.634911:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880597.634912:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880597.634914:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880597.634916:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000088d6bc1. 02000000:00000001:1.0:1788880597.634921:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880597.634922:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880597.634924:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:1.0:1788880597.634939:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880597.634941:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880597.634950:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880597.634956:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880597.634958:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880597.634960:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880597.634962:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880597.694698:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880597.694705:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880597.694710:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880597.694713:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880597.694715:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880597.694718:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880597.694720:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880597.694723:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880597.694724:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880597.694726:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880597.694780:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880597.694783:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880597.694801:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880598.718610:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880598.718617:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880598.718622:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880598.718623:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880598.718685:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880598.718689:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880598.718692:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880599.742385:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880599.742393:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880599.742400:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880599.742401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880599.742461:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880599.742465:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880599.742468:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880600.766339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880600.766346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880600.766352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880600.766353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880600.766421:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880600.766425:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880600.766428:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880601.790419:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880601.790424:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880601.790430:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880601.790431:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880601.791176:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880601.791184:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880601.791188:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880602.750396:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306233808, at 4306233856 00000100:00000040:2.0:1788880602.750396:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306233808, at 4306233856 00000100:00000010:0.0:1788880602.750425:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c99cc67. 00000100:00000010:2.0:1788880602.750429:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c61ceeab. 00000100:00000040:0.0:1788880602.750674:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880602.750681:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880602.750689:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b8846eac. 00000020:00000040:0.0:1788880602.750695:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880602.750700:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880602.750704:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880602.750711:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880602.750734:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b016e4f4. 00000100:00000001:0.0:1788880602.750743:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880602.750757:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880602.750769:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880602.750772:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880602.750779:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b8846eac] to pc [ptlrpcd_00_00+0] req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880602.750813:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880602.750816:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880602.750820:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880602.750823:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000802a821. 00000020:00000040:0.0:1788880602.750826:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880602.750829:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880602.750842:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880602.750844:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880602.750846:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880602.750852:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880602.750856:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880602.750861:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880602.750870:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000010:0.0:1788880602.750872:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f0a0057b. 00000100:00000001:0.0:1788880602.750877:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880602.750880:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880602.750885:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880602.750887:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880602.750891:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880602.750894:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880602.750899:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780671104:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880602.750904:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880602.750911:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880602.750914:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880602.750917:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880602.750923:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000082966425. 02000000:00000001:2.0:1788880602.750925:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880602.750932:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000048045eba. 00000400:00000010:2.0:1788880602.750941:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c013629f. 00000100:00000200:2.0:1788880602.750949:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780671104, portal 25 00000100:00000001:2.0:1788880602.750951:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880602.750953:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364096 : -126822860187520 : ffff8ca7c0781880) 00000100:00000040:2.0:1788880602.750957:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880618 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880602.750965:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880602.750968:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880602.750970:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000084794ee4. 00000100:00000200:2.0:1788880602.750973:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780671104, offset 0 00000400:00000200:2.0:1788880602.750979:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880602.750997:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880602.751004:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880602.751008:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880602.751012:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880602.751016:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880602.751019:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [317:317:256:4294967295] 192.168.204.111@tcp LPNI seq info [317:317:8:4294967295] 00000400:00000200:2.0:1788880602.751033:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880602.751039:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880602.751046:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 00000800:00000200:2.0:1788880602.751061:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880602.751070:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880602.751076:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880602.751091:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880602.751096:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306238927 00000100:00000010:2.0:1788880602.751099:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c61ceeab. 00000100:00000001:2.0:1788880602.751104:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:2.0:1788880602.751109:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306238927 00000100:00000001:2.0:1788880602.751115:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306233808 00000100:00000001:2.0:1788880602.751117:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306238927 00000100:00000001:1.0:1788880602.751118:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:2.0:1788880602.751119:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ca0fac21. 00000100:00000001:2.0:1788880602.751124:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880602.751128:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306238927 00000100:00000001:1.0:1788880602.751129:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880602.751132:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306233808 00000100:00000001:2.0:1788880602.751136:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880602.751138:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880602.751142:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880602.751144:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880602.751152:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880602.751154:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880602.751156:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880602.751157:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880602.751157:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880602.751160:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880602.751282:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880602.751284:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880602.751288:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:0.0:1788880602.751291:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000802a821] to pc [ptlrpcd_00_01+1] req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:1.0:1788880602.751294:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000400:00000200:1.0:1788880602.751305:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880602.751315:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880602.751322:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000084794ee4 00000100:00000001:0.0:1788880602.751327:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880602.751331:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880602.751336:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000400:00000010:1.0:1788880602.751339:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000084794ee4. 00000100:00000010:0.0:1788880602.751341:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006f591845. 00000020:00000040:0.0:1788880602.751346:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880602.751348:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880602.751350:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880602.751354:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880602.751357:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880602.751358:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880618 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880602.751364:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006df406c6. 00000100:00000001:0.0:1788880602.751372:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880602.751379:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880602.751381:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:1.0:1788880602.751387:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880618 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880602.751412:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880602.751415:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880602.751424:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880602.751429:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006f591845] to pc [ptlrpcd_00_02+2] req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880602.751431:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880602.751458:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880602.751461:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880602.751464:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788880602.751470:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11571) 00000100:00000001:1.0:1788880602.751484:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788880602.751489:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:1.0:1788880602.751491:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.751498:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880602.751500:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.751502:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880602.751503:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880602.751505:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880602.751511:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880602.751520:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880602.751523:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880602.751528:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880602.751532:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880602.751534:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880602.751536:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880602.751539:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880602.751542:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00100000:1.0:1788880602.751544:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006f591845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780671360:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880602.751545:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880602.751551:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880602.751552:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880602.751561:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880602.751570:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880602.751570:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880602.751575:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880602.751576:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880602.751577:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880602.751579:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880602.751584:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880602.751584:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880602.751590:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880602.751592:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880602.751594:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880602.751595:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a581a743. 02000000:00000001:2.0:1788880602.751597:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880602.751602:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880602.751606:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780671232:192.168.204.111@tcp:400: 00000400:00000010:1.0:1788880602.751612:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000744a548a. 00000100:00000001:2.0:1788880602.751613:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880602.751620:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:1.0:1788880602.751621:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 02000000:00000001:2.0:1788880602.751623:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880602.751627:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:1.0:1788880602.751629:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780671360, portal 10 02000000:00000010:2.0:1788880602.751631:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008cdf44b0. 02000000:00000001:2.0:1788880602.751634:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880602.751638:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000998db106. 00000100:00000001:1.0:1788880602.751641:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880602.751645:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849363200 : -126822860188416 : ffff8ca7c0781500) 00000400:00000010:2.0:1788880602.751645:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000032b7316b. 00000100:00000200:2.0:1788880602.751654:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780671232, portal 10 00000100:00000001:2.0:1788880602.751659:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:1.0:1788880602.751660:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880602.751662:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849370368 : -126822860181248 : ffff8ca7c0783100) 00000100:00000040:2.0:1788880602.751677:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880602.751679:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880602.751690:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880602.751696:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000974f52f4. 00000100:00000001:2.0:1788880602.751697:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:1.0:1788880602.751705:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780671360, offset 0 00000100:00000040:2.0:1788880602.751705:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880602.751711:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cc94f399. 00000400:00000200:1.0:1788880602.751724:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880602.751738:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880602.751744:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880602.751748:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880602.751752:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880602.751755:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880602.751758:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [318:318:256:4294967295] 192.168.204.111@tcp LPNI seq info [318:318:8:4294967295] 00000400:00000200:1.0:1788880602.751768:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880602.751774:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880602.751779:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000800:00000200:1.0:1788880602.751785:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880602.751792:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880602.751797:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880602.751819:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880602.751824:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 18485 usecs (19 jiffies) 00000100:00000001:1.0:1788880602.751827:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306233876 00000100:00000001:1.0:1788880602.751840:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306238927 00000100:00000001:1.0:1788880602.751842:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306233876 00000100:00000001:1.0:1788880602.751844:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306238927 00000100:00000001:1.0:1788880602.751846:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306233876 00000100:00000001:1.0:1788880602.751848:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306238927 00000100:00000001:1.0:1788880602.751856:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306233876 00000100:00000001:1.0:1788880602.751858:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306238927 00000100:00000001:1.0:1788880602.751860:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880602.751863:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880602.751866:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880602.751869:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880602.751956:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780671232, offset 0 00000400:00000200:2.0:1788880602.751967:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880602.751975:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880602.751979:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880602.751981:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880602.751984:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880602.751986:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880602.751989:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [319:319:255:4294967295] 192.168.204.111@tcp LPNI seq info [319:319:7:4294967295] 00000400:00000200:2.0:1788880602.751997:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880602.752002:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880602.752006:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000045330cf1. 00000800:00000200:2.0:1788880602.752011:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880602.752015:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880602.752024:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000045330cf1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880602.752029:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880602.752032:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 282318 usecs (283 jiffies) 00000100:00000001:2.0:1788880602.752036:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306234140 00000100:00000001:2.0:1788880602.752039:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306238927 00000100:00000001:2.0:1788880602.752040:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306234140 00000100:00000001:2.0:1788880602.752042:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306238927 00000100:00000001:2.0:1788880602.752044:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306234140 00000100:00000001:2.0:1788880602.752046:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306238927 00000100:00000001:2.0:1788880602.752048:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306234140 00000100:00000001:2.0:1788880602.752050:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306238927 00000100:00000001:2.0:1788880602.752052:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880602.752054:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880602.752056:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880602.752059:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880602.752063:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880602.752064:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880602.752066:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880602.752068:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880602.752069:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880602.752071:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880602.752148:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880602.752152:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:3.0:1788880602.752156:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880602.752161:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880602.752165:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000974f52f4 00000400:00000010:3.0:1788880602.752166:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000974f52f4. 00000100:00000001:3.0:1788880602.752169:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880602.752173:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880602.752182:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880602.752184:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880602.752191:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880602.752192:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880602.752252:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880602.752261:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000045330cf1. 00000400:00000200:3.0:1788880602.752265:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880602.752269:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880602.752272:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cc94f399 00000400:00000010:3.0:1788880602.752273:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cc94f399. 00000100:00000001:3.0:1788880602.752276:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880602.752279:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880602.752287:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880602.752355:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880602.752363:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880602.752365:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880602.753856:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880602.753875:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880602.753880:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880602.753884:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880602.753895:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880602.753908:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b09880 00000400:00000200:2.0:1788880602.753917:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xabd [1] + 224 00000800:00000001:2.0:1788880602.753925:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880602.753978:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880602.753981:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880602.753995:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880602.754003:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880602.754010:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880602.754021:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880602.754045:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880602.754053:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880602.754057:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880602.754058:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880602.754060:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880602.754065:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880602.754134:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09900 00000400:00000200:2.0:1788880602.754144:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xac9 [1] + 224 00000100:00000001:3.0:1788880602.754145:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880602.754158:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880602.754165:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880602.754168:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880602.754171:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880602.754175:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880602.754178:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880602.754187:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880602.754208:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880602.754211:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880602.754213:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.754264:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880602.754268:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880602.754274:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880602.754276:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880602.754278:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000998db106. 00000400:00000200:1.0:1788880602.754282:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000032b7316b 00000400:00000010:1.0:1788880602.754290:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000032b7316b. 00000100:00000001:1.0:1788880602.754294:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880602.754361:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880602.754372:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880602.754380:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880602.754382:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.754385:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880602.754388:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880602.754391:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880602.754394:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880602.754397:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880602.754408:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=120 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880602.754420:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=120 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880602.754424:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880602.754426:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880602.754429:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880602.754437:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880602.754438:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880602.754445:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000001:3.0:1788880602.754445:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880602.754448:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:1.0:1788880602.754449:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880602.754454:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000010:3.0:1788880602.754455:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000048045eba. 00000400:00000200:3.0:1788880602.754463:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c013629f 00000100:00000001:1.0:1788880602.754465:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880602.754468:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c013629f. 00000100:00000040:1.0:1788880602.754472:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880602.754480:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880602.754482:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880602.754483:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880602.754485:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880602.754490:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780671232:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880602.754495:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.754497:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880602.754500:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000802a821 x1875776780671232/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880602.754507:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880602.754510:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880602.754513:0:201285:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000010:1.0:1788880602.754514:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008cdf44b0. 02000000:00000001:1.0:1788880602.754523:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:3.0:1788880602.754526:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788880602.754528:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880602.754532:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880602.754539:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880602.754546:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f0a0057b. 02000000:00000001:1.0:1788880602.754556:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880602.754560:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000200:3.0:1788880602.754560:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880602.754567:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000802a821. 00000100:00000001:1.0:1788880602.754573:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880602.754579:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.754584:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.754595:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880602.754595:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880602.754599:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880602.754601:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.754603:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880602.754605:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880602.754607:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880602.754612:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880602.754624:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880602.754627:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880602.754633:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880602.754639:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=130 v=5 (5 5 5 0) 00000100:00001000:3.0:1788880602.754643:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=130 v=5 (1 1 1 0) 00000100:00000001:3.0:1788880602.754646:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880602.754660:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880602.754663:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880602.754665:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880602.754667:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880602.754672:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880602.754678:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880602.754681:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880602.754683:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880602.754686:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880602.754693:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780671104:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880602.754697:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880602.754700:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880602.754702:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b8846eac x1875776780671104/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880602.754708:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880602.754710:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880602.754712:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000082966425. 02000000:00000001:3.0:1788880602.754716:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880602.754718:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880602.754719:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880602.754722:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880602.754723:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b016e4f4. 02000000:00000001:3.0:1788880602.754727:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880602.754729:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880602.754730:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b8846eac. 00000100:00000001:3.0:1788880602.754733:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880602.754734:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880602.754736:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880602.754742:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880602.754744:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880602.754746:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880602.754748:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880602.754971:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880602.754988:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880602.754990:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880602.754992:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880602.754998:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880602.755007:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09980 00000400:00000200:0.0:1788880602.755011:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xac5 [1] + 224 00000800:00000001:0.0:1788880602.755016:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880602.755022:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880602.755024:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880602.755027:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880602.755031:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880602.755035:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880602.755055:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880602.755068:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880602.755078:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880602.755081:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.755096:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880602.755099:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880602.755103:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880602.755111:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880602.755113:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000744a548a. 00000400:00000200:1.0:1788880602.755116:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000400:00000010:1.0:1788880602.755118:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:1.0:1788880602.755121:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880602.755126:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880602.755133:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880602.755139:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880602.755141:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.755143:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880602.755145:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880602.755146:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880602.755148:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880602.755150:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880602.755155:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=130 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880602.755158:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=87 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880602.755161:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880602.755162:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880602.755164:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880602.755167:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880602.755174:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880602.755176:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880602.755176:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880602.755178:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880602.755179:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880602.755180:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880602.755185:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880602.755186:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880602.755187:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880602.755193:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880602.755198:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006f591845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780671360:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880602.755201:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.755202:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880602.755203:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006f591845 x1875776780671360/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880618 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880602.755208:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880602.755209:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880602.755210:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a581a743. 02000000:00000001:1.0:1788880602.755212:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880602.755213:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880602.755214:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880602.755216:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880602.755217:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006df406c6. 02000000:00000001:1.0:1788880602.755219:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880602.755220:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880602.755221:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006f591845. 00000100:00000001:1.0:1788880602.755223:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880602.755224:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.755225:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880602.755228:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880602.755229:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880602.755230:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880602.755249:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880602.814366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880602.814377:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880602.814384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880602.814385:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880602.814466:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880602.814473:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880602.814476:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880603.838367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880603.838374:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880603.838380:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880603.838381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880603.838500:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880603.838504:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880603.838507:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880604.862414:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880604.862424:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880604.862446:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880604.862449:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880604.862453:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880604.862456:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880604.862457:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880604.862460:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880604.862465:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880604.862466:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880604.862506:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880604.862510:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880604.862512:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880605.886874:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880605.886884:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880605.886892:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880605.886893:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880605.886978:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880605.886986:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880605.886990:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880606.910361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880606.910367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880606.910371:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880606.910372:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880606.910397:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880606.910402:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880606.910405:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880607.870312:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880607.870319:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306238927, at 4306238976 00000100:00000040:3.0:1788880607.870320:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306238927, at 4306238976 00000100:00000001:0.0:1788880607.870324:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880607.870331:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ea9971ca. 00000100:00000010:3.0:1788880607.870334:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000043ac658c. 00000100:00000010:0.0:1788880607.870335:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d4e901df. 00000020:00000040:0.0:1788880607.870342:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880607.870346:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880607.870349:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880607.870354:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880607.870361:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000004e0f6846. 00000100:00000001:0.0:1788880607.870368:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880607.870393:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880607.870403:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880607.870405:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880607.870411:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d4e901df] to pc [ptlrpcd_00_03+3] req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880607.870436:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880607.870439:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880607.870442:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880607.870445:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000b4a4221. 00000020:00000040:0.0:1788880607.870448:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880607.870450:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880607.870452:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880607.870453:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880607.870454:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880607.870457:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 00000100:00000001:0.0:1788880607.870461:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880607.870461:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880607.870465:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880607.870468:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:1.0:1788880607.870474:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880607.870486:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880607.870489:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880607.870492:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:0.0:1788880607.870497:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000b4a4221] to pc [ptlrpcd_00_00+0] req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880607.870497:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880607.870504:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880607.870508:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880607.870525:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880607.870529:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780671488:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880607.870533:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880607.870534:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880607.870537:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880607.870538:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880607.870542:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880607.870543:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1788880607.870547:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:0.0:1788880607.870548:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000363de9c9. 00000020:00000040:0.0:1788880607.870552:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000010:1.0:1788880607.870553:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002b77e71a. 00000100:00000001:0.0:1788880607.870555:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880607.870557:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880607.870558:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880607.870560:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880607.870571:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000744a548a. 02000000:00000010:0.0:1788880607.870572:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c93f60a2. 00000100:00000001:0.0:1788880607.870578:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880607.870580:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000100:00000040:0.0:1788880607.870583:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000200:1.0:1788880607.870588:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780671488, portal 25 00000100:00000001:1.0:1788880607.870593:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880607.870596:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880607.870596:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849369472 : -126822860182144 : ffff8ca7c0782d80) 00000100:00000001:0.0:1788880607.870599:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880607.870606:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880623 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880607.870607:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000363de9c9] to pc [ptlrpcd_00_01+1] req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880607.870620:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880607.870623:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880607.870628:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788880607.870635:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11576) 00000100:00000001:1.0:1788880607.870638:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880607.870644:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880607.870650:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306238927, at 4306238976 00000400:00000010:1.0:1788880607.870651:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000641230df. 00000100:00000200:1.0:1788880607.870656:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780671488, offset 0 00000100:00000010:0.0:1788880607.870658:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c61ceeab. 00000400:00000200:1.0:1788880607.870665:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880607.870678:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880607.870682:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880607.870682:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880607.870685:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788880607.870690:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:0.0:1788880607.870692:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880607.870697:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880607.870702:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880607.870707:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880607.870713:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [320:320:256:4294967295] 192.168.204.111@tcp LPNI seq info [320:320:8:4294967295] 00000100:00000001:0.0:1788880607.870722:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880607.870724:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880607.870728:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880607.870731:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880607.870746:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880607.870746:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00100000:0.0:1788880607.870754:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780671744:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880607.870760:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:1.0:1788880607.870760:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880607.870763:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880607.870766:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880607.870767:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 02000000:00000001:0.0:1788880607.870769:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880607.870774:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000356bd8da. 00000800:00000200:1.0:1788880607.870778:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:0.0:1788880607.870784:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880607.870789:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dfa9de1d. 00000800:00000200:1.0:1788880607.870791:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:0.0:1788880607.870794:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000800:00000200:1.0:1788880607.870798:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000200:0.0:1788880607.870799:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780671744, portal 10 00000100:00000001:0.0:1788880607.870805:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880607.870806:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849365888 : -126822860185728 : ffff8ca7c0781f80) 00000100:00000040:0.0:1788880607.870814:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880607.870815:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880607.870822:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306244047 00000100:00000001:0.0:1788880607.870826:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:1.0:1788880607.870829:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000100:00000040:0.0:1788880607.870830:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880607.870835:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d6cef011. 00000100:00000001:1.0:1788880607.870838:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000200:0.0:1788880607.870839:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780671744, offset 0 00000100:00000001:1.0:1788880607.870845:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306244047 00000400:00000200:0.0:1788880607.870849:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880607.870851:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000400:00000200:0.0:1788880607.870857:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880607.870857:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880607.870861:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880607.870864:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880607.870868:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880607.870871:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306244047 00000400:00000200:0.0:1788880607.870873:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880607.870879:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [321:321:255:4294967295] 192.168.204.111@tcp LPNI seq info [321:321:7:4294967295] 00000100:00000001:1.0:1788880607.870881:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306238927 00000100:00000001:1.0:1788880607.870889:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306244047 00000100:00000001:3.0:1788880607.870892:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:1.0:1788880607.870895:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000100:00000001:3.0:1788880607.870898:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880607.870899:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880607.870902:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880607.870910:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880607.870911:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880607.870917:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880607.870918:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000100:00000001:1.0:1788880607.870924:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880607.870925:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880607.870928:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880607.870935:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880607.870939:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880607.870939:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880607.870943:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880607.870945:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880607.870947:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880607.870950:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880607.870951:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 18328 usecs (19 jiffies) 00000100:00000001:1.0:1788880607.870953:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880607.870958:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306238995 00000100:00000001:1.0:1788880607.870960:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880607.870962:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880607.870967:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:2.0:1788880607.870968:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880607.870972:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880607.870979:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306244047 00000100:00000001:2.0:1788880607.870979:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880607.870986:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880607.870988:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306238995 00000400:00000200:3.0:1788880607.870988:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880607.870996:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000641230df 00000100:00000001:0.0:1788880607.871000:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306244047 00000100:00000040:2.0:1788880607.871000:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880607.871002:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000641230df. 00000100:00000001:0.0:1788880607.871009:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306238995 00000100:00000001:3.0:1788880607.871011:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880607.871020:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306244047 00000100:00000200:3.0:1788880607.871025:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880623 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880607.871031:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306238995 00000100:00000001:0.0:1788880607.871040:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306244047 00000100:00000001:2.0:1788880607.871050:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880607.871051:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880607.871057:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880607.871058:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880607.871060:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880607.871068:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880607.871070:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1788880607.871070:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880623 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880607.871072:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880607.871078:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880607.871082:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880607.871087:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000001:2.0:1788880607.871087:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880607.871097:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:2.0:1788880607.871098:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780671616:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880607.871098:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880607.871101:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880607.871108:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880607.871109:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880607.871115:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880607.871119:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880607.871126:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880607.871129:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880607.871132:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880607.871141:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f8a9d8a8. 02000000:00000001:2.0:1788880607.871150:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880607.871156:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000edc7a681. 00000400:00000010:2.0:1788880607.871164:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000170fdf47. 00000100:00000200:2.0:1788880607.871169:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780671616, portal 10 00000100:00000001:2.0:1788880607.871171:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880607.871173:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849373056 : -126822860178560 : ffff8ca7c0783b80) 00000100:00000040:2.0:1788880607.871180:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880607.871188:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:3.0:1788880607.871189:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:2.0:1788880607.871194:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880607.871196:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000400:00000010:2.0:1788880607.871200:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009c31f261. 00000400:00000200:3.0:1788880607.871204:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:2.0:1788880607.871205:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780671616, offset 0 00000400:00000200:3.0:1788880607.871213:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880607.871220:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880607.871228:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d6cef011 00000400:00000200:2.0:1788880607.871260:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:3.0:1788880607.871271:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d6cef011. 00000100:00000001:3.0:1788880607.871282:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:2.0:1788880607.871287:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880607.871294:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880607.871297:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880607.871300:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880607.871304:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [322:322:255:4294967295] 192.168.204.111@tcp LPNI seq info [322:322:7:4294967295] 00000400:00000200:2.0:1788880607.871315:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880607.871322:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880607.871327:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000eaa15d72. 00000800:00000200:2.0:1788880607.871333:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880607.871341:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880607.871345:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000eaa15d72 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880607.871350:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880607.871354:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 279292 usecs (280 jiffies) 00000100:00000200:3.0:1788880607.871360:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880607.871362:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306239257 00000100:00000001:2.0:1788880607.871368:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306244047 00000100:00000001:2.0:1788880607.871376:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306239257 00000100:00000001:2.0:1788880607.871382:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306244047 00000100:00000001:2.0:1788880607.871388:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306239257 00000100:00000001:3.0:1788880607.871393:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880607.871394:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306244047 00000100:00000001:2.0:1788880607.871400:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306239257 00000100:00000040:3.0:1788880607.871402:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880607.871405:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306244047 00000100:00000001:2.0:1788880607.871412:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880607.871424:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880607.871431:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880607.871435:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880607.871436:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880607.871439:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880607.871449:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880607.871453:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880607.871457:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880607.871467:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880607.871469:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880607.871471:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880607.871485:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880607.871496:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000eaa15d72. 00000400:00000200:3.0:1788880607.871501:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880607.871506:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880607.871510:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009c31f261 00000400:00000010:3.0:1788880607.871511:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009c31f261. 00000100:00000001:3.0:1788880607.871515:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880607.871518:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880607.871526:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880607.871528:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880607.871534:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880607.871536:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880607.873626:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880607.873669:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880607.873674:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880607.873676:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880607.873687:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880607.873702:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09b00 00000400:00000200:2.0:1788880607.873710:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xadd [1] + 224 00000800:00000001:2.0:1788880607.873718:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880607.873721:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880607.873724:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880607.873728:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880607.873746:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880607.873754:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880607.873764:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880607.873785:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880607.873794:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880607.873797:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880607.873799:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880607.873801:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880607.873804:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880607.873809:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09a80 00000400:00000200:2.0:1788880607.873814:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xae5 [1] + 224 00000800:00000001:2.0:1788880607.873817:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880607.873819:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880607.873821:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880607.873824:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880607.873826:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880607.873840:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880607.873842:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880607.873847:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880607.873852:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880607.873856:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880607.873860:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880607.873864:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dfa9de1d. 00000100:00000001:2.0:1788880607.873868:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:0.0:1788880607.873869:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880607.873873:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788880607.873877:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:2.0:1788880607.873881:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880607.873884:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880607.873888:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880607.873893:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880607.873897:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880607.873900:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880607.873902:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880607.873906:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880607.873907:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:3.0:1788880607.873911:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880607.873914:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880607.873917:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000edc7a681. 00000100:00000001:0.0:1788880607.873921:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:2.0:1788880607.873921:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b09a00 00000400:00000200:3.0:1788880607.873922:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000170fdf47 00000100:00000001:0.0:1788880607.873925:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880607.873926:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000170fdf47. 00000100:00000001:0.0:1788880607.873931:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880607.873931:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880607.873935:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xad5 [1] + 224 02000000:00000001:0.0:1788880607.873937:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000200:3.0:1788880607.873939:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880607.873941:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880607.873945:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880607.873946:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880607.873950:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880607.873953:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880607.873959:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880607.873959:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880607.873963:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880607.873966:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880607.873968:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880607.873970:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00001000:0.0:1788880607.873971:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=135 v=5 (5 5 5 0) 00000100:00000001:2.0:1788880607.873974:0:201280:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1788880607.873974:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00001000:0.0:1788880607.873977:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=92 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880607.873977:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000200:2.0:1788880607.873981:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880607.873981:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880607.873983:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000001:3.0:1788880607.873986:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880607.873987:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880607.873993:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00001000:3.0:1788880607.873994:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=125 v=5 (5 5 5 0) 00000100:00000040:2.0:1788880607.873997:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880607.874000:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=125 v=5 (1 1 1 0) 00010000:00080000:0.0:1788880607.874001:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880607.874014:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880607.874017:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880607.874021:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880607.874025:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880607.874026:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880607.874028:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880607.874030:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880607.874031:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000800:00000001:2.0:1788880607.874033:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1788880607.874034:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000040:3.0:1788880607.874035:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:2.0:1788880607.874039:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880607.874040:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880607.874040:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880607.874043:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880607.874044:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880607.874052:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880607.874053:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880607.874074:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880607.874076:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880607.874078:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880607.874080:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880607.874082:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880607.874082:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880607.874084:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880607.874088:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880607.874093:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880607.874094:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880607.874100:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880607.874109:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880607.874118:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000744a548a. 00000100:00100000:3.0:1788880607.874119:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780671616:192.168.204.111@tcp:400: 00000100:00100000:0.0:1788880607.874129:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780671744:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880607.874135:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000100:00000001:0.0:1788880607.874136:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880607.874136:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880607.874144:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:0.0:1788880607.874145:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880607.874153:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880607.874155:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1788880607.874159:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000b4a4221 x1875776780671616/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880607.874163:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000363de9c9 x1875776780671744/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880607.874168:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880607.874191:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880607.874196:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880607.874204:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f8a9d8a8. 00000100:00000001:0.0:1788880607.874206:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880607.874212:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:0.0:1788880607.874213:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000020:00000001:3.0:1788880607.874219:0:201285:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000010:0.0:1788880607.874220:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000356bd8da. 00000020:00000040:3.0:1788880607.874224:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880607.874227:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880607.874229:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000200:1.0:1788880607.874230:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880607.874266:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 00000020:00000001:0.0:1788880607.874271:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880607.874276:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:3.0:1788880607.874279:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880607.874282:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:3.0:1788880607.874283:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000020:00000001:0.0:1788880607.874288:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880607.874290:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:3.0:1788880607.874290:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000b4a4221. 02000000:00000010:0.0:1788880607.874293:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c93f60a2. 00000100:00000001:1.0:1788880607.874303:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880607.874304:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880607.874305:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:1.0:1788880607.874307:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:0.0:1788880607.874309:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880607.874309:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880607.874311:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000010:0.0:1788880607.874312:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000363de9c9. 00000100:00000001:1.0:1788880607.874317:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880607.874319:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880607.874323:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880607.874324:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880607.874324:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880607.874333:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880607.874338:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=135 v=5 (5 5 5 0) 00000100:00000001:0.0:1788880607.874339:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880607.874339:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880607.874344:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880607.874344:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880607.874348:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880607.874349:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880607.874351:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880607.874354:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880607.874367:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=135 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880607.874373:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880607.874375:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880607.874378:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880607.874380:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880607.874383:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880607.874388:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880607.874394:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880607.874397:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880607.874399:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880607.874402:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880607.874409:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780671488:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880607.874414:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880607.874416:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880607.874418:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d4e901df x1875776780671488/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880623 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880607.874424:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880607.874426:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880607.874429:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002b77e71a. 02000000:00000001:1.0:1788880607.874433:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880607.874435:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880607.874437:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880607.874439:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880607.874441:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000004e0f6846. 02000000:00000001:1.0:1788880607.874445:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880607.874446:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880607.874448:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d4e901df. 00000100:00000001:1.0:1788880607.874451:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880607.874452:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880607.874454:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880607.874460:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880607.874467:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880607.874469:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880607.874471:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880607.934369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880607.934376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880607.934383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880607.934388:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880607.934390:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880607.934393:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880607.934396:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880607.934399:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880607.934401:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880607.934403:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880607.934452:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880607.934455:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880607.934457:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880608.958350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880608.958357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880608.958363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880608.958364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880608.958389:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880608.958393:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880608.958396:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880609.982320:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880609.982327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880609.982332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880609.982333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880609.982368:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880609.982371:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880609.982373:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880611.006344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880611.006351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880611.006358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880611.006360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880611.006392:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880611.006395:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880611.006397:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880612.030395:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880612.030401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880612.030407:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880612.030408:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880612.030469:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880612.030476:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880612.030481:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880612.990346:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306244047, at 4306244096 00000100:00000040:1.0:1788880612.990347:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306244047, at 4306244096 00000100:00000040:3.0:1788880612.990347:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306244047, at 4306244096 00000100:00000040:0.0:1788880612.990362:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:1.0:1788880612.990363:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000010:3.0:1788880612.990369:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000001:0.0:1788880612.990372:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880612.990372:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ca0fac21. 00000100:00000010:0.0:1788880612.990383:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000363de9c9. 00000020:00000040:0.0:1788880612.990389:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880612.990395:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880612.990398:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880612.990403:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880612.990411:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c93f60a2. 00000100:00000001:0.0:1788880612.990419:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880612.990428:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880612.990439:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880612.990441:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880612.990447:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000363de9c9] to pc [ptlrpcd_00_02+2] req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880612.990479:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880612.990482:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880612.990485:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880612.990488:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c2a263b1. 00000020:00000040:0.0:1788880612.990490:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880612.990493:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880612.990495:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880612.990497:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880612.990500:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000ac392bfe. 00000100:00000001:0.0:1788880612.990503:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.990506:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880612.990508:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880612.990510:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788880612.990512:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:1.0:1788880612.990517:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880612.990519:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880612.990526:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880612.990528:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.990529:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880612.990533:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c2a263b1] to pc [ptlrpcd_00_03+3] req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880612.990533:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880612.990538:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880612.990541:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880612.990550:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880612.990551:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880612.990555:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880612.990556:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880612.990562:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880612.990563:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00100000:3.0:1788880612.990567:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000010:0.0:1788880612.990568:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000fd20c80. 00000100:00000001:1.0:1788880612.990572:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880612.990573:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:0.0:1788880612.990574:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880612.990576:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880612.990579:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880612.990580:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:1.0:1788880612.990582:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880612.990583:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880612.990584:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880612.990586:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880612.990588:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880612.990589:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880612.990590:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880612.990593:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:0.0:1788880612.990594:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000004e0f6846. 00000100:00100000:1.0:1788880612.990596:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780671872:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880612.990598:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.990603:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880612.990604:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880612.990607:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880612.990608:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880612.990612:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880612.990615:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880612.990618:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880612.990624:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000087acc4de. 00000100:00000001:0.0:1788880612.990628:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:1.0:1788880612.990629:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880612.990647:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880612.990649:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880612.990650:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000744a548a. 00000100:00000001:0.0:1788880612.990654:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880612.990655:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880612.990658:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788880612.990659:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 02000000:00000001:3.0:1788880612.990663:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880612.990666:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000fd20c80] to pc [ptlrpcd_00_00+0] req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880612.990667:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780671872, portal 25 00000100:00100000:3.0:1788880612.990670:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780672000:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880612.990672:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880612.990675:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849365888 : -126822860185728 : ffff8ca7c0781f80) 00000100:00000001:3.0:1788880612.990680:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880612.990685:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:1.0:1788880612.990686:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880628 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880612.990689:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880612.990694:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880612.990700:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:3.0:1788880612.990700:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000abce1aea. 02000000:00000001:3.0:1788880612.990705:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880612.990706:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880612.990710:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 00000400:00000010:3.0:1788880612.990712:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000d609d5db. 00000100:00000001:0.0:1788880612.990713:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880612.990715:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780671872, offset 0 00000100:00000040:0.0:1788880612.990718:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880612.990721:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f73021f1. 00000100:00000040:0.0:1788880612.990724:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:1.0:1788880612.990726:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:3.0:1788880612.990738:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780672000, portal 10 00000400:00000200:1.0:1788880612.990740:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880612.990743:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880612.990747:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880612.990752:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880612.990753:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364992 : -126822860186624 : ffff8ca7c0781c00) 00000400:00000200:1.0:1788880612.990758:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880612.990762:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:3.0:1788880612.990765:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880612.990770:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [323:323:256:4294967295] 192.168.204.111@tcp LPNI seq info [323:323:8:4294967295] 00000100:00000001:2.0:1788880612.990772:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880612.990776:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880612.990778:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880612.990780:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880612.990785:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880612.990788:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880612.990789:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000100:00000200:3.0:1788880612.990796:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780672000, offset 0 00000400:00000200:1.0:1788880612.990798:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:3.0:1788880612.990809:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880612.990815:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880612.990816:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880612.990819:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880612.990821:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880612.990822:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 02000000:00000001:2.0:1788880612.990824:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:3.0:1788880612.990827:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:2.0:1788880612.990828:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880612.990833:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:2.0:1788880612.990835:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880612.990835:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00100000:2.0:1788880612.990844:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780672128:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880612.990844:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:1.0:1788880612.990849:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880612.990852:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880612.990854:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:1.0:1788880612.990855:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880612.990858:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:3.0:1788880612.990861:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [324:324:255:4294967295] 192.168.204.111@tcp LPNI seq info [324:324:7:4294967295] 02000000:00000001:2.0:1788880612.990862:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880612.990866:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880612.990869:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880612.990871:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000039f224f6. 00000100:00000001:1.0:1788880612.990876:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306249167 02000000:00000001:2.0:1788880612.990877:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880612.990879:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880612.990884:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306244047 00000400:00000010:2.0:1788880612.990885:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a7a7e63a. 00000800:00000200:3.0:1788880612.990888:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880612.990893:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306249167 00000800:00000010:3.0:1788880612.990894:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000088542ca3. 00000400:00000010:2.0:1788880612.990895:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000010:1.0:1788880612.990898:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000100:00000200:2.0:1788880612.990902:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780672128, portal 10 00000800:00000200:3.0:1788880612.990903:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880612.990904:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:2.0:1788880612.990907:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880612.990910:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849357824 : -126822860193792 : ffff8ca7c0780000) 00000100:00000001:1.0:1788880612.990912:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306249167 00000800:00000200:3.0:1788880612.990914:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:1.0:1788880612.990917:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000800:00000200:3.0:1788880612.990919:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000088542ca3 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:2.0:1788880612.990922:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880612.990923:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880612.990925:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.990929:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306249167 00000100:00000040:3.0:1788880612.990931:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 276335 usecs (277 jiffies) 00000100:00000010:1.0:1788880612.990936:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000100:00000001:3.0:1788880612.990938:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306244373 00000100:00000001:1.0:1788880612.990942:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880612.990943:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306249167 00000100:00000001:2.0:1788880612.990947:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880612.990949:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880612.990949:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306244373 00000100:00000040:2.0:1788880612.990955:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880612.990955:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306249167 00000100:00000001:1.0:1788880612.990956:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880612.990961:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d208f26. 00000100:00000001:3.0:1788880612.990961:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306244373 00000100:00000001:1.0:1788880612.990962:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880612.990966:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780672128, offset 0 00000100:00000001:3.0:1788880612.990967:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306249167 00000100:00000001:1.0:1788880612.990968:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880612.990973:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306244373 00000100:00000001:1.0:1788880612.990974:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880612.990977:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880612.990978:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880612.990980:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880612.990984:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306249167 00000100:00000001:3.0:1788880612.990986:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880612.990989:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880612.990990:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880612.990994:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.990998:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880612.990998:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880612.990999:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880612.991004:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880612.991008:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880612.991019:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880612.991022:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [325:325:254:4294967295] 192.168.204.111@tcp LPNI seq info [325:325:6:4294967295] 00000400:00000200:2.0:1788880612.991032:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880612.991038:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880612.991042:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e53a8f34. 00000800:00000200:2.0:1788880612.991046:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880612.991051:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880612.991053:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e53a8f34 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880612.991057:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880612.991060:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 18171 usecs (19 jiffies) 00000100:00000001:2.0:1788880612.991063:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306244115 00000800:00000200:1.0:1788880612.991064:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880612.991066:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306249167 00000800:00000010:1.0:1788880612.991068:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:2.0:1788880612.991070:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306244115 00000400:00000200:1.0:1788880612.991075:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880612.991075:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306249167 00000100:00000001:2.0:1788880612.991080:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306244115 00000400:00000200:1.0:1788880612.991083:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880612.991085:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306249167 00000400:00000200:1.0:1788880612.991090:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000400:00000010:1.0:1788880612.991098:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:2.0:1788880612.991100:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306244115 00000100:00000001:1.0:1788880612.991105:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880612.991105:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306249167 00000100:00000001:2.0:1788880612.991110:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788880612.991112:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880628 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880612.991114:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880612.991119:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880612.991123:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880612.991127:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880612.991129:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788880612.991132:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880628 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880612.991133:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880612.991140:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880612.991142:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880612.991144:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880612.991146:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.991147:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880612.991149:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880612.991161:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880612.991163:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000088542ca3. 00000400:00000200:1.0:1788880612.991168:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880612.991172:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880612.991174:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000010:1.0:1788880612.991176:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:1.0:1788880612.991178:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880612.991180:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880612.991186:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880612.991188:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880612.991192:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11581) 00000100:00000001:1.0:1788880612.991193:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.991198:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880612.991259:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880612.991262:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880612.991278:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880612.991282:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e53a8f34. 00000400:00000200:1.0:1788880612.991285:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880612.991288:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880612.991290:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d208f26 00000400:00000010:1.0:1788880612.991292:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d208f26. 00000100:00000001:1.0:1788880612.991294:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880612.991296:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880612.991303:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880612.991311:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880612.991317:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.991319:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880612.992910:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880612.992920:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880612.992923:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880612.992926:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880612.992933:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880612.992944:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b09b80 00000400:00000200:3.0:1788880612.992951:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xaed [1] + 224 00000800:00000001:3.0:1788880612.992957:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880612.992962:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880612.992965:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880612.992968:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880612.992972:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880612.992976:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880612.992985:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880612.993120:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880612.993123:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880612.993138:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000001:1.0:1788880612.993151:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788880612.993152:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880612.993155:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:3.0:1788880612.993159:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880612.993160:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000744a548a. 00000800:00000001:3.0:1788880612.993164:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000200:1.0:1788880612.993165:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000400:00000010:1.0:1788880612.993168:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000800:00000001:3.0:1788880612.993168:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880612.993172:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880612.993177:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880612.993179:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000200:1.0:1788880612.993189:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880612.993193:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880612.993194:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880612.993196:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880612.993197:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880612.993199:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880612.993200:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880612.993202:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880612.993206:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=140 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880612.993209:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=140 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880612.993211:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880612.993212:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880612.993214:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880612.993216:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.993218:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880612.993220:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880612.993229:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09c80 00000100:00000001:1.0:1788880612.993259:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880612.993265:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880612.993267:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880612.993268:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xafd [1] + 224 00000800:00000001:3.0:1788880612.993273:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788880612.993281:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880612.993291:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780671872:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880612.993295:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880612.993297:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880612.993298:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000363de9c9 x1875776780671872/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880612.993303:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880612.993305:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880612.993307:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000087acc4de. 02000000:00000001:1.0:1788880612.993309:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880612.993310:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880612.993312:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880612.993313:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880612.993315:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c93f60a2. 02000000:00000001:1.0:1788880612.993318:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880612.993320:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880612.993321:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000363de9c9. 00000100:00000001:1.0:1788880612.993325:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880612.993326:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880612.993327:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880612.993331:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880612.993332:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.993333:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880612.993335:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880612.993336:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880612.993344:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880612.993347:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880612.993350:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880612.993352:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880612.993358:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880612.993369:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880612.993373:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880612.993377:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880612.993379:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880612.993381:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880612.993392:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880612.993399:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880612.993401:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880612.993402:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880612.993403:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a7a7e63a. 00000400:00000200:2.0:1788880612.993405:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880612.993406:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880612.993408:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880612.993411:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880612.993416:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880612.993423:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880612.993425:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880612.993425:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880612.993429:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880612.993432:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880612.993433:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880612.993434:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880612.993435:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880612.993438:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09c00 00000100:00001000:2.0:1788880612.993439:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=140 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880612.993444:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=97 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880612.993447:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:3.0:1788880612.993448:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xaf5 [1] + 224 00000100:00000001:2.0:1788880612.993449:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880612.993452:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880612.993457:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880612.993457:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:2.0:1788880612.993463:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880612.993465:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880612.993466:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000200:3.0:1788880612.993469:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:2.0:1788880612.993470:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880612.993473:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880612.993475:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880612.993479:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:2.0:1788880612.993481:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880612.993485:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880612.993486:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880612.993488:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:3.0:1788880612.993491:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880612.993494:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880612.993501:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880612.993512:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780672128:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880612.993512:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880612.993518:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880612.993522:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880612.993525:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000fd20c80 x1875776780672128/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880612.993535:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880612.993536:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880612.993537:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000039f224f6. 02000000:00000001:2.0:1788880612.993539:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880612.993540:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880612.993541:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880612.993543:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880612.993543:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:2.0:1788880612.993545:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000004e0f6846. 02000000:00000001:2.0:1788880612.993549:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000800:00000001:3.0:1788880612.993550:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:2.0:1788880612.993551:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880612.993554:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000fd20c80. 00000800:00000001:3.0:1788880612.993556:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880612.993557:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880612.993560:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880612.993563:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880612.993566:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880612.993568:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.993570:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880612.993570:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880612.993571:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.993572:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880612.993576:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880612.993579:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880612.993580:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000d609d5db. 00000400:00000200:1.0:1788880612.993583:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f73021f1 00000400:00000010:1.0:1788880612.993585:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f73021f1. 00000100:00000001:1.0:1788880612.993587:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880612.993591:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880612.993608:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880612.993612:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880612.993613:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880612.993614:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880612.993615:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880612.993616:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880612.993617:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880612.993618:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880612.993627:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=130 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880612.993629:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=130 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880612.993630:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880612.993631:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880612.993632:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880612.993643:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.993645:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880612.993646:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880612.993647:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880612.993648:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880612.993651:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880612.993657:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880612.993659:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880612.993660:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880612.993663:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880612.993670:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780672000:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880612.993674:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880612.993675:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880612.993677:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c2a263b1 x1875776780672000/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880628 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880612.993694:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880612.993695:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880612.993696:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000abce1aea. 02000000:00000001:1.0:1788880612.993698:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880612.993699:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880612.993699:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880612.993701:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880612.993701:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000ac392bfe. 02000000:00000001:1.0:1788880612.993703:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880612.993704:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880612.993704:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c2a263b1. 00000100:00000001:1.0:1788880612.993706:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880612.993707:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880612.993708:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880612.993710:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880612.993711:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880612.993712:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880612.993713:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880613.054345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880613.054351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880613.054356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880613.054357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880613.054378:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880613.054382:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880613.054384:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880614.078345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880614.078352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880614.078357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880614.078358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880614.078417:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880614.078422:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880614.078425:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880615.102351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880615.102362:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880615.102364:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880615.102367:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880615.102372:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880615.102375:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880615.102377:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880615.102379:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880615.102383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880615.102385:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880615.102440:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880615.102444:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880615.102448:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880616.126341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880616.126347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880616.126353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880616.126355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880616.126418:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880616.126422:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880616.126424:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880617.150582:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880617.150591:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880617.150598:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880617.150600:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880617.150743:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880617.150748:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880617.150751:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880618.110368:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306249167, at 4306249216 00000100:00000010:1.0:1788880618.110389:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000040:3.0:1788880618.110389:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306249167, at 4306249216 00000100:00000040:2.0:1788880618.110417:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306249167, at 4306249216 00000100:00000010:3.0:1788880618.110431:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:2.0:1788880618.110434:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000040:1.0:1788880618.110551:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880618.110558:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880618.110566:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001cf64abe. 00000020:00000040:1.0:1788880618.110572:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880618.110577:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880618.110580:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880618.110586:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880618.110594:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:1.0:1788880618.110603:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880618.110617:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880618.110627:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880618.110630:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880618.110637:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001cf64abe] to pc [ptlrpcd_00_01+1] req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880618.110658:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880618.110660:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880618.110662:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880618.110668:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000070db75b2. 00000020:00000040:1.0:1788880618.110670:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880618.110672:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880618.110673:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880618.110680:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880618.110681:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880618.110686:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:1.0:1788880618.110687:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:2.0:1788880618.110690:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880618.110692:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880618.110697:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:2.0:1788880618.110702:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880618.110739:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880618.110744:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880618.110745:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306249167, at 4306249216 00000100:00000001:2.0:1788880618.110748:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880618.110753:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880618.110754:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000070db75b2] to pc [ptlrpcd_00_02+2] req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880618.110759:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 02000000:00000001:2.0:1788880618.110763:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880618.110766:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880618.110769:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880618.110776:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880618.110782:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880618.110798:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780672256:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880618.110803:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880618.110808:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880618.110811:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880618.110811:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880618.110825:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880618.110828:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880618.110829:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880618.110838:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000010:2.0:1788880618.110839:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e7b82f46. 00000100:00000001:0.0:1788880618.110842:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:1.0:1788880618.110843:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 02000000:00000001:2.0:1788880618.110844:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880618.110847:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:1.0:1788880618.110850:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880618.110855:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:2.0:1788880618.110856:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a7a7e63a. 02000000:00000001:1.0:1788880618.110858:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:0.0:1788880618.110863:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880618.110864:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880618.110866:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 02000000:00000010:1.0:1788880618.110870:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:1.0:1788880618.110876:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880618.110879:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780672256, portal 25 00000100:00000040:1.0:1788880618.110887:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880618.110894:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880618.110899:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880618.110899:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938054144 : -126823771497472 : ffff8ca78a269c00) 00000100:00000001:0.0:1788880618.110904:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880618.110913:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880618.110913:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880618.110916:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880618.110919:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880618.110919:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880634 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880618.110920:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880618.110922:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880618.110933:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880618.110934:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000008b25f66] to pc [ptlrpcd_00_03+3] req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880618.110946:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780672384:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880618.110971:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880618.110978:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880618.110982:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880618.110983:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880618.110984:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880618.110986:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880618.110988:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880618.110997:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000010:0.0:1788880618.111003:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000056ba8425. 00000100:00000040:2.0:1788880618.111003:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880618.111010:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11586) 02000000:00000001:0.0:1788880618.111013:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880618.111016:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000793778c8. 00000100:00000001:3.0:1788880618.111017:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880618.111021:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:0.0:1788880618.111023:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dfa9de1d. 00000100:00000200:2.0:1788880618.111027:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780672256, offset 0 00000400:00000010:0.0:1788880618.111033:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000400:00000200:2.0:1788880618.111045:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:0.0:1788880618.111053:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780672384, portal 10 00000100:00000001:0.0:1788880618.111057:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880618.111060:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938046976 : -126823771504640 : ffff8ca78a268000) 00000400:00000200:2.0:1788880618.111061:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880618.111072:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:0.0:1788880618.111073:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880618.111082:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880618.111093:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:2.0:1788880618.111094:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:0.0:1788880618.111099:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880618.111103:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 00000100:00000200:0.0:1788880618.111107:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780672384, offset 0 00000400:00000200:2.0:1788880618.111115:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880618.111121:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880618.111126:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [326:326:256:4294967295] 192.168.204.111@tcp LPNI seq info [326:326:8:4294967295] 00000400:00000200:2.0:1788880618.111140:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880618.111147:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880618.111152:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000400:00000200:0.0:1788880618.111157:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:2.0:1788880618.111164:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880618.111166:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880618.111174:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:0.0:1788880618.111178:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880618.111178:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:3.0:1788880618.111188:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880618.111190:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880618.111194:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880618.111208:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880618.111214:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880618.111217:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880618.111221:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880618.111223:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [327:327:255:4294967295] 192.168.204.111@tcp LPNI seq info [327:327:7:4294967295] 00000100:00000001:2.0:1788880618.111224:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306254287 02000000:00000001:3.0:1788880618.111227:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880618.111286:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:2.0:1788880618.111288:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ca0fac21. 02000000:00000001:3.0:1788880618.111294:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880618.111296:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00100000:3.0:1788880618.111303:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780672512:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880618.111311:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306254288 00000100:00000001:3.0:1788880618.111312:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880618.111317:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:3.0:1788880618.111318:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:2.0:1788880618.111320:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001e508d7a. 02000000:00000001:3.0:1788880618.111326:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880618.111331:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 02000000:00000001:3.0:1788880618.111334:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:0.0:1788880618.111339:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880618.111341:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306254288 02000000:00000010:3.0:1788880618.111343:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004080df20. 02000000:00000001:3.0:1788880618.111350:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880618.111351:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000da26fd2f. 00000100:00000010:2.0:1788880618.111352:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ce680e22. 00000100:00000001:2.0:1788880618.111361:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000010:3.0:1788880618.111364:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000e9a6e97c. 00000100:00000001:2.0:1788880618.111366:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306254288 00000800:00000200:0.0:1788880618.111370:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:2.0:1788880618.111375:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb1a14f4. 00000400:00000010:3.0:1788880618.111377:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007fe20b6f. 00000100:00000001:2.0:1788880618.111380:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000200:3.0:1788880618.111383:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780672512, portal 10 00000100:00000001:2.0:1788880618.111388:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880618.111389:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880618.111392:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:0.0:1788880618.111401:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000da26fd2f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880618.111401:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880618.111401:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000001:2.0:1788880618.111412:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880618.111414:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880618.111416:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880618.111416:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880618.111423:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 273436 usecs (274 jiffies) 00000100:00000001:2.0:1788880618.111426:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:1.0:1788880618.111428:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880618.111430:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880618.111435:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306249491 00000100:00000001:2.0:1788880618.111440:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880618.111440:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000010:1.0:1788880618.111441:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000100:00000001:0.0:1788880618.111447:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306254287 00000100:00000001:2.0:1788880618.111447:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880618.111451:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880618.111453:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880618.111457:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880618.111459:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:3.0:1788880618.111461:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8336eb2. 00000100:00000001:0.0:1788880618.111477:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306249491 00000100:00000200:3.0:1788880618.111477:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780672512, offset 0 00000400:00000200:1.0:1788880618.111479:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880618.111486:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306254288 00000100:00000001:0.0:1788880618.111491:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306249491 00000400:00000200:1.0:1788880618.111494:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000793778c8 00000100:00000001:0.0:1788880618.111495:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306254288 00000400:00000010:1.0:1788880618.111499:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000793778c8. 00000100:00000001:0.0:1788880618.111500:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306249491 00000100:00000001:0.0:1788880618.111505:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306254288 00000100:00000001:0.0:1788880618.111507:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880618.111508:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880618.111512:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880618.111517:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:1.0:1788880618.111519:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880634 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880618.111522:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880618.111528:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880618.111555:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880618.111560:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:3.0:1788880618.111560:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880618.111565:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880618.111571:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880618.111575:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880618.111576:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880618.111579:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880618.111580:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880634 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880618.111589:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880618.111590:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880618.111602:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880618.111608:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880618.111609:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880618.111616:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:3.0:1788880618.111621:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [328:328:254:4294967295] 192.168.204.111@tcp LPNI seq info [328:328:6:4294967295] 00000400:00000200:3.0:1788880618.111632:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880618.111642:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880618.111647:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000088542ca3. 00000800:00000200:3.0:1788880618.111653:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880618.111658:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:3.0:1788880618.111659:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000010:1.0:1788880618.111664:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000da26fd2f. 00000800:00000200:3.0:1788880618.111666:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000088542ca3 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880618.111673:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880618.111674:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880618.111685:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 18008 usecs (19 jiffies) 00000400:00000200:1.0:1788880618.111691:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880618.111694:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306249236 00000400:00000200:1.0:1788880618.111697:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000100:00000001:3.0:1788880618.111699:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306254287 00000400:00000010:1.0:1788880618.111701:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000001:3.0:1788880618.111705:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306249236 00000100:00000001:1.0:1788880618.111708:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880618.111710:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306254288 00000100:00000200:1.0:1788880618.111714:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880618.111716:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306249236 00000100:00000001:3.0:1788880618.111721:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306254288 00000100:00000001:3.0:1788880618.111729:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306249236 00000100:00000001:3.0:1788880618.111735:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306254288 00000100:00000001:3.0:1788880618.111744:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880618.111753:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880618.111757:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880618.111758:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880618.111763:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880618.111766:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880618.111770:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880618.111774:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880618.111780:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880618.111784:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880618.111785:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880618.111789:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880618.111791:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880618.111794:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880618.111818:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880618.111822:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000088542ca3. 00000400:00000200:1.0:1788880618.111827:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880618.111831:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880618.111833:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8336eb2 00000400:00000010:1.0:1788880618.111842:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8336eb2. 00000100:00000001:1.0:1788880618.111846:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880618.111850:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880618.111858:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880618.111860:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880618.111866:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880618.111868:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880618.114076:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880618.114088:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880618.114093:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880618.114096:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880618.114104:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880618.114115:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09d80 00000400:00000200:1.0:1788880618.114124:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb0d [1] + 224 00000800:00000001:1.0:1788880618.114130:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880618.114146:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880618.114149:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880618.114153:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880618.114160:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880618.114165:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880618.114174:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880618.114200:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880618.114208:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880618.114211:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880618.114285:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880618.114289:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880618.114295:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880618.114297:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880618.114299:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dfa9de1d. 00000400:00000200:0.0:1788880618.114304:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880618.114319:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788880618.114322:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880618.114327:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880618.114335:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880618.114341:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880618.114342:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880618.114344:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880618.114347:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880618.114357:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880618.114359:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880618.114362:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880618.114370:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=136 v=5 (5 5 5 0) 00000100:00001000:0.0:1788880618.114374:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=136 v=5 (1 1 1 0) 00000100:00000001:0.0:1788880618.114376:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880618.114378:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880618.114380:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880618.114383:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880618.114386:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880618.114388:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880618.114390:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880618.114392:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880618.114396:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880618.114402:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880618.114404:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880618.114406:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880618.114409:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880618.114415:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780672384:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880618.114420:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880618.114422:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880618.114425:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000070db75b2 x1875776780672384/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880618.114431:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880618.114434:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880618.114436:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000056ba8425. 02000000:00000001:0.0:1788880618.114439:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880618.114441:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880618.114442:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880618.114445:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880618.114447:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:0.0:1788880618.114455:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880618.114457:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880618.114458:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000070db75b2. 00000100:00000001:0.0:1788880618.114461:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880618.114462:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880618.114464:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880618.114468:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880618.114470:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880618.114472:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880618.114474:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880618.114622:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880618.114628:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880618.114630:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880618.114632:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880618.114636:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880618.114643:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09e00 00000400:00000200:3.0:1788880618.114648:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb15 [1] + 224 00000800:00000001:3.0:1788880618.114651:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880618.114654:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880618.114655:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880618.114658:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880618.114662:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880618.114665:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880618.114674:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880618.114689:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880618.114694:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880618.114697:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880618.114699:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880618.114701:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880618.114704:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880618.114709:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b09d00 00000400:00000200:3.0:1788880618.114713:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xb05 [1] + 224 00000800:00000001:3.0:1788880618.114717:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880618.114722:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880618.114723:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880618.114726:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880618.114730:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880618.114731:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880618.114734:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880618.114738:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880618.114742:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000200:3.0:1788880618.114742:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880618.114751:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880618.114752:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000e9a6e97c. 00000400:00000200:1.0:1788880618.114758:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007fe20b6f 00000400:00000010:1.0:1788880618.114761:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007fe20b6f. 00000100:00000001:1.0:1788880618.114766:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880618.114770:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880618.114779:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880618.114779:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880618.114786:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880618.114788:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880618.114790:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880618.114791:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880618.114796:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880618.114798:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880618.114799:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880618.114801:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880618.114803:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880618.114807:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=146 v=5 (5 5 5 0) 00000100:00001000:1.0:1788880618.114810:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=103 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880618.114813:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880618.114814:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880618.114816:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880618.114819:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880618.114825:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880618.114827:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880618.114829:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880618.114830:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880618.114831:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880618.114834:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880618.114839:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880618.114841:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880618.114842:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880618.114844:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880618.114856:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780672512:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880618.114860:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880618.114862:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880618.114864:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780672512/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880618.114870:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880618.114871:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880618.114872:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004080df20. 02000000:00000001:1.0:1788880618.114875:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880618.114876:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880618.114877:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880618.114879:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880618.114880:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:1.0:1788880618.114882:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880618.114883:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880618.114884:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:1.0:1788880618.114887:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880618.114888:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880618.114889:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880618.114892:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880618.114894:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880618.114895:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880618.114897:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880618.114943:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880618.114946:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880618.114949:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880618.114951:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880618.114952:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a7a7e63a. 00000400:00000200:2.0:1788880618.114955:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880618.114957:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880618.114959:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880618.114963:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880618.114970:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880618.114976:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880618.114977:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880618.114978:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880618.114980:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880618.114981:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880618.114982:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880618.114984:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880618.114986:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=146 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880618.114989:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=146 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880618.114991:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880618.114992:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880618.114994:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880618.114995:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880618.114997:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880618.114999:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880618.115017:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880618.115018:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880618.115019:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880618.115022:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880618.115028:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780672256:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880618.115038:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880618.115040:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880618.115042:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000001cf64abe x1875776780672256/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880634 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880618.115048:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880618.115049:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880618.115051:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e7b82f46. 02000000:00000001:2.0:1788880618.115053:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880618.115055:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880618.115056:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880618.115058:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880618.115059:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 02000000:00000001:2.0:1788880618.115061:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880618.115062:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880618.115063:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000001cf64abe. 00000100:00000001:2.0:1788880618.115066:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880618.115066:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880618.115068:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880618.115071:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880618.115072:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880618.115074:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880618.115075:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880618.174337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880618.174345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880618.174350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880618.174353:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880618.174355:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880618.174358:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880618.174360:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880618.174363:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880618.174364:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880618.174365:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880618.174413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880618.174418:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880618.174421:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880619.198351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880619.198375:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880619.198383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880619.198385:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880619.198437:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880619.198440:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880619.198442:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880620.222384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880620.222393:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880620.222401:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880620.222402:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880620.222474:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880620.222491:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880620.222495:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880621.246408:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880621.246416:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880621.246423:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880621.246424:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880621.246562:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880621.246566:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880621.246569:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880622.271275:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880622.271281:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880622.271287:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880622.271289:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880622.271322:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880622.271325:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880622.271327:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880623.230330:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306254287, at 4306254336 00000100:00000040:1.0:1788880623.230330:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306254288, at 4306254336 00000100:00000040:2.0:1788880623.230330:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306254288, at 4306254336 00000100:00000010:1.0:1788880623.230348:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001e508d7a. 00000100:00000010:0.0:1788880623.230350:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ca0fac21. 00000100:00000010:2.0:1788880623.230352:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ce680e22. 00000100:00000040:3.0:1788880623.230418:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880623.230424:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880623.230432:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788880623.230437:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880623.230452:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880623.230455:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880623.230459:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880623.230467:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000088d6bc1. 00000100:00000001:3.0:1788880623.230473:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880623.230482:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880623.230504:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880623.230509:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880623.230515:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_00+0] req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880623.230543:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880623.230546:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880623.230549:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880623.230564:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000100:00000001:2.0:1788880623.230567:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:3.0:1788880623.230568:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880623.230572:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880623.230572:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880623.230573:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880623.230575:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880623.230576:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880623.230577:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880623.230580:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880623.230581:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880623.230586:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f92e72a2. 00000100:00000001:3.0:1788880623.230592:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880623.230599:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880623.230604:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880623.230606:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880623.230612:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880623.230615:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788880623.230616:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880623.230620:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880623.230620:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880623.230627:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780672640:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880623.230630:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_01+1] req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880623.230631:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880623.230636:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880623.230638:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880623.230642:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880623.230646:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000033d66d82. 02000000:00000001:2.0:1788880623.230648:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880623.230653:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a7a7e63a. 00000100:00000001:3.0:1788880623.230654:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880623.230658:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000040:3.0:1788880623.230658:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880623.230664:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000200:2.0:1788880623.230665:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780672640, portal 25 00000100:00000001:0.0:1788880623.230666:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880623.230668:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000010:3.0:1788880623.230668:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000100:00000001:0.0:1788880623.230669:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880623.230671:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000001:0.0:1788880623.230672:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:3.0:1788880623.230673:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:2.0:1788880623.230677:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880639 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880623.230678:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:0.0:1788880623.230679:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880623.230681:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880623.230684:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880623.230690:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b899fe8d. 00000100:00000001:3.0:1788880623.230696:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880623.230704:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880623.230705:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788880623.230705:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880623.230708:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880623.230712:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880623.230712:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880623.230715:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880623.230718:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c56cc487. 02000000:00000001:0.0:1788880623.230720:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880623.230725:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780672640, offset 0 00000100:00100000:0.0:1788880623.230727:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780672768:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880623.230728:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880623.230731:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880623.230733:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880623.230736:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880623.230737:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880623.230740:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880623.230741:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_02+2] req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880623.230743:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880623.230747:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003a17641d. 00000400:00000200:2.0:1788880623.230748:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880623.230752:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880623.230757:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:0.0:1788880623.230758:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dfa9de1d. 00000400:00000200:2.0:1788880623.230763:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:0.0:1788880623.230769:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000400:00000200:2.0:1788880623.230769:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880623.230773:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:0.0:1788880623.230774:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780672768, portal 10 00000100:00000001:0.0:1788880623.230776:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880623.230778:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000400:00000200:2.0:1788880623.230780:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [329:329:256:4294967295] 192.168.204.111@tcp LPNI seq info [329:329:8:4294967295] 00000100:00000001:3.0:1788880623.230781:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880623.230786:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880623.230786:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880623.230793:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880623.230798:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880623.230799:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880623.230803:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880623.230804:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788880623.230804:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:0.0:1788880623.230805:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000100:00000001:1.0:1788880623.230807:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:0.0:1788880623.230812:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780672768, offset 0 00000100:00000040:3.0:1788880623.230813:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11592) 00000100:00000040:1.0:1788880623.230816:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880623.230827:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880623.230827:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880623.230832:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788880623.230833:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306254288, at 4306254336 00000100:00000001:1.0:1788880623.230836:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880623.230837:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000400:00000200:0.0:1788880623.230838:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788880623.230842:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000010:3.0:1788880623.230842:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb1a14f4. 02000000:00000001:1.0:1788880623.230845:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880623.230846:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880623.230851:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:1.0:1788880623.230852:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880623.230853:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00100000:1.0:1788880623.230858:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780672896:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880623.230859:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880623.230864:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880623.230866:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880623.230867:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:2.0:1788880623.230867:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880623.230869:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880623.230872:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880623.230874:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [330:330:255:4294967295] 192.168.204.111@tcp LPNI seq info [330:330:7:4294967295] 00000800:00000200:2.0:1788880623.230875:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 320 niov 1 nkiov 1 02000000:00000001:1.0:1788880623.230876:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880623.230880:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880623.230886:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e84cf675. 02000000:00000001:1.0:1788880623.230889:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880623.230892:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880623.230893:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:1.0:1788880623.230894:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000744a548a. 00000100:00000001:2.0:1788880623.230898:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306259407 00000800:00000200:0.0:1788880623.230899:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880623.230906:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000400:00000010:1.0:1788880623.230907:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000800:00000200:0.0:1788880623.230912:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:1.0:1788880623.230913:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780672896, portal 10 00000100:00000010:2.0:1788880623.230917:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ce680e22. 00000800:00000200:0.0:1788880623.230918:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880623.230919:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880623.230922:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000800:00000200:0.0:1788880623.230923:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880623.230924:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880623.230930:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880623.230932:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306259407 00000100:00000040:1.0:1788880623.230933:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880623.230934:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 270605 usecs (271 jiffies) 00000800:00000200:3.0:1788880623.230935:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880623.230940:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306254607 00000100:00000010:2.0:1788880623.230940:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000059817523. 00000800:00000010:3.0:1788880623.230941:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000100:00000001:0.0:1788880623.230946:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306259407 00000400:00000200:3.0:1788880623.230948:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880623.230951:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880623.230952:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306254607 00000100:00000001:1.0:1788880623.230955:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788880623.230961:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880623.230962:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306259407 00000100:00000001:0.0:1788880623.230964:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306259407 00000100:00000040:1.0:1788880623.230965:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880623.230971:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000780e164. 00000400:00000010:1.0:1788880623.230972:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 00000400:00000200:3.0:1788880623.230972:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c56cc487 00000400:00000010:3.0:1788880623.230977:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c56cc487. 00000100:00000001:2.0:1788880623.230978:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880623.230984:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306259407 00000100:00000001:3.0:1788880623.230985:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880623.230990:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780672896, offset 0 00000100:00000001:0.0:1788880623.230991:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306254607 00000100:00000010:2.0:1788880623.230991:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003053b935. 00000100:00000200:3.0:1788880623.230992:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880639 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880623.230998:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306259407 00000100:00000001:2.0:1788880623.230998:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788880623.231005:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880623.231006:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306254607 00000100:00000001:2.0:1788880623.231006:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880623.231012:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306259407 00000100:00000001:2.0:1788880623.231013:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880623.231015:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880623.231016:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880623.231016:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880623.231021:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880623.231021:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880623.231023:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880639 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880623.231024:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880623.231028:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880623.231029:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880623.231034:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880623.231040:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880623.231040:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880623.231041:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880623.231041:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880623.231045:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:1.0:1788880623.231046:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880623.231046:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880623.231047:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880623.231049:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880623.231052:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880623.231052:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880623.231055:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880623.231056:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880623.231056:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880623.231061:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880623.231062:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880623.231062:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [331:331:254:4294967295] 192.168.204.111@tcp LPNI seq info [331:331:6:4294967295] 00000100:00000001:2.0:1788880623.231069:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880623.231078:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880623.231084:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880623.231088:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000800:00000200:1.0:1788880623.231092:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880623.231097:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880623.231100:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880623.231102:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880623.231105:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 17865 usecs (18 jiffies) 00000100:00000001:1.0:1788880623.231108:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306254354 00000100:00000001:1.0:1788880623.231110:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306259407 00000100:00000001:1.0:1788880623.231112:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306254354 00000100:00000001:1.0:1788880623.231114:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306259407 00000100:00000001:1.0:1788880623.231123:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306254354 00000100:00000001:1.0:1788880623.231125:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306259407 00000100:00000001:1.0:1788880623.231127:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306254354 00000100:00000001:1.0:1788880623.231128:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306259407 00000100:00000001:1.0:1788880623.231130:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880623.231132:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880623.231134:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880623.231136:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880623.231140:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880623.231142:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:3.0:1788880623.231142:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880623.231148:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:3.0:1788880623.231148:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000100:00000001:1.0:1788880623.231151:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880623.231154:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880623.231155:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880623.231157:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880623.231162:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880623.231166:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000400:00000010:3.0:1788880623.231168:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:3.0:1788880623.231171:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880623.231175:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880623.231182:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880623.231184:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880623.231209:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880623.231211:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880623.231319:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880623.231322:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:3.0:1788880623.231324:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880623.231326:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880623.231328:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000400:00000010:3.0:1788880623.231330:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:3.0:1788880623.231332:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880623.231335:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880623.231342:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880623.231344:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880623.231349:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880623.231350:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880623.232756:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880623.232767:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880623.232770:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880623.232772:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880623.232779:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880623.232788:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b09e80 00000400:00000200:1.0:1788880623.232794:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xb1d [1] + 224 00000800:00000001:1.0:1788880623.232799:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880623.232804:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880623.232807:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880623.232810:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880623.232814:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880623.232818:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880623.232826:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880623.232848:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880623.232860:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880623.232862:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880623.232880:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880623.232883:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880623.232889:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880623.232890:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880623.232898:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a7a7e63a. 00000400:00000200:2.0:1788880623.232901:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880623.232903:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880623.232906:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880623.232910:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880623.232917:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880623.232924:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880623.232925:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880623.232928:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880623.232930:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880623.232932:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880623.232934:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880623.232936:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880623.232941:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=151 v=5 (5 5 5 0) 00000100:00001000:2.0:1788880623.232945:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=151 v=5 (1 1 1 0) 00000100:00000001:2.0:1788880623.232947:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880623.232948:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880623.232951:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880623.232953:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880623.232956:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880623.232960:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880623.232966:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880623.232969:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880623.232970:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880623.232973:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880623.232980:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780672640:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880623.232984:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880623.232986:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880623.232988:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780672640/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880623.232999:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880623.233001:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880623.233004:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000033d66d82. 02000000:00000001:2.0:1788880623.233007:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880623.233008:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880623.233010:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880623.233012:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880623.233014:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000088d6bc1. 02000000:00000001:2.0:1788880623.233017:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880623.233019:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880623.233020:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:2.0:1788880623.233023:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880623.233024:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880623.233026:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880623.233030:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880623.233031:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880623.233033:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880623.233034:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880623.233185:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880623.233189:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880623.233191:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880623.233192:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880623.233195:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880623.233201:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09f00 00000400:00000200:3.0:1788880623.233205:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb25 [1] + 224 00000800:00000001:3.0:1788880623.233208:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880623.233209:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880623.233210:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880623.233212:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880623.233215:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880623.233217:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880623.233222:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880623.233255:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880623.233259:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880623.233262:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880623.233264:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880623.233265:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880623.233268:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880623.233273:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b09f80 00000100:00000001:0.0:1788880623.233278:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880623.233278:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb2d [1] + 224 00000100:00000001:0.0:1788880623.233282:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880623.233285:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880623.233287:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880623.233291:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:3.0:1788880623.233292:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880623.233293:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dfa9de1d. 00000400:00000200:3.0:1788880623.233297:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880623.233299:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880623.233303:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000400:00000200:3.0:1788880623.233304:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880623.233309:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880623.233311:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880623.233316:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880623.233317:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880623.233341:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880623.233343:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880623.233353:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880623.233356:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880623.233358:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880623.233361:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880623.233362:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880623.233364:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880623.233365:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880623.233369:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=141 v=5 (5 5 5 0) 00000100:00000001:3.0:1788880623.233370:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880623.233376:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880623.233377:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=141 v=5 (1 1 1 0) 00000100:00000001:0.0:1788880623.233380:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880623.233382:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880623.233382:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880623.233386:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880623.233389:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880623.233391:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880623.233393:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880623.233393:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880623.233394:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880623.233397:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880623.233398:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880623.233401:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880623.233404:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880623.233408:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000001:1.0:1788880623.233408:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880623.233412:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000744a548a. 00000100:00000001:0.0:1788880623.233413:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880623.233415:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880623.233418:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000100:00000040:0.0:1788880623.233420:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880623.233421:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:1.0:1788880623.233425:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880623.233430:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880623.233436:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780672768:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880623.233455:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880623.233457:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880623.233461:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880623.233467:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780672768/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880623.233469:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880623.233472:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880623.233476:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880623.233478:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880623.233480:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880623.233481:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:0.0:1788880623.233483:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880623.233485:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880623.233487:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003a17641d. 02000000:00000001:1.0:1788880623.233489:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880623.233492:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:1.0:1788880623.233493:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=151 v=5 (5 5 5 0) 00000020:00000001:0.0:1788880623.233494:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880623.233497:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00001000:1.0:1788880623.233498:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=108 v=5 (1 0 0 0) 00000020:00000001:0.0:1788880623.233501:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880623.233502:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000010:0.0:1788880623.233504:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f92e72a2. 00000100:00000001:1.0:1788880623.233505:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880623.233508:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:0.0:1788880623.233509:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880623.233511:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00010000:00080000:1.0:1788880623.233513:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880623.233514:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:0.0:1788880623.233520:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00010000:00000001:1.0:1788880623.233522:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880623.233524:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880623.233524:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880623.233527:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880623.233528:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880623.233528:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880623.233534:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880623.233535:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880623.233541:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880623.233543:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880623.233545:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880623.233548:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880623.233555:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880623.233556:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880623.233557:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880623.233560:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880623.233564:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780672896:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880623.233567:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880623.233568:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880623.233569:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780672896/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880639 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880623.233573:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880623.233574:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880623.233575:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e84cf675. 02000000:00000001:1.0:1788880623.233577:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880623.233578:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880623.233578:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880623.233580:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880623.233581:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b899fe8d. 02000000:00000001:1.0:1788880623.233582:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880623.233583:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880623.233584:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:1.0:1788880623.233586:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880623.233586:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880623.233587:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880623.233590:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880623.233591:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880623.233592:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880623.233593:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880623.294347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880623.294355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880623.294362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880623.294363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880623.294453:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880623.294457:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880623.294459:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880624.318349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880624.318361:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880624.318363:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880624.318367:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880624.318373:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880624.318376:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880624.318379:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880624.318381:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880624.318387:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880624.318389:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880624.318459:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880624.318462:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880624.318465:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880625.342312:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880625.342317:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880625.342322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880625.342323:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880625.342391:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880625.342395:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880625.342410:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880626.366428:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880626.366439:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880626.366448:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880626.366449:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880626.366740:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880626.366746:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880626.366749:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880627.390435:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880627.390443:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880627.390450:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880627.390454:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880627.390456:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880627.390460:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880627.390463:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880627.390466:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880627.390468:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880627.390471:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880627.390569:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880627.390572:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880627.390575:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880628.350390:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306259407, at 4306259456 00000100:00000040:2.0:1788880628.350390:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306259407, at 4306259456 00000100:00000040:3.0:1788880628.350395:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306259407, at 4306259456 00000100:00000010:1.0:1788880628.350415:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000059817523. 00000100:00000010:2.0:1788880628.350417:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000780e164. 00000100:00000040:0.0:1788880628.350480:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880628.350486:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880628.350497:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cdd8dec9. 00000020:00000040:0.0:1788880628.350503:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880628.350507:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880628.350510:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880628.350516:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880628.350524:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006df406c6. 00000100:00000001:0.0:1788880628.350532:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880628.350545:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880628.350556:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880628.350559:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880628.350565:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cdd8dec9] to pc [ptlrpcd_00_03+3] req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880628.350596:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880628.350599:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880628.350603:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880628.350605:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:0.0:1788880628.350608:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c2a263b1. 00000100:00000001:1.0:1788880628.350610:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:0.0:1788880628.350612:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880628.350614:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880628.350616:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880628.350619:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:1.0:1788880628.350620:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880628.350620:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880628.350623:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880628.350626:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880628.350630:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b016e4f4. 00000100:00000001:1.0:1788880628.350637:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880628.350638:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880628.350639:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880628.350645:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880628.350660:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880628.350663:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880628.350665:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880628.350668:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cdd8dec9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780673024:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880628.350672:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880628.350680:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880628.350681:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:1.0:1788880628.350683:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880628.350684:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880628.350687:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880628.350692:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c2a263b1] to pc [ptlrpcd_00_00+0] req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880628.350693:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cd0104b. 02000000:00000001:1.0:1788880628.350698:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880628.350708:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000744a548a. 00000400:00000010:1.0:1788880628.350718:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000100:00000001:0.0:1788880628.350724:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880628.350725:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780673024, portal 25 00000100:00000040:0.0:1788880628.350728:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880628.350730:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880628.350733:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849372160 : -126822860179456 : ffff8ca7c0783800) 00000100:00000001:0.0:1788880628.350735:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1788880628.350739:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880628.350741:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880644 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880628.350742:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000fd20c80. 00000100:00000001:2.0:1788880628.350744:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:0.0:1788880628.350748:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880628.350748:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880628.350754:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880628.350757:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880628.350761:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880628.350763:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880628.350766:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000010:1.0:1788880628.350767:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 02000000:00000001:0.0:1788880628.350770:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880628.350774:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780673024, offset 0 02000000:00000010:0.0:1788880628.350777:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f0a0057b. 00000100:00000001:2.0:1788880628.350782:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880628.350784:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880628.350785:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880628.350789:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880628.350793:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880628.350797:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880628.350801:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880628.350804:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880628.350807:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:2.0:1788880628.350810:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780673152:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880628.350811:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880628.350815:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880628.350816:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:0.0:1788880628.350820:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000fd20c80] to pc [ptlrpcd_00_01+1] req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880628.350820:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880628.350822:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880628.350825:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:2.0:1788880628.350826:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880628.350830:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000006c03106. 00000400:00000200:1.0:1788880628.350835:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:2.0:1788880628.350835:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880628.350839:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880628.350840:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a7a7e63a. 00000100:00000040:0.0:1788880628.350844:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:1.0:1788880628.350844:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:2.0:1788880628.350847:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000200:2.0:1788880628.350851:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780673152, portal 10 00000400:00000200:1.0:1788880628.350852:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:0.0:1788880628.350854:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880628.350856:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880628.350858:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364992 : -126822860186624 : ffff8ca7c0781c00) 00000400:00000200:1.0:1788880628.350863:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [332:332:256:4294967295] 192.168.204.111@tcp LPNI seq info [332:332:8:4294967295] 00000100:00000040:0.0:1788880628.350864:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11597) 00000100:00000040:2.0:1788880628.350868:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880628.350874:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880628.350877:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880628.350879:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000005509e00. 00000100:00000001:0.0:1788880628.350882:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880628.350883:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780673152, offset 0 00000100:00000001:0.0:1788880628.350888:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880628.350888:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880628.350894:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880628.350894:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880628.350903:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880628.350903:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880628.350907:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880628.350911:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000010:1.0:1788880628.350912:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000040:0.0:1788880628.350913:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880628.350915:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880628.350920:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880628.350926:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [333:333:255:4294967295] 192.168.204.111@tcp LPNI seq info [333:333:7:4294967295] 00000800:00000200:1.0:1788880628.350928:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880628.350940:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880628.350950:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880628.350950:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880628.350957:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e53a8f34. 00000800:00000200:1.0:1788880628.350959:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880628.350964:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000200:2.0:1788880628.350965:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880628.350967:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880628.350971:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880628.350975:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880628.350978:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880628.350978:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880628.350980:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00100000:0.0:1788880628.350986:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780673280:192.168.204.111@tcp:400: 00000800:00000200:2.0:1788880628.350986:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e53a8f34 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880628.350987:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306264527 00000100:00000001:2.0:1788880628.350993:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880628.350994:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880628.350999:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880628.351001:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 267821 usecs (268 jiffies) 02000000:00000001:0.0:1788880628.351002:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880628.351002:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306259407 02000000:00000001:0.0:1788880628.351007:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880628.351008:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306259724 00000100:00000001:1.0:1788880628.351010:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306264527 02000000:00000010:0.0:1788880628.351012:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008d264e32. 00000100:00000001:2.0:1788880628.351014:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306264527 02000000:00000001:0.0:1788880628.351017:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880628.351018:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000100:00000001:2.0:1788880628.351020:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306259724 00000400:00000010:0.0:1788880628.351024:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dfa9de1d. 00000100:00000001:1.0:1788880628.351027:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000010:0.0:1788880628.351030:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000001:1.0:1788880628.351032:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306264527 00000100:00000001:2.0:1788880628.351033:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306264527 00000100:00000200:0.0:1788880628.351035:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780673280, portal 10 00000100:00000010:1.0:1788880628.351039:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000100:00000001:2.0:1788880628.351039:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306259724 00000100:00000001:0.0:1788880628.351040:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880628.351043:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849357824 : -126822860193792 : ffff8ca7c0780000) 00000100:00000001:1.0:1788880628.351045:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880628.351051:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306264527 00000100:00000010:1.0:1788880628.351055:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000100:00000001:2.0:1788880628.351055:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306264527 00000100:00000001:2.0:1788880628.351060:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306259724 00000100:00000001:1.0:1788880628.351062:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880628.351066:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880628.351067:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306264527 00000100:00000010:3.0:1788880628.351070:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003053b935. 00000100:00000001:1.0:1788880628.351071:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880628.351072:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880628.351077:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880628.351077:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880628.351083:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880628.351083:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880628.351088:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880628.351094:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880628.351095:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880628.351097:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880628.351098:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880628.351102:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880628.351103:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880628.351105:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880628.351106:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880628.351110:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880628.351110:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880628.351116:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880628.351116:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880628.351138:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880628.351142:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:3.0:1788880628.351147:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880628.351153:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880628.351158:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:3.0:1788880628.351160:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:3.0:1788880628.351166:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880628.351170:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880644 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880628.351180:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880628.351183:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880644 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880628.351190:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880628.351191:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880628.351293:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880628.351296:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e53a8f34. 00000400:00000200:3.0:1788880628.351299:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880628.351303:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880628.351305:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000005509e00 00000400:00000010:3.0:1788880628.351307:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000005509e00. 00000100:00000001:3.0:1788880628.351310:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880628.351312:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880628.351319:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880628.351321:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880628.351327:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880628.351329:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000040:0.0:1788880628.351392:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880628.351399:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880628.351401:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880628.351404:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 00000100:00000200:0.0:1788880628.351409:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780673280, offset 0 00000400:00000200:0.0:1788880628.351413:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880628.351418:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880628.351421:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880628.351423:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880628.351425:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880628.351427:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880628.351429:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [334:334:256:4294967295] 192.168.204.111@tcp LPNI seq info [334:334:8:4294967295] 00000400:00000200:0.0:1788880628.351436:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880628.351446:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880628.351449:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000800:00000200:0.0:1788880628.351453:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880628.351456:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880628.351459:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880628.351469:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880628.351473:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 17710 usecs (18 jiffies) 00000100:00000001:0.0:1788880628.351475:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306259475 00000100:00000001:0.0:1788880628.351477:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306264527 00000100:00000001:0.0:1788880628.351479:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306259475 00000100:00000001:0.0:1788880628.351481:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306264527 00000100:00000001:0.0:1788880628.351483:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306259475 00000100:00000001:0.0:1788880628.351485:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306264527 00000100:00000001:0.0:1788880628.351488:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306259475 00000100:00000001:0.0:1788880628.351490:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306264527 00000100:00000001:0.0:1788880628.351492:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880628.351494:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880628.351496:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880628.351498:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880628.351503:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880628.351504:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880628.351507:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880628.351508:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880628.351510:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880628.351511:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880628.351544:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880628.351548:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000400:00000200:1.0:1788880628.351552:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880628.351556:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880628.351559:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000400:00000010:1.0:1788880628.351561:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000001:1.0:1788880628.351564:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880628.351567:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880628.351581:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880628.351583:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880628.351589:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880628.351591:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880628.353248:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880628.353298:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880628.353302:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880628.353305:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880628.353314:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880628.353326:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0a000 00000400:00000200:3.0:1788880628.353359:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xb35 [1] + 224 00000800:00000001:3.0:1788880628.353367:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880628.353372:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880628.353374:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880628.353378:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880628.353384:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880628.353390:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880628.353400:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880628.353418:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880628.353426:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880628.353429:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880628.353432:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880628.353433:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880628.353439:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880628.353446:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a080 00000100:00000001:1.0:1788880628.353449:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880628.353453:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880628.353453:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb3d [1] + 224 00000800:00000001:3.0:1788880628.353460:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880628.353461:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880628.353463:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:3.0:1788880628.353464:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880628.353465:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000744a548a. 00000400:00000200:3.0:1788880628.353469:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880628.353471:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000400:00000010:1.0:1788880628.353474:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000400:00000200:3.0:1788880628.353475:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880628.353477:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880628.353480:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880628.353483:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880628.353488:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880628.353497:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880628.353507:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880628.353510:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880628.353512:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880628.353518:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880628.353522:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880628.353527:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880628.353529:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880628.353530:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880628.353532:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880628.353536:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:1.0:1788880628.353541:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=6 v=5 (5 5 5 5) 00000800:00000001:3.0:1788880628.353542:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880628.353547:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=6 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880628.353551:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880628.353553:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880628.353555:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:1.0:1788880628.353557:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880628.353560:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880628.353560:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880628.353566:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880628.353566:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880628.353571:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880628.353572:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000040:1.0:1788880628.353574:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788880628.353577:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880628.353581:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880628.353584:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a7a7e63a. 00000400:00000200:3.0:1788880628.353586:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a100 00000400:00000200:2.0:1788880628.353589:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880628.353594:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880628.353599:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880628.353600:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880628.353604:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:3.0:1788880628.353606:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb45 [1] + 224 00000100:00000001:1.0:1788880628.353607:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880628.353607:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880628.353615:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880628.353617:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880628.353625:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880628.353629:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00100000:1.0:1788880628.353637:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cdd8dec9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780673024:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880628.353640:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880628.353648:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880628.353649:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880628.353653:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:3.0:1788880628.353657:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880628.353659:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cdd8dec9 x1875776780673024/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880628.353663:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880628.353684:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880628.353691:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880628.353692:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880628.353695:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880628.353695:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880628.353700:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000010:1.0:1788880628.353702:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cd0104b. 02000000:00000001:2.0:1788880628.353704:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880628.353706:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880628.353709:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880628.353711:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:2.0:1788880628.353712:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880628.353715:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880628.353718:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00001000:2.0:1788880628.353719:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=146 v=5 (5 5 5 0) 00000100:00000001:3.0:1788880628.353720:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:1.0:1788880628.353724:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00001000:2.0:1788880628.353726:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=146 v=5 (1 1 1 0) 02000000:00000010:1.0:1788880628.353728:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006df406c6. 00000100:00000001:2.0:1788880628.353730:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:3.0:1788880628.353733:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:1.0:1788880628.353734:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880628.353734:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880628.353739:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00010000:00000001:2.0:1788880628.353739:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:3.0:1788880628.353740:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:1.0:1788880628.353743:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cdd8dec9. 00010000:00000001:2.0:1788880628.353743:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880628.353750:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880628.353750:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:1.0:1788880628.353753:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880628.353753:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880628.353758:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880628.353759:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880628.353760:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880628.353762:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880628.353764:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880628.353764:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880628.353768:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880628.353768:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880628.353770:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880628.353772:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000001:0.0:1788880628.353774:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880628.353775:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880628.353778:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dfa9de1d. 00000400:00000200:0.0:1788880628.353786:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000100:00000001:2.0:1788880628.353793:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880628.353796:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:2.0:1788880628.353797:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880628.353800:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880628.353802:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:2.0:1788880628.353805:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880628.353810:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880628.353823:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780673152:192.168.204.111@tcp:400: 00000100:00000200:0.0:1788880628.353829:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880628.353831:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880628.353835:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880628.353840:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c2a263b1 x1875776780673152/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880628.353841:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880628.353844:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880628.353849:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880628.353851:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880628.353853:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880628.353854:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880628.353856:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880628.353862:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=6 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880628.353867:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00001000:0.0:1788880628.353868:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=113 v=5 (1 0 0 0) 02000000:00000001:2.0:1788880628.353870:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880628.353873:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000010:2.0:1788880628.353874:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000006c03106. 00000100:00000001:0.0:1788880628.353876:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880628.353879:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880628.353883:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880628.353883:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880628.353887:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880628.353889:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880628.353893:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:0.0:1788880628.353895:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880628.353896:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b016e4f4. 00000100:00000001:0.0:1788880628.353899:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:2.0:1788880628.353901:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:0.0:1788880628.353902:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:2.0:1788880628.353904:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880628.353906:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000010:2.0:1788880628.353907:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c2a263b1. 00000100:00000001:0.0:1788880628.353908:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880628.353912:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:0.0:1788880628.353913:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880628.353914:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880628.353919:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880628.353925:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880628.353926:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880628.353928:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880628.353929:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880628.353931:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880628.353932:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880628.353935:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880628.353938:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880628.353957:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780673280:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880628.353961:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880628.353963:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880628.353966:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000fd20c80 x1875776780673280/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880644 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880628.353971:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880628.353973:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880628.353975:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008d264e32. 02000000:00000001:0.0:1788880628.353978:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880628.353979:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880628.353981:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880628.353982:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880628.353984:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f0a0057b. 02000000:00000001:0.0:1788880628.353986:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880628.353987:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880628.353988:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000fd20c80. 00000100:00000001:0.0:1788880628.353990:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880628.353991:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880628.353993:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880628.353997:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880628.353998:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880628.353999:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880628.354001:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880628.414358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880628.414369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880628.414378:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880628.414379:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880628.414433:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880628.414438:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880628.414442:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880629.438329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880629.438349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880629.438355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880629.438356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880629.438417:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880629.438421:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880629.438424:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880630.463322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880630.463329:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880630.463335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880630.463337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880630.463372:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880630.463376:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880630.463380:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880631.486331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880631.486339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880631.486351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880631.486353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880631.486403:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880631.486408:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880631.486411:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880632.510344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880632.510350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880632.510355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880632.510357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880632.510377:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880632.510382:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880632.510385:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880633.470354:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306264527, at 4306264576 00000100:00000040:1.0:1788880633.470354:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306264527, at 4306264576 00000100:00000040:2.0:1788880633.470354:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306264527, at 4306264576 00000100:00000040:3.0:1788880633.470370:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:0.0:1788880633.470372:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ce680e22. 00000100:00000010:1.0:1788880633.470377:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:2.0:1788880633.470379:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000001:3.0:1788880633.470379:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880633.470389:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880633.470395:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880633.470400:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880633.470403:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880633.470408:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880633.470426:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000446e4ede. 00000100:00000001:3.0:1788880633.470433:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880633.470456:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880633.470466:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880633.470469:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880633.470475:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_02+2] req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880633.470498:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880633.470501:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880633.470504:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880633.470507:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880633.470509:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880633.470512:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880633.470513:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880633.470516:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880633.470519:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880633.470520:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b899fe8d. 00000100:00000001:2.0:1788880633.470525:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880633.470525:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880633.470529:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880633.470533:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880633.470533:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880633.470538:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880633.470539:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880633.470556:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880633.470559:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880633.470559:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880633.470562:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880633.470565:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_03+3] req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880633.470568:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880633.470571:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880633.470576:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880633.470584:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780673408:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880633.470589:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880633.470593:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880633.470599:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880633.470600:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880633.470605:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 02000000:00000001:2.0:1788880633.470607:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000020:00000040:3.0:1788880633.470608:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880633.470609:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880633.470610:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880633.470613:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880633.470614:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880633.470615:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880633.470617:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880633.470617:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000010:2.0:1788880633.470621:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000006c62949. 02000000:00000001:3.0:1788880633.470621:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880633.470625:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880633.470626:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880633.470633:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a7a7e63a. 02000000:00000010:3.0:1788880633.470634:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f92e72a2. 00000100:00000001:1.0:1788880633.470641:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880633.470641:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880633.470644:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880633.470644:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000040:3.0:1788880633.470649:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880633.470650:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:2.0:1788880633.470653:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780673408, portal 25 02000000:00000001:1.0:1788880633.470654:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880633.470659:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1788880633.470661:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880633.470663:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000100:00100000:1.0:1788880633.470669:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780673536:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880633.470671:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:2.0:1788880633.470674:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880649 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880633.470674:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880633.470677:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880633.470682:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880633.470683:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_00+0] req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880633.470686:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880633.470690:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880633.470696:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cd0104b. 00000100:00000001:2.0:1788880633.470696:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880633.470699:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880633.470711:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000744a548a. 00000100:00000040:2.0:1788880633.470715:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880633.470717:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000400:00000010:2.0:1788880633.470719:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000841b943f. 00000100:00000200:1.0:1788880633.470722:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780673536, portal 10 00000100:00000001:3.0:1788880633.470722:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880633.470724:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780673408, offset 0 00000100:00000001:1.0:1788880633.470726:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:3.0:1788880633.470727:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880633.470730:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000040:3.0:1788880633.470736:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:2.0:1788880633.470737:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880633.470742:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880633.470745:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11602) 00000100:00000001:0.0:1788880633.470753:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880633.470755:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880633.470757:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880633.470762:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880633.470762:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306264527, at 4306264576 00000100:00000001:0.0:1788880633.470768:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880633.470768:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:3.0:1788880633.470771:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000400:00000200:2.0:1788880633.470772:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:0.0:1788880633.470777:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880633.470777:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880633.470782:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:1.0:1788880633.470785:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000400:00000200:2.0:1788880633.470789:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:1.0:1788880633.470792:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780673536, offset 0 00000100:00000001:3.0:1788880633.470797:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880633.470799:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [335:335:256:4294967295] 192.168.204.111@tcp LPNI seq info [335:335:8:4294967295] 00000100:00000001:0.0:1788880633.470800:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880633.470801:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880633.470803:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880633.470809:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880633.470810:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880633.470813:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880633.470817:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880633.470824:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780673664:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880633.470825:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880633.470832:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880633.470836:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880633.470839:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880633.470840:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:2.0:1788880633.470840:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880633.470844:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880633.470847:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:0.0:1788880633.470850:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007f19b4dd. 00000400:00000200:1.0:1788880633.470851:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000010:2.0:1788880633.470851:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000eaa15d72. 00000400:00000200:1.0:1788880633.470855:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1788880633.470856:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880633.470860:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:0.0:1788880633.470862:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dfa9de1d. 00000800:00000200:2.0:1788880633.470864:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880633.470867:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [336:336:255:4294967295] 192.168.204.111@tcp LPNI seq info [336:336:7:4294967295] 00000400:00000010:0.0:1788880633.470869:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000200:0.0:1788880633.470874:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780673664, portal 10 00000800:00000200:2.0:1788880633.470878:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880633.470879:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880633.470880:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880633.470884:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000eaa15d72 type 1, nob 320 niov 1 nkiov 1 00000800:00000200:1.0:1788880633.470888:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880633.470892:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000800:00000010:1.0:1788880633.470894:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000800:00000200:1.0:1788880633.470899:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:0.0:1788880633.470902:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880633.470902:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880633.470905:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880633.470909:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880633.470916:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880633.470916:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306269647 00000100:00000001:0.0:1788880633.470918:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880633.470921:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 265098 usecs (266 jiffies) 00000100:00000010:2.0:1788880633.470924:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000780e164. 00000100:00000001:1.0:1788880633.470925:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306264842 00000100:00000040:0.0:1788880633.470926:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880633.470931:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000010:0.0:1788880633.470932:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000033f4d237. 00000100:00000200:0.0:1788880633.470938:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780673664, offset 0 00000100:00000001:1.0:1788880633.470940:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306269647 00000100:00000001:2.0:1788880633.470940:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306269647 00000400:00000200:0.0:1788880633.470947:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880633.470947:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306264842 00000100:00000010:2.0:1788880633.470948:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb1a14f4. 00000100:00000001:2.0:1788880633.470954:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880633.470958:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880633.470959:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306269647 00000100:00000001:2.0:1788880633.470959:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306269647 00000100:00000010:2.0:1788880633.470964:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b63100a0. 00000800:00000200:3.0:1788880633.470965:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:0.0:1788880633.470966:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880633.470970:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306264842 00000100:00000001:2.0:1788880633.470970:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000010:3.0:1788880633.470972:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000eaa15d72. 00000100:00000001:2.0:1788880633.470976:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306269647 00000100:00000001:1.0:1788880633.470977:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306269647 00000400:00000200:0.0:1788880633.470979:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880633.470979:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880633.470982:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306264842 00000100:00000010:2.0:1788880633.470984:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000488d1158. 00000400:00000200:0.0:1788880633.470986:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880633.470991:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788880633.470991:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880633.470993:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880633.470999:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306269647 00000100:00000001:2.0:1788880633.470999:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880633.471001:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [337:337:254:4294967295] 192.168.204.111@tcp LPNI seq info [337:337:6:4294967295] 00000100:00000001:1.0:1788880633.471004:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880633.471005:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880633.471008:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880633.471011:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880633.471014:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880633.471017:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880633.471018:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880633.471026:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880633.471028:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880633.471028:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880633.471032:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880633.471032:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:0.0:1788880633.471033:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880633.471034:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000841b943f 00000100:00000001:1.0:1788880633.471039:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880633.471039:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:0.0:1788880633.471041:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f0a0057b. 00000100:00000001:1.0:1788880633.471042:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880633.471042:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880633.471046:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880633.471047:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880633.471048:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000841b943f. 00000800:00000200:0.0:1788880633.471050:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880633.471054:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880633.471054:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880633.471057:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:0.0:1788880633.471059:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880633.471063:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f0a0057b type 1, nob 320 niov 1 nkiov 1 00000100:00000200:3.0:1788880633.471065:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880649 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880633.471068:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880633.471074:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 17561 usecs (18 jiffies) 00000100:00000001:0.0:1788880633.471079:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306264594 00000100:00000001:3.0:1788880633.471082:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880633.471083:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306269647 00000100:00000001:0.0:1788880633.471087:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306264594 00000100:00000040:3.0:1788880633.471088:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880649 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880633.471090:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306269647 00000100:00000001:0.0:1788880633.471095:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306264594 00000100:00000001:0.0:1788880633.471099:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306269647 00000100:00000001:0.0:1788880633.471102:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306264594 00000100:00000001:0.0:1788880633.471107:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306269647 00000100:00000001:3.0:1788880633.471107:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880633.471109:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880633.471112:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880633.471113:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880633.471116:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880633.471117:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880633.471120:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880633.471123:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880633.471125:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880633.471127:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880633.471128:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880633.471130:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880633.471136:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880633.471139:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:3.0:1788880633.471142:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880633.471147:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880633.471150:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000400:00000010:3.0:1788880633.471152:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:3.0:1788880633.471159:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880633.471162:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880633.471170:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880633.471172:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880633.471178:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880633.471179:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880633.471231:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880633.471265:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f0a0057b. 00000400:00000200:3.0:1788880633.471269:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880633.471272:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880633.471275:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000033f4d237 00000400:00000010:3.0:1788880633.471276:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000033f4d237. 00000100:00000001:3.0:1788880633.471279:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880633.471282:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880633.471289:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880633.471291:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880633.471297:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880633.471298:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880633.473652:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880633.473662:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880633.473666:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880633.473669:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880633.473676:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880633.473686:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0a180 00000400:00000200:1.0:1788880633.473693:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xb4d [1] + 224 00000800:00000001:1.0:1788880633.473699:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880633.473702:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880633.473704:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880633.473709:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880633.473722:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880633.473726:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880633.473736:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880633.473757:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880633.473773:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880633.473777:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880633.473779:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880633.473782:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880633.473787:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880633.473790:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880633.473795:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a200 00000100:00000001:2.0:1788880633.473795:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880633.473803:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb51 [1] + 224 00000800:00000001:1.0:1788880633.473807:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880633.473810:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880633.473811:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:1.0:1788880633.473812:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:2.0:1788880633.473815:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:1.0:1788880633.473818:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:2.0:1788880633.473819:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a7a7e63a. 00000400:00000200:2.0:1788880633.473825:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880633.473826:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880633.473830:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880633.473831:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880633.473836:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880633.473838:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880633.473859:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880633.473862:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880633.473884:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880633.473884:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880633.473888:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880633.473890:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880633.473891:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880633.473895:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:1.0:1788880633.473896:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880633.473898:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:1.0:1788880633.473899:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880633.473902:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880633.473903:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880633.473906:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880633.473912:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:2.0:1788880633.473915:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=11 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880633.473918:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880633.473921:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a280 00000100:00001000:2.0:1788880633.473923:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=11 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880633.473923:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880633.473928:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb5d [1] + 224 00000100:00000001:2.0:1788880633.473928:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880633.473929:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880633.473934:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:1.0:1788880633.473936:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:2.0:1788880633.473937:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880633.473937:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880633.473941:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880633.473941:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:1.0:1788880633.473943:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880633.473945:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000744a548a. 00000400:00000200:1.0:1788880633.473947:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880633.473952:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000400:00000200:1.0:1788880633.473955:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:2.0:1788880633.473955:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880633.473962:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:3.0:1788880633.473968:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880633.473969:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880633.473973:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:3.0:1788880633.473975:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880633.473976:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880633.473980:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880633.473983:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880633.473993:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880633.474010:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880633.474011:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880633.474011:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780673408:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880633.474020:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880633.474025:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880633.474031:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780673408/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880633.474046:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880633.474047:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880633.474050:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880633.474053:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000006c62949. 00000100:00000001:3.0:1788880633.474053:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880633.474054:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880633.474059:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880633.474060:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:3.0:1788880633.474062:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:1.0:1788880633.474063:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000001:2.0:1788880633.474064:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880633.474065:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:1.0:1788880633.474068:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:2.0:1788880633.474068:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880633.474069:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880633.474075:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880633.474079:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00001000:3.0:1788880633.474082:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=151 v=5 (5 5 5 0) 02000000:00000010:2.0:1788880633.474083:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000446e4ede. 00000100:00001000:3.0:1788880633.474089:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=151 v=5 (1 1 1 0) 00000100:00000001:0.0:1788880633.474091:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880633.474092:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880633.474093:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880633.474095:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880633.474097:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880633.474097:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880633.474101:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00010000:00000001:3.0:1788880633.474101:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000010:2.0:1788880633.474102:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000400:00000001:0.0:1788880633.474105:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880633.474107:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00010000:00000001:3.0:1788880633.474107:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880633.474109:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880633.474112:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880633.474113:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880633.474114:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dfa9de1d. 00000100:00000040:3.0:1788880633.474116:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880633.474118:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880633.474120:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000200:0.0:1788880633.474121:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000100:00000001:2.0:1788880633.474121:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880633.474124:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880633.474126:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:2.0:1788880633.474126:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880633.474129:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880633.474131:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880633.474132:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880633.474140:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880633.474141:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880633.474146:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880633.474149:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880633.474155:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880633.474163:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880633.474165:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880633.474168:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880633.474172:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880633.474175:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880633.474178:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00100000:3.0:1788880633.474179:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780673536:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880633.474180:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880633.474186:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880633.474187:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880633.474191:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=11 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880633.474195:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=118 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880633.474197:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880633.474200:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880633.474203:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880633.474203:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780673536/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880633.474207:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880633.474213:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880633.474215:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880633.474218:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880633.474221:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cd0104b. 00010000:00000001:0.0:1788880633.474226:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880633.474228:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880633.474230:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000020:00000001:3.0:1788880633.474231:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:0.0:1788880633.474262:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880633.474265:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880633.474266:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880633.474269:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000040:0.0:1788880633.474273:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880633.474274:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880633.474277:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b899fe8d. 02000000:00000001:3.0:1788880633.474283:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880633.474285:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880633.474286:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880633.474290:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000010:3.0:1788880633.474290:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:0.0:1788880633.474293:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880633.474295:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880633.474297:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880633.474298:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880633.474300:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880633.474304:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880633.474306:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880633.474310:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880633.474313:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880633.474320:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780673664:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880633.474325:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880633.474327:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880633.474330:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780673664/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880649 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880633.474336:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880633.474337:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880633.474339:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007f19b4dd. 02000000:00000001:0.0:1788880633.474342:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880633.474343:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880633.474344:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880633.474346:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880633.474348:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f92e72a2. 02000000:00000001:0.0:1788880633.474351:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880633.474352:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880633.474353:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:0.0:1788880633.474355:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880633.474356:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880633.474358:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880633.474361:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880633.474363:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880633.474365:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880633.474366:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880633.534353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880633.534360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880633.534366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880633.534367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880633.534419:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880633.534423:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880633.534425:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880634.558488:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880634.558501:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880634.558504:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880634.558508:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880634.558515:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880634.558518:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880634.558520:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880634.558522:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880634.558527:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880634.558529:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880634.558575:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880634.558581:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880634.558588:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880635.582351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880635.582361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880635.582368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880635.582369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880635.582435:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880635.582440:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880635.582443:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880636.606456:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880636.606462:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880636.606466:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880636.606467:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880636.606780:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880636.606784:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880636.606787:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880637.630333:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880637.630339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880637.630344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880637.630347:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880637.630349:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880637.630352:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880637.630355:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880637.630357:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880637.630359:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880637.630361:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880637.630403:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880637.630407:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880637.630408:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880638.590359:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306269647, at 4306269696 00000100:00000010:0.0:1788880638.590376:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000780e164. 00000100:00000040:3.0:1788880638.590449:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306269647, at 4306269696 00000100:00000040:0.0:1788880638.590450:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:3.0:1788880638.590456:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000488d1158. 00000100:00000001:0.0:1788880638.590474:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880638.590480:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000fd20c80. 00000020:00000040:0.0:1788880638.590484:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880638.590489:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880638.590491:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:1.0:1788880638.590491:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306269647, at 4306269696 02000000:00000001:0.0:1788880638.590496:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880638.590502:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 00000100:00000010:1.0:1788880638.590505:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb1a14f4. 00000100:00000001:0.0:1788880638.590509:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880638.590521:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880638.590532:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880638.590535:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880638.590541:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000fd20c80] to pc [ptlrpcd_00_01+1] req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880638.590571:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880638.590573:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880638.590577:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880638.590579:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000363de9c9. 00000020:00000040:0.0:1788880638.590582:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880638.590584:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880638.590585:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880638.590587:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880638.590590:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000ac392bfe. 00000100:00000001:0.0:1788880638.590593:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880638.590596:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880638.590611:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880638.590612:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880638.590615:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000363de9c9] to pc [ptlrpcd_00_02+2] req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.590661:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880638.590665:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880638.590668:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880638.590673:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.590681:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880638.590683:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880638.590686:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880638.590688:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880638.590690:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880638.590691:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780673920:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880638.590694:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880638.590700:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880638.590701:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880638.590703:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880638.590713:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000aea170a1. 02000000:00000001:1.0:1788880638.590715:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880638.590721:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000744a548a. 00000400:00000010:1.0:1788880638.590726:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000007206624. 00000100:00000200:1.0:1788880638.590731:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780673920, portal 10 00000100:00000001:1.0:1788880638.590732:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880638.590735:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849365888 : -126822860185728 : ffff8ca7c0781f80) 00000100:00000040:1.0:1788880638.590742:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.590749:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880638.590751:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880638.590752:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000994ad42e. 00000100:00000200:1.0:1788880638.590754:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780673920, offset 0 00000400:00000200:1.0:1788880638.590759:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880638.590759:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880638.590769:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880638.590774:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880638.590774:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788880638.590790:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880638.590790:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880638.590807:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880638.590811:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880638.590814:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880638.590818:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [338:338:256:4294967295] 192.168.204.111@tcp LPNI seq info [338:338:8:4294967295] 00000100:00000001:3.0:1788880638.590837:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1788880638.590843:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880638.590844:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880638.590853:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:1.0:1788880638.590854:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880638.590862:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 02000000:00000001:3.0:1788880638.590862:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880638.590873:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880638.590876:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880638.590883:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880638.590886:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00100000:3.0:1788880638.590899:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780673792:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880638.590906:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880638.590908:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880638.590912:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:1.0:1788880638.590913:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 262438 usecs (263 jiffies) 02000000:00000001:3.0:1788880638.590917:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.590919:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306269959 02000000:00000001:3.0:1788880638.590924:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:1.0:1788880638.590925:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000100:00000001:1.0:1788880638.590931:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000100:00000001:1.0:1788880638.590935:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306269959 00000100:00000010:1.0:1788880638.590937:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 02000000:00000010:3.0:1788880638.590937:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000784e3b9. 00000100:00000001:1.0:1788880638.590944:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 02000000:00000001:3.0:1788880638.590945:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.590948:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306269959 00000100:00000001:1.0:1788880638.590953:0:201287:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, simply extend its deadline from 4306269647 00000400:00000010:3.0:1788880638.590956:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000100:00000001:1.0:1788880638.590957:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306269959 00000100:00000010:1.0:1788880638.590970:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000100:00000001:1.0:1788880638.590973:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880638.590975:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788880638.590975:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000001:1.0:1788880638.590984:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.590987:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880638.590988:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780673792, portal 25 00000100:00000001:1.0:1788880638.590989:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880638.591000:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880638.591003:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849357824 : -126822860193792 : ffff8ca7c0780000) 00000100:00000040:3.0:1788880638.591010:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880654 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880638.591019:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880638.591027:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880638.591028:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880638.591031:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880638.591034:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d4e901df. 00000100:00000040:3.0:1788880638.591039:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000020:00000040:0.0:1788880638.591040:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000010:3.0:1788880638.591044:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cd783db0. 00000100:00000001:0.0:1788880638.591046:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880638.591049:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000200:3.0:1788880638.591052:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780673792, offset 0 02000000:00000001:0.0:1788880638.591053:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880638.591060:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000004e0f6846. 00000100:00000001:0.0:1788880638.591070:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880638.591077:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880638.591078:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880638.591092:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880638.591094:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880638.591098:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:0.0:1788880638.591103:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d4e901df] to pc [ptlrpcd_00_03+3] req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880638.591109:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880638.591118:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880638.591126:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880638.591130:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880638.591133:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [339:339:255:4294967295] 192.168.204.111@tcp LPNI seq info [339:339:7:4294967295] 00000400:00000200:3.0:1788880638.591145:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880638.591155:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880638.591159:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880638.591174:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:3.0:1788880638.591176:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880638.591178:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880638.591184:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880638.591186:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b899fe8d. 00000100:00100000:1.0:1788880638.591188:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:1.0:1788880638.591193:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880638.591198:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880638.591199:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880638.591201:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880638.591202:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880638.591204:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880638.591205:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:3.0:1788880638.591211:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:1.0:1788880638.591212:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880638.591212:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.591258:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880638.591261:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880638.591265:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:3.0:1788880638.591266:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:1.0:1788880638.591268:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1788880638.591275:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b899fe8d type 1, nob 320 niov 1 nkiov 1 02000000:00000001:1.0:1788880638.591276:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880638.591286:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880638.591287:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780674048:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880638.591294:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306274768 00000100:00000001:1.0:1788880638.591295:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:2.0:1788880638.591297:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 02000000:00000001:1.0:1788880638.591302:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880638.591305:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880638.591307:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 02000000:00000001:1.0:1788880638.591308:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880638.591311:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306269959 00000400:00000200:2.0:1788880638.591313:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000010:1.0:1788880638.591314:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cd0104b. 02000000:00000001:1.0:1788880638.591323:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880638.591328:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306274768 00000400:00000010:1.0:1788880638.591329:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b75bbc07. 00000400:00000200:2.0:1788880638.591331:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880638.591341:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306269959 00000400:00000010:1.0:1788880638.591342:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005377d7ad. 00000400:00000200:2.0:1788880638.591342:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000994ad42e 00000400:00000010:2.0:1788880638.591352:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000994ad42e. 00000100:00000001:3.0:1788880638.591356:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306274768 00000100:00000001:2.0:1788880638.591358:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880638.591359:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780674048, portal 10 00000100:00000001:1.0:1788880638.591365:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880638.591365:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306269959 00000100:00000200:2.0:1788880638.591367:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.591370:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849369472 : -126822860182144 : ffff8ca7c0782d80) 00000100:00000001:3.0:1788880638.591375:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306274768 00000100:00000040:1.0:1788880638.591385:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880638.591386:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306269959 00000100:00000001:2.0:1788880638.591390:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880638.591398:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880638.591404:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880638.591409:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880638.591422:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880638.591433:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880638.591438:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880638.591445:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880638.591446:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880638.591448:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880638.591449:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000010:1.0:1788880638.591450:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009822ffbc. 00000100:00000001:3.0:1788880638.591453:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880638.591463:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:1.0:1788880638.591464:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780674048, offset 0 00000100:00000001:3.0:1788880638.591467:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880638.591470:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880638.591471:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880638.591477:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880638.591484:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880638.591489:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880638.591492:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880638.591495:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880638.591498:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880638.591500:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [340:340:255:4294967295] 192.168.204.111@tcp LPNI seq info [340:340:7:4294967295] 00000100:00000001:3.0:1788880638.591509:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880638.591512:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880638.591513:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880638.591515:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880638.591519:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b899fe8d. 00000800:00000200:1.0:1788880638.591525:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880638.591527:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000010:1.0:1788880638.591530:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000400:00000200:2.0:1788880638.591535:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880638.591538:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cd783db0 00000800:00000200:1.0:1788880638.591540:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:2.0:1788880638.591542:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cd783db0. 00000100:00000001:2.0:1788880638.591547:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:1.0:1788880638.591548:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880638.591552:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880638.591557:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880638.591559:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880654 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880638.591562:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 17422 usecs (18 jiffies) 00000100:00000001:1.0:1788880638.591567:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306269715 00000100:00000001:1.0:1788880638.591572:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306274768 00000100:00000001:2.0:1788880638.591574:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880638.591577:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306269715 00000100:00000040:2.0:1788880638.591579:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880654 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.591582:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306274768 00000100:00000001:1.0:1788880638.591587:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306269715 00000100:00000001:1.0:1788880638.591592:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306274768 00000100:00000001:2.0:1788880638.591593:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.591595:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306269715 00000100:00000001:2.0:1788880638.591596:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880638.591600:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306274768 00000100:00000001:1.0:1788880638.591603:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880638.591605:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880638.591608:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880638.591609:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880638.591613:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000100:00000001:1.0:1788880638.591615:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880638.591617:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880638.591621:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880638.591623:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009822ffbc 00000400:00000010:2.0:1788880638.591626:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009822ffbc. 00000100:00000001:2.0:1788880638.591629:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880638.591631:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880638.591639:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880638.591641:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880638.591647:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880638.591649:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880638.591696:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880638.591701:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880638.591706:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788880638.591711:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11607) 00000100:00000001:1.0:1788880638.591719:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880638.591722:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880638.591726:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.591730:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880638.593063:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880638.593078:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880638.593082:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880638.593085:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880638.593094:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880638.593143:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a380 00000400:00000200:3.0:1788880638.593173:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb65 [1] + 224 00000800:00000001:3.0:1788880638.593181:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880638.593188:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880638.593191:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880638.593194:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880638.593202:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880638.593208:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880638.593218:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880638.593259:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880638.593268:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880638.593272:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880638.593287:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880638.593291:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880638.593299:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880638.593301:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880638.593303:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000744a548a. 00000400:00000200:1.0:1788880638.593308:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000007206624 00000400:00000010:1.0:1788880638.593310:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000007206624. 00000100:00000001:1.0:1788880638.593313:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880638.593317:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880638.593325:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.593332:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880638.593334:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880638.593337:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880638.593340:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880638.593342:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880638.593344:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880638.593347:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880638.593356:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=6 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880638.593361:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=6 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880638.593364:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880638.593365:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880638.593368:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880638.593372:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.593376:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880638.593377:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880638.593379:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880638.593381:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880638.593385:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.593392:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880638.593401:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880638.593403:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880638.593406:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880638.593413:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780673920:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880638.593418:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.593420:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880638.593422:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000363de9c9 x1875776780673920/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.593429:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880638.593432:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880638.593435:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000aea170a1. 02000000:00000001:1.0:1788880638.593438:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880638.593440:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880638.593442:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880638.593444:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880638.593446:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000ac392bfe. 02000000:00000001:1.0:1788880638.593451:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880638.593453:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880638.593454:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000363de9c9. 00000100:00000001:1.0:1788880638.593459:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880638.593460:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880638.593462:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.593464:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880638.593465:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880638.593466:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.593468:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880638.593646:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880638.593652:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880638.593654:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880638.593656:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880638.593660:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880638.593669:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0a300 00000400:00000200:1.0:1788880638.593675:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xb6d [1] + 224 00000800:00000001:1.0:1788880638.593684:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880638.593688:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880638.593690:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880638.593693:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880638.593696:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880638.593700:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880638.593708:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.593721:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880638.593726:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880638.593728:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880638.593888:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880638.593902:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880638.593904:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880638.593906:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880638.593909:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880638.593911:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880638.593915:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880638.593919:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880638.593923:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:2.0:1788880638.593929:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880638.593930:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880638.593935:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880638.593939:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880638.593947:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880638.593949:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a400 00000400:00000200:2.0:1788880638.593966:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb75 [1] + 224 00000100:00000200:3.0:1788880638.593970:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880638.593980:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880638.593984:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880638.593987:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880638.593990:0:201286:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788880638.593991:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880638.593993:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880638.593996:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:2.0:1788880638.593998:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880638.594000:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880638.594006:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880638.594009:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00001000:3.0:1788880638.594013:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=16 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880638.594017:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:3.0:1788880638.594025:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=16 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880638.594030:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880638.594032:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880638.594035:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880638.594037:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880638.594040:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880638.594047:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880638.594058:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880638.594073:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880638.594092:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880638.594105:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880638.594110:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880638.594118:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880638.594121:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:3.0:1788880638.594133:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780673792:192.168.204.111@tcp:400: 00000800:00000001:2.0:1788880638.594136:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880638.594145:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880638.594157:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880638.594163:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880638.594167:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000fd20c80 x1875776780673792/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.594176:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880638.594176:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880638.594180:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880638.594183:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:3.0:1788880638.594186:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000784e3b9. 00000400:00000001:1.0:1788880638.594193:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:3.0:1788880638.594193:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880638.594200:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000001:1.0:1788880638.594202:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000020:00000040:3.0:1788880638.594206:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000010:1.0:1788880638.594208:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b75bbc07. 00000020:00000001:3.0:1788880638.594214:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880638.594218:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 00000400:00000200:1.0:1788880638.594219:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005377d7ad 00000400:00000010:1.0:1788880638.594225:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005377d7ad. 02000000:00000001:3.0:1788880638.594225:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880638.594232:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880638.594259:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000fd20c80. 00000100:00000001:3.0:1788880638.594263:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880638.594264:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880638.594267:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880638.594274:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880638.594277:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880638.594286:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880638.594286:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880638.594295:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880638.594300:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880638.594311:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.594320:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880638.594323:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880638.594326:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880638.594328:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880638.594330:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880638.594332:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880638.594335:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880638.594342:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=16 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880638.594346:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=123 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880638.594348:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880638.594357:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880638.594360:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880638.594364:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880638.594371:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.594374:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880638.594375:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880638.594377:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880638.594379:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880638.594382:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.594388:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880638.594390:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880638.594392:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880638.594395:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880638.594403:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780674048:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880638.594408:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880638.594411:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880638.594413:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d4e901df x1875776780674048/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880654 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880638.594419:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880638.594422:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880638.594424:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cd0104b. 02000000:00000001:1.0:1788880638.594428:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880638.594429:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880638.594431:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880638.594434:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880638.594435:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000004e0f6846. 02000000:00000001:1.0:1788880638.594440:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880638.594442:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880638.594443:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d4e901df. 00000100:00000001:1.0:1788880638.594451:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880638.594452:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880638.594454:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880638.594459:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880638.594460:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880638.594463:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880638.594465:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880638.654373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880638.654382:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880638.654388:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880638.654389:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880638.654498:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880638.654502:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880638.654505:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880639.678626:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880639.678634:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880639.678640:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880639.678641:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880639.678729:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880639.678733:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880639.678736:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880640.702386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880640.702396:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880640.702405:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880640.702407:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880640.702452:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880640.702459:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880640.702464:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880641.726375:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880641.726384:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880641.726397:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880641.726401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880641.726466:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880641.726470:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880641.726473:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880642.750411:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880642.750419:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880642.750425:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880642.750426:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880642.750487:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880642.750490:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880642.750493:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880643.710343:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306274768, at 4306274816 00000100:00000040:0.0:1788880643.710346:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880643.710354:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880643.710360:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000b4a4221. 00000100:00000010:2.0:1788880643.710366:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b63100a0. 00000020:00000040:0.0:1788880643.710378:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880643.710383:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880643.710386:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880643.710391:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880643.710391:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306274768, at 4306274816 02000000:00000010:0.0:1788880643.710400:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c93f60a2. 00000100:00000010:3.0:1788880643.710400:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000001:0.0:1788880643.710410:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880643.710421:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306274768, at 4306274816 00000100:00000040:0.0:1788880643.710422:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000010:1.0:1788880643.710432:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000001:0.0:1788880643.710435:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880643.710439:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880643.710445:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000b4a4221] to pc [ptlrpcd_00_00+0] req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880643.710481:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880643.710483:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880643.710486:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880643.710488:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006f591845. 00000020:00000040:0.0:1788880643.710490:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880643.710492:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880643.710494:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880643.710495:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880643.710498:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000004e0f6846. 00000100:00000001:0.0:1788880643.710500:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880643.710504:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880643.710510:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880643.710511:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880643.710514:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006f591845] to pc [ptlrpcd_00_01+1] req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880643.710521:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880643.710523:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880643.710532:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880643.710534:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b8846eac. 00000020:00000040:0.0:1788880643.710536:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880643.710538:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880643.710539:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880643.710540:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880643.710543:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 00000100:00000001:0.0:1788880643.710556:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880643.710560:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880643.710566:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880643.710567:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880643.710571:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b8846eac] to pc [ptlrpcd_00_02+2] req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880643.710585:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880643.710589:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880643.710591:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880643.710595:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880643.710598:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880643.710599:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880643.710609:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788880643.710617:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11612) 00000100:00000001:3.0:1788880643.710624:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880643.710626:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880643.710629:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306274768, at 4306274816 02000000:00000001:3.0:1788880643.710631:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880643.710634:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:0.0:1788880643.710637:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 02000000:00000001:3.0:1788880643.710638:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880643.710643:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006f591845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780674304:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880643.710648:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880643.710653:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880643.710656:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880643.710656:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880643.710659:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00100000:0.0:1788880643.710661:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:0.0:1788880643.710664:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1788880643.710669:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e5dc9915. 00000100:00000001:0.0:1788880643.710671:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:3.0:1788880643.710672:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880643.710674:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880643.710675:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880643.710679:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880643.710680:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880643.710681:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000100:00000001:0.0:1788880643.710683:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880643.710687:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880643.710691:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880643.710691:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000200:3.0:1788880643.710700:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780674304, portal 10 00000100:00000001:0.0:1788880643.710701:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880643.710704:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880643.710706:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849363200 : -126822860188416 : ffff8ca7c0781500) 00000100:00000001:0.0:1788880643.710711:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880643.710715:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880643.710716:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880643.710718:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880643.710720:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880643.710725:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780674432:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880643.710733:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880643.710737:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880643.710739:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880643.710741:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880643.710744:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003a17641d. 02000000:00000001:0.0:1788880643.710745:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880643.710748:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dfa9de1d. 00000400:00000010:0.0:1788880643.710752:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000200:0.0:1788880643.710754:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780674432, portal 10 00000100:00000001:0.0:1788880643.710756:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880643.710757:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364096 : -126822860187520 : ffff8ca7c0781880) 00000100:00000040:0.0:1788880643.710762:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880643.710769:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880643.710772:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880643.710774:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009fe01c6d. 00000100:00000200:0.0:1788880643.710777:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780674432, offset 0 00000400:00000200:0.0:1788880643.710782:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880643.710801:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880643.710806:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880643.710811:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880643.710814:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880643.710817:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880643.710826:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [341:341:256:4294967295] 192.168.204.111@tcp LPNI seq info [341:341:8:4294967295] 00000100:00000001:3.0:1788880643.710828:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880643.710834:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880643.710838:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007839347f. 00000400:00000200:0.0:1788880643.710843:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:3.0:1788880643.710844:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780674304, offset 0 00000800:00000200:0.0:1788880643.710851:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880643.710854:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880643.710858:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ac392bfe. 00000400:00000200:3.0:1788880643.710868:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:0.0:1788880643.710874:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880643.710875:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880643.710881:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880643.710885:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788880643.710888:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880643.710892:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788880643.710894:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ac392bfe type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880643.710899:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [342:342:255:4294967295] 192.168.204.111@tcp LPNI seq info [342:342:7:4294967295] 00000400:00000200:3.0:1788880643.710908:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880643.710913:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880643.710917:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880643.710918:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000088d6bc1. 00000100:00000040:0.0:1788880643.710925:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 17282 usecs (18 jiffies) 00000800:00000200:3.0:1788880643.710926:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880643.710931:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306274834 00000800:00000200:3.0:1788880643.710935:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:0.0:1788880643.710938:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d2bf5f. 00000800:00000200:3.0:1788880643.710940:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000088d6bc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880643.710945:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880643.710946:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880643.710952:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 259821 usecs (260 jiffies) 00000100:00000001:3.0:1788880643.710955:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306275076 00000100:00000001:0.0:1788880643.710956:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306274834 00000100:00000001:3.0:1788880643.710960:0:201286:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, simply extend its deadline from 4306274834 00000100:00000010:0.0:1788880643.710961:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000a4f85af. 00000100:00000001:0.0:1788880643.710967:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880643.710969:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306275076 00000100:00000001:0.0:1788880643.710974:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306274834 00000100:00000001:3.0:1788880643.710976:0:201286:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, simply extend its deadline from 4306274834 00000100:00000010:0.0:1788880643.710979:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009a4dee8c. 00000100:00000001:0.0:1788880643.710983:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880643.710988:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306274834 00000100:00000010:0.0:1788880643.710990:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000506ae1c0. 00000100:00000001:0.0:1788880643.710992:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880643.710998:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880643.710998:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306275076 00000100:00000001:0.0:1788880643.711005:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.711006:0:201286:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, simply extend its deadline from 4306274834 00000100:00000001:0.0:1788880643.711008:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.711011:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306275076 00000100:00000001:0.0:1788880643.711012:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.711018:0:201286:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, simply extend its deadline from 4306274834 00000100:00000001:3.0:1788880643.711030:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880643.711033:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.711035:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.711047:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880643.711062:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880643.711063:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880643.711065:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880643.711066:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880643.711067:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880643.711069:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:3.0:1788880643.711071:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880643.711073:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880643.711075:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880643.711077:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880643.711079:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880643.711080:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880643.711084:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880643.711089:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880643.711094:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.711094:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880643.711095:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880643.711099:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880643.711100:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880643.711102:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880643.711105:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780674176:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880643.711107:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880643.711109:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880643.711109:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880643.711110:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880643.711112:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b4722d49. 02000000:00000001:3.0:1788880643.711113:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880643.711116:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007f8335d2. 00000400:00000010:3.0:1788880643.711119:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eab8d071. 00000100:00000200:3.0:1788880643.711120:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780674176, portal 25 00000100:00000001:3.0:1788880643.711122:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880643.711122:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849373056 : -126822860178560 : ffff8ca7c0783b80) 00000100:00000040:3.0:1788880643.711130:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880659 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880643.711134:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880643.711135:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880643.711137:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000100:00000200:3.0:1788880643.711138:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780674176, offset 0 00000400:00000200:3.0:1788880643.711141:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880643.711145:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880643.711146:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880643.711148:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880643.711149:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880643.711151:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880643.711152:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [343:343:254:4294967295] 192.168.204.111@tcp LPNI seq info [343:343:6:4294967295] 00000400:00000200:3.0:1788880643.711157:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880643.711160:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880643.711162:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000088542ca3. 00000800:00000200:3.0:1788880643.711165:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880643.711167:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880643.711169:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000088542ca3 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880643.711171:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.711173:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306279887 00000100:00000001:3.0:1788880643.711175:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306275076 00000100:00000001:3.0:1788880643.711177:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306279887 00000100:00000001:3.0:1788880643.711179:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306275076 00000100:00000001:3.0:1788880643.711180:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306279887 00000100:00000001:3.0:1788880643.711182:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306275076 00000100:00000001:3.0:1788880643.711183:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306279887 00000100:00000001:3.0:1788880643.711184:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306275076 00000100:00000001:3.0:1788880643.711189:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880643.711190:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.711192:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.711193:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880643.711391:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880643.711395:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ac392bfe. 00000400:00000200:1.0:1788880643.711400:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880643.711404:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880643.711409:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009fe01c6d 00000400:00000010:1.0:1788880643.711411:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009fe01c6d. 00000100:00000001:1.0:1788880643.711420:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880643.711423:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880643.711432:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880643.711435:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880643.711441:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880643.711442:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880643.711453:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880643.711455:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000088d6bc1. 00000400:00000200:1.0:1788880643.711457:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880643.711460:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880643.711463:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007839347f 00000400:00000010:1.0:1788880643.711464:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007839347f. 00000100:00000001:1.0:1788880643.711466:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880643.711468:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880643.711474:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880643.711476:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880643.711481:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880643.711482:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880643.711519:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880643.711521:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000088542ca3. 00000400:00000200:1.0:1788880643.711523:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880643.711526:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880643.711528:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000400:00000010:1.0:1788880643.711530:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000100:00000001:1.0:1788880643.711532:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880643.711534:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880659 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880643.711540:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880643.712176:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880659 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880643.712184:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880643.712185:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880643.716882:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880643.716898:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880643.716902:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880643.716905:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880643.716916:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880643.716931:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a500 00000400:00000200:1.0:1788880643.716940:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb7d [1] + 224 00000800:00000001:1.0:1788880643.716946:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880643.716951:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880643.716954:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880643.716959:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880643.716966:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880643.716973:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880643.716985:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880643.717009:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880643.717016:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880643.717020:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880643.717021:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880643.717023:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880643.717027:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880643.717032:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0a480 00000400:00000200:1.0:1788880643.717038:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xb8d [1] + 224 00000800:00000001:1.0:1788880643.717041:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880643.717044:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880643.717046:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880643.717060:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880643.717077:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880643.717081:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880643.717089:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880643.717109:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880643.717113:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880643.717117:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880643.717119:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880643.717121:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880643.717124:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880643.717130:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a580 00000400:00000200:1.0:1788880643.717135:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb81 [1] + 224 00000800:00000001:1.0:1788880643.717139:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880643.717144:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880643.717146:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880643.717149:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880643.717152:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880643.717155:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880643.717162:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880643.717177:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880643.717181:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880643.717183:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880643.717228:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880643.717262:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880643.717264:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880643.717268:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880643.717271:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880643.717274:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880643.717277:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dfa9de1d. 00000400:00000200:0.0:1788880643.717281:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880643.717284:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788880643.717287:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:3.0:1788880643.717288:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880643.717292:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000200:0.0:1788880643.717293:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880643.717294:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007f8335d2. 00000400:00000200:3.0:1788880643.717300:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eab8d071 00000400:00000010:3.0:1788880643.717303:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eab8d071. 00000100:00000001:3.0:1788880643.717307:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880643.717311:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880643.717339:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880643.717346:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880643.717348:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880643.717349:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880643.717351:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880643.717355:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880643.717360:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880643.717360:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880643.717363:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880643.717364:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880643.717369:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880643.717370:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880643.717371:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880643.717374:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880643.717375:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880643.717378:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880643.717379:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=21 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880643.717384:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=21 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880643.717385:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=21 v=5 (1 1 1 1) 00000100:00001000:0.0:1788880643.717393:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=128 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880643.717393:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880643.717395:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880643.717398:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00010000:00000001:3.0:1788880643.717400:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880643.717401:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880643.717405:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880643.717406:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880643.717411:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:0.0:1788880643.717412:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880643.717418:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880643.717428:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.717433:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880643.717434:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880643.717438:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880643.717438:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880643.717441:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880643.717442:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880643.717445:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880643.717449:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880643.717458:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880643.717460:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880643.717466:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880643.717468:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880643.717477:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880643.717478:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780674176:192.168.204.111@tcp:400: 00000400:00000001:3.0:1788880643.717488:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880643.717491:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00100000:0.0:1788880643.717493:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780674432:192.168.204.111@tcp:400: 00000400:00000010:3.0:1788880643.717494:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:3.0:1788880643.717499:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000100:00000001:0.0:1788880643.717502:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880643.717503:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880643.717506:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880643.717509:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880643.717516:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880643.717521:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b8846eac x1875776780674432/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880643.717524:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880643.717533:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880643.717537:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880643.717539:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:0.0:1788880643.717542:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003a17641d. 00000100:00000001:3.0:1788880643.717542:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880643.717548:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880643.717549:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:3.0:1788880643.717550:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:0.0:1788880643.717552:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880643.717553:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000040:0.0:1788880643.717557:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880643.717557:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880643.717562:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880643.717563:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880643.717567:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 00000100:00001000:3.0:1788880643.717569:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=11 v=5 (5 5 5 5) 02000000:00000001:0.0:1788880643.717571:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00001000:3.0:1788880643.717573:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=11 v=5 (1 1 1 1) 02000000:00000001:0.0:1788880643.717575:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880643.717578:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b8846eac. 00000100:00000001:3.0:1788880643.717579:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880643.717582:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880643.717584:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00010000:00000001:3.0:1788880643.717585:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880643.717588:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880643.717589:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880643.717592:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880643.717594:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880643.717597:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880643.717599:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880643.717601:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880643.717603:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788880643.717605:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880643.717606:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880643.717611:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880643.717615:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.717620:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880643.717630:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880643.717632:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880643.717636:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880643.717643:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006f591845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780674304:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880643.717647:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880643.717649:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880643.717652:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000b4a4221 x1875776780674176/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880643.717658:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880643.717660:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880643.717661:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b4722d49. 02000000:00000001:3.0:1788880643.717664:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880643.717666:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880643.717667:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880643.717669:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880643.717670:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c93f60a2. 02000000:00000001:3.0:1788880643.717672:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880643.717673:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880643.717675:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000b4a4221. 00000100:00000001:3.0:1788880643.717678:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880643.717679:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880643.717680:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880643.717682:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006f591845 x1875776780674304/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880659 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880643.717688:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880643.717689:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880643.717690:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e5dc9915. 02000000:00000001:3.0:1788880643.717692:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880643.717693:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880643.717694:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880643.717696:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880643.717697:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000004e0f6846. 02000000:00000001:3.0:1788880643.717699:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880643.717705:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880643.717707:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006f591845. 00000100:00000001:3.0:1788880643.717710:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880643.717711:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880643.717713:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880643.717717:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880643.717718:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880643.717720:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880643.717722:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880643.774351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880643.774357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880643.774363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880643.774365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880643.774430:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880643.774434:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880643.774437:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880644.798351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880644.798368:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880644.798372:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880644.798392:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880644.798396:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880644.798398:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880644.798400:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880644.798402:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880644.798407:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880644.798408:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880644.798452:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880644.798456:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880644.798459:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880645.822361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880645.822368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880645.822374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880645.822375:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880645.822447:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880645.822452:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880645.822455:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880646.846359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880646.846366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880646.846372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880646.846373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880646.846399:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880646.846404:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880646.846408:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880647.870440:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880647.870447:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880647.870452:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880647.870456:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880647.870458:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880647.870461:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880647.870463:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880647.870466:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880647.870468:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880647.870470:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880647.870528:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880647.870532:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880647.870535:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880648.830377:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306279887, at 4306279936 00000100:00000010:1.0:1788880648.830402:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000a4f85af. 00000100:00000040:2.0:1788880648.830436:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306279887, at 4306279936 00000100:00000010:2.0:1788880648.830460:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009a4dee8c. 00000100:00000040:3.0:1788880648.831028:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306279887, at 4306279936 00000100:00000010:3.0:1788880648.831048:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000506ae1c0. 00000100:00000040:0.0:1788880648.831094:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306279887, at 4306279936 00000100:00000010:0.0:1788880648.831101:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d2bf5f. 00000100:00000040:1.0:1788880648.831158:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880648.831164:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880648.831172:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880648.831177:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880648.831181:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880648.831184:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880648.831188:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880648.831196:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:1.0:1788880648.831203:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880648.831226:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880648.831256:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880648.831259:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880648.831266:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000008b25f66] to pc [ptlrpcd_00_03+3] req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880648.831316:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880648.831320:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880648.831322:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880648.831327:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880648.831334:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880648.831336:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880648.831345:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880648.831347:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880648.831350:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880648.831353:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780674560:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880648.831357:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880648.831363:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880648.831365:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880648.831367:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880648.831371:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002ce43d73. 02000000:00000001:0.0:1788880648.831373:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880648.831378:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dfa9de1d. 00000400:00000010:0.0:1788880648.831383:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000200:0.0:1788880648.831387:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780674560, portal 25 00000100:00000001:0.0:1788880648.831390:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880648.831391:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000040:0.0:1788880648.831400:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880664 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880648.831407:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880648.831409:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880648.831415:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a08ac001. 00000100:00000200:0.0:1788880648.831419:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780674560, offset 0 00000400:00000200:0.0:1788880648.831424:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880648.831437:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880648.831443:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880648.831447:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880648.831450:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880648.831453:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880648.831461:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [344:344:256:4294967295] 192.168.204.111@tcp LPNI seq info [344:344:8:4294967295] 00000400:00000200:0.0:1788880648.831472:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880648.831472:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880648.831475:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000200:0.0:1788880648.831477:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880648.831478:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880648.831481:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000003b122924. 00000800:00000010:0.0:1788880648.831482:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000da26fd2f. 00000020:00000040:1.0:1788880648.831483:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880648.831485:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000800:00000200:0.0:1788880648.831487:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:1.0:1788880648.831487:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880648.831489:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880648.831493:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:1.0:1788880648.831496:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880648.831500:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:1.0:1788880648.831500:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880648.831502:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:0.0:1788880648.831505:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000da26fd2f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880648.831507:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880648.831510:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880648.831512:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880648.831517:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003b122924] to pc [ptlrpcd_00_00+0] req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880648.831527:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880648.831531:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306285008 00000100:00000010:0.0:1788880648.831535:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d2bf5f. 00000100:00000001:0.0:1788880648.831538:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1788880648.831538:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880648.831540:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880648.831543:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306285008 00000100:00000001:1.0:1788880648.831544:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880648.831545:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000751d0eed. 00000100:00000010:1.0:1788880648.831545:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002dcbd5d0. 00000020:00000040:1.0:1788880648.831547:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880648.831548:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880648.831549:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880648.831550:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306285008 02000000:00000001:1.0:1788880648.831550:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000010:0.0:1788880648.831552:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006bd48548. 02000000:00000001:1.0:1788880648.831552:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880648.831554:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880648.831556:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306285008 00000100:00000010:0.0:1788880648.831558:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e602733. 00000100:00000001:0.0:1788880648.831560:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 02000000:00000010:1.0:1788880648.831560:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:0.0:1788880648.831562:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880648.831562:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880648.831565:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880648.831565:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880648.831568:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880648.831569:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880648.831572:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880648.831573:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880648.831576:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788880648.831576:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002dcbd5d0] to pc [ptlrpcd_00_01+1] req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880648.831577:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880648.831579:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880648.831581:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880648.831582:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880648.831583:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880648.831584:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880648.831585:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880648.831588:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788880648.831591:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11617) 00000800:00000200:3.0:1788880648.831716:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880648.831720:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000da26fd2f. 00000400:00000200:3.0:1788880648.831723:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880648.831727:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880648.831731:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a08ac001 00000400:00000010:3.0:1788880648.831732:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a08ac001. 00000100:00000001:3.0:1788880648.831735:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880648.831738:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880664 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880648.831745:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880648.831747:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880664 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880648.831751:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880648.831752:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880648.831763:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880648.831769:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880648.831770:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880648.831772:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880648.831776:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880648.831777:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880648.831778:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880648.831780:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880648.831781:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880648.831782:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780674816:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880648.831784:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880648.831786:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880648.831787:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880648.831788:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880648.831789:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cadd275. 02000000:00000001:3.0:1788880648.831790:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880648.831793:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000400:00000010:3.0:1788880648.831795:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000200:3.0:1788880648.831797:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780674816, portal 10 00000100:00000001:3.0:1788880648.831799:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880648.831799:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938047872 : -126823771503744 : ffff8ca78a268380) 00000100:00000040:3.0:1788880648.831804:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880648.831809:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880648.831811:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880648.831813:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880648.831816:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880648.831818:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880648.831823:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880648.831832:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880648.831833:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880648.831835:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880648.831837:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880648.831840:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880648.831842:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780674688:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880648.831845:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880648.831848:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880648.831850:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880648.831851:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880648.831855:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000001f6835d8. 02000000:00000001:2.0:1788880648.831857:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880648.831867:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a7a7e63a. 00000400:00000010:2.0:1788880648.831871:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000200:2.0:1788880648.831874:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780674688, portal 10 00000100:00000001:2.0:1788880648.831875:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880648.831876:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938062208 : -126823771489408 : ffff8ca78a26bb80) 00000100:00000040:2.0:1788880648.831882:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880648.831889:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880648.831892:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880648.831894:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001c5b8d0f. 00000100:00000200:2.0:1788880648.831897:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780674688, offset 0 00000400:00000200:2.0:1788880648.831901:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880648.831907:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880648.831910:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880648.831913:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880648.831916:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880648.831919:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880648.831922:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [345:345:256:4294967295] 192.168.204.111@tcp LPNI seq info [345:345:8:4294967295] 00000400:00000200:2.0:1788880648.831928:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880648.831931:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880648.831933:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000045330cf1. 00000800:00000200:2.0:1788880648.831936:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880648.831939:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880648.831940:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000045330cf1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880648.831966:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880648.831971:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 257311 usecs (258 jiffies) 00000400:00000010:3.0:1788880648.831974:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eab8d071. 00000100:00000001:2.0:1788880648.831975:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306280195 00000100:00000200:3.0:1788880648.831979:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780674816, offset 0 00000100:00000001:2.0:1788880648.831980:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306285008 00000100:00000001:2.0:1788880648.831984:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306280195 00000100:00000001:2.0:1788880648.831988:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306285008 00000400:00000200:3.0:1788880648.831989:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880648.831996:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880648.831999:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306280195 00000400:00000200:3.0:1788880648.832001:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880648.832003:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306285008 00000400:00000200:3.0:1788880648.832007:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880648.832008:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306280195 00000100:00000001:2.0:1788880648.832012:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306285008 00000400:00000200:3.0:1788880648.832013:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880648.832017:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880648.832019:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880648.832021:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880648.832024:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880648.832027:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880648.832031:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [346:346:255:4294967295] 192.168.204.111@tcp LPNI seq info [346:346:7:4294967295] 00000100:00000001:2.0:1788880648.832033:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880648.832035:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880648.832039:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880648.832041:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880648.832044:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880648.832048:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880648.832048:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880648.832057:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880648.832058:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880648.832060:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000045330cf1. 00000800:00000010:3.0:1788880648.832061:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000088542ca3. 00000400:00000200:1.0:1788880648.832064:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:3.0:1788880648.832065:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880648.832067:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880648.832070:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001c5b8d0f 00000800:00000200:3.0:1788880648.832070:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:1.0:1788880648.832072:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001c5b8d0f. 00000800:00000200:3.0:1788880648.832072:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000088542ca3 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880648.832075:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880648.832075:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880648.832077:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 17191 usecs (18 jiffies) 00000100:00000200:1.0:1788880648.832078:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880648.832080:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306279955 00000100:00000001:3.0:1788880648.832082:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306285008 00000100:00000001:3.0:1788880648.832084:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306279955 00000100:00000001:1.0:1788880648.832085:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880648.832086:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306285008 00000100:00000040:1.0:1788880648.832087:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880648.832088:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306279955 00000100:00000001:3.0:1788880648.832089:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306285008 00000100:00000001:3.0:1788880648.832091:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306279955 00000100:00000001:3.0:1788880648.832093:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306285008 00000100:00000001:1.0:1788880648.832094:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880648.832094:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880648.832095:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880648.832096:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880648.832098:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880648.832099:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880648.832103:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880648.832104:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:1.0:1788880648.832106:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880648.832106:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:1.0:1788880648.832107:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000088542ca3. 00000100:00000001:3.0:1788880648.832107:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880648.832108:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880648.832110:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880648.832113:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880648.832115:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eab8d071 00000400:00000010:1.0:1788880648.832121:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eab8d071. 00000100:00000001:1.0:1788880648.832124:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880648.832126:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880648.832133:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880648.832135:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880648.832141:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880648.832142:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880648.832179:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880648.833527:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880648.833536:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880648.833538:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880648.833539:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880648.833545:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880648.833553:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0a600 00000400:00000200:1.0:1788880648.833560:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xb95 [1] + 224 00000800:00000001:1.0:1788880648.833566:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880648.833569:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880648.833572:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880648.833575:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880648.833579:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880648.833585:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880648.833594:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880648.833610:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880648.833620:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880648.833624:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880648.833625:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880648.833627:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880648.833632:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880648.833638:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a680 00000100:00000001:0.0:1788880648.833647:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880648.833652:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880648.833652:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xba1 [1] + 224 00000400:00000001:0.0:1788880648.833659:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:1.0:1788880648.833659:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880648.833662:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880648.833665:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dfa9de1d. 00000400:00000200:0.0:1788880648.833669:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880648.833671:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000800:00000001:1.0:1788880648.833672:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880648.833674:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788880648.833676:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880648.833680:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880648.833685:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880648.833687:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880648.833691:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880648.833708:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880648.833713:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880648.833716:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880648.833719:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880648.833723:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880648.833727:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880648.833729:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880648.833730:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880648.833733:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880648.833737:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=26 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880648.833740:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=26 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880648.833741:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880648.833742:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880648.833745:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880648.833747:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880648.833749:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880648.833749:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880648.833751:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880648.833755:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880648.833756:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880648.833760:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880648.833762:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880648.833763:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880648.833767:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:1.0:1788880648.833768:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880648.833769:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880648.833773:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880648.833777:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a700 00000100:00100000:0.0:1788880648.833786:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780674560:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880648.833789:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xb9d [1] + 224 00000100:00000001:0.0:1788880648.833791:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880648.833794:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880648.833797:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780674560/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880648.833797:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880648.833804:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000800:00000001:1.0:1788880648.833805:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880648.833806:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000200:1.0:1788880648.833809:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000010:0.0:1788880648.833813:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002ce43d73. 00000400:00000200:1.0:1788880648.833814:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:0.0:1788880648.833819:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880648.833820:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000001:0.0:1788880648.833822:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880648.833827:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000200:1.0:1788880648.833827:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880648.833831:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880648.833834:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:0.0:1788880648.833840:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:1.0:1788880648.833842:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880648.833843:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880648.833846:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:0.0:1788880648.833851:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880648.833854:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880648.833857:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880648.833860:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880648.833862:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880648.833864:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880648.833866:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880648.833867:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880648.833870:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880648.833886:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880648.833904:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880648.833907:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880648.833917:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880648.833918:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880648.833920:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:3.0:1788880648.833923:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880648.833924:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880648.833927:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880648.833931:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880648.833939:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880648.833945:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880648.833946:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880648.833948:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880648.833950:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880648.833962:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880648.833964:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880648.833965:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880648.833970:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=26 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880648.833973:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=133 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880648.833974:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880648.833975:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880648.833977:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880648.833980:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880648.833986:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880648.833989:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880648.833990:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880648.833992:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880648.833993:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880648.833995:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880648.834000:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880648.834002:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880648.834009:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880648.834013:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880648.834042:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780674816:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880648.834047:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880648.834048:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880648.834051:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002dcbd5d0 x1875776780674816/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880648.834058:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880648.834060:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880648.834062:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cadd275. 02000000:00000001:3.0:1788880648.834066:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880648.834067:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:3.0:1788880648.834068:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880648.834071:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880648.834072:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000001:3.0:1788880648.834076:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000001:2.0:1788880648.834079:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000010:3.0:1788880648.834079:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 00000400:00000001:2.0:1788880648.834083:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:3.0:1788880648.834084:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000010:2.0:1788880648.834086:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a7a7e63a. 02000000:00000001:3.0:1788880648.834087:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880648.834090:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002dcbd5d0. 00000400:00000200:2.0:1788880648.834093:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000100:00000001:3.0:1788880648.834095:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880648.834097:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880648.834099:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880648.834102:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:3.0:1788880648.834103:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880648.834106:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880648.834111:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880648.834112:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880648.834113:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880648.834119:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880648.834130:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880648.834137:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880648.834139:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880648.834142:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880648.834143:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880648.834145:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880648.834146:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880648.834148:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880648.834152:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=16 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880648.834160:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=16 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880648.834162:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880648.834163:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880648.834165:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880648.834167:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880648.834170:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880648.834171:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880648.834172:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880648.834174:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880648.834177:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880648.834184:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880648.834186:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880648.834187:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880648.834190:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880648.834197:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780674688:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880648.834201:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880648.834203:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880648.834205:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000003b122924 x1875776780674688/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880664 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880648.834210:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880648.834211:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880648.834213:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000001f6835d8. 02000000:00000001:2.0:1788880648.834215:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880648.834216:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880648.834218:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880648.834219:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880648.834221:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 02000000:00000001:2.0:1788880648.834223:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880648.834224:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880648.834225:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000003b122924. 00000100:00000001:2.0:1788880648.834227:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880648.834256:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880648.834259:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880648.834262:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880648.834263:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880648.834265:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880648.834266:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880648.894355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880648.894362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880648.894368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880648.894369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880648.894488:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880648.894492:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880648.894495:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880649.918719:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880649.918726:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880649.918731:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880649.918732:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880649.918802:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880649.918805:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880649.918808:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880650.942388:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880650.942396:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880650.942401:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880650.942401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880650.942462:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880650.942467:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880650.942471:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880651.966435:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880651.966442:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880651.966448:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880651.966450:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880651.966960:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880651.966964:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880651.966967:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880652.990417:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880652.990424:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880652.990429:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880652.990430:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880652.990544:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880652.990550:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880652.990553:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880653.950302:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306285008, at 4306285056 00000100:00000040:3.0:1788880653.950302:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306285008, at 4306285056 00000100:00000040:1.0:1788880653.950311:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880653.950311:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306285008, at 4306285056 00000100:00000001:1.0:1788880653.950318:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880653.950320:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e602733. 00000100:00000010:0.0:1788880653.950322:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d2bf5f. 00000100:00000010:2.0:1788880653.950324:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006bd48548. 00000100:00000010:1.0:1788880653.950326:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c09231ec. 00000020:00000040:1.0:1788880653.950330:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880653.950334:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880653.950338:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880653.950344:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880653.950360:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:1.0:1788880653.950367:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880653.950379:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880653.950386:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880653.950388:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880653.950392:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c09231ec] to pc [ptlrpcd_00_02+2] req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880653.950426:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880653.950428:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880653.950432:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880653.950435:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000020:00000040:1.0:1788880653.950437:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880653.950439:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880653.950439:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880653.950440:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880653.950442:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880653.950443:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:1.0:1788880653.950445:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:2.0:1788880653.950445:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880653.950449:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880653.950451:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880653.950452:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880653.950459:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880653.950460:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880653.950461:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880653.950462:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880653.950462:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880653.950466:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880653.950468:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca5e4c87] to pc [ptlrpcd_00_03+3] req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880653.950468:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880653.950472:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880653.950475:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880653.950478:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780674944:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880653.950481:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880653.950482:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880653.950484:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880653.950487:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:2.0:1788880653.950487:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:1.0:1788880653.950489:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000094565713. 02000000:00000001:2.0:1788880653.950489:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880653.950492:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:2.0:1788880653.950492:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880653.950494:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880653.950494:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000010:2.0:1788880653.950496:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000784e3b9. 02000000:00000001:1.0:1788880653.950498:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880653.950498:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880653.950499:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880653.950500:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880653.950503:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:1.0:1788880653.950504:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000400:00000010:2.0:1788880653.950504:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a7a7e63a. 00000100:00000001:1.0:1788880653.950508:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880653.950511:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880653.950520:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880653.950522:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:2.0:1788880653.950523:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000001:0.0:1788880653.950526:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880653.950530:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780674944, portal 25 00000100:00000001:1.0:1788880653.950531:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880653.950533:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880653.950536:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938056832 : -126823771494784 : ffff8ca78a26a680) 02000000:00000001:0.0:1788880653.950537:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880653.950539:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880653.950541:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880653.950542:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880669 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880653.950547:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880653.950548:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000094565713] to pc [ptlrpcd_00_00+0] req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880653.950556:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780675072:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880653.950557:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880653.950564:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880653.950566:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880653.950568:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880653.950570:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880653.950572:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880653.950573:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880653.950575:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008f790e4a. 00000100:00000040:1.0:1788880653.950576:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:0.0:1788880653.950578:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000c9708566. 00000100:00000200:2.0:1788880653.950578:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780674944, offset 0 00000100:00000040:1.0:1788880653.950581:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788880653.950585:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11622) 00000400:00000200:2.0:1788880653.950587:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880653.950592:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880653.950599:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dfa9de1d. 00000100:00000040:1.0:1788880653.950599:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306285008, at 4306285056 00000400:00000200:2.0:1788880653.950602:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:0.0:1788880653.950606:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000010:1.0:1788880653.950607:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000751d0eed. 00000100:00000001:3.0:1788880653.950607:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880653.950608:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880653.950612:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:0.0:1788880653.950613:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780675072, portal 10 00000400:00000200:2.0:1788880653.950616:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880653.950616:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880653.950618:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880653.950621:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000400:00000200:2.0:1788880653.950625:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880653.950626:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880653.950632:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880653.950634:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880653.950643:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [347:347:256:4294967295] 192.168.204.111@tcp LPNI seq info [347:347:8:4294967295] 00000100:00000001:1.0:1788880653.950647:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880653.950651:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880653.950655:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880653.950658:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880653.950663:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:2.0:1788880653.950665:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:3.0:1788880653.950666:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880653.950673:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880653.950675:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00100000:3.0:1788880653.950679:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780675200:192.168.204.111@tcp:400: 00000800:00000200:2.0:1788880653.950680:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880653.950685:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880653.950688:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000010:2.0:1788880653.950690:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 00000400:00000010:0.0:1788880653.950691:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005c796512. 02000000:00000001:3.0:1788880653.950692:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880653.950696:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880653.950697:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780675072, offset 0 02000000:00000001:3.0:1788880653.950698:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:2.0:1788880653.950703:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000010:3.0:1788880653.950704:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000fa0f7569. 00000400:00000200:0.0:1788880653.950709:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880653.950709:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880653.950714:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000800:00000200:2.0:1788880653.950715:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880653.950720:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:3.0:1788880653.950721:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000800:00000200:2.0:1788880653.950722:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880653.950728:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:3.0:1788880653.950728:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780675200, portal 10 00000100:00000001:3.0:1788880653.950732:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880653.950735:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880653.950735:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055040 : -126823771496576 : ffff8ca78a269f80) 00000400:00000200:0.0:1788880653.950739:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880653.950742:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880653.950744:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:3.0:1788880653.950746:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880653.950749:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306290127 00000400:00000200:0.0:1788880653.950752:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [348:348:255:4294967295] 192.168.204.111@tcp LPNI seq info [348:348:7:4294967295] 00000100:00000010:2.0:1788880653.950758:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b63100a0. 00000100:00000001:2.0:1788880653.950766:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880653.950767:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880653.950773:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306290127 00000100:00000040:3.0:1788880653.950773:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880653.950779:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a0959d87. 00000400:00000010:3.0:1788880653.950779:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000097c86a00. 00000400:00000200:0.0:1788880653.950785:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880653.950786:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000200:3.0:1788880653.950786:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780675200, offset 0 00000100:00000001:2.0:1788880653.950791:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306290127 00000100:00000010:2.0:1788880653.950796:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ca0fac21. 00000800:00000200:0.0:1788880653.950800:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880653.950800:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880653.950801:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880653.950807:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306290127 00000800:00000010:0.0:1788880653.950809:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f0a0057b. 00000400:00000200:3.0:1788880653.950811:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:2.0:1788880653.950812:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001e508d7a. 00000800:00000200:0.0:1788880653.950818:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880653.950818:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788880653.950818:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880653.950824:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880653.950825:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880653.950831:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880653.950831:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788880653.950834:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880653.950842:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880653.950843:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880653.950845:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:0.0:1788880653.950846:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f0a0057b type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880653.950846:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:1.0:1788880653.950849:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000400:00000200:3.0:1788880653.950852:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [349:349:254:4294967295] 192.168.204.111@tcp LPNI seq info [349:349:6:4294967295] 00000100:00000001:0.0:1788880653.950864:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880653.950867:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788880653.950869:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:0.0:1788880653.950871:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 254796 usecs (255 jiffies) 00000100:00000001:2.0:1788880653.950871:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880653.950875:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880653.950877:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306285311 00000100:00000001:2.0:1788880653.950878:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880653.950880:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880653.950881:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880653.950884:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306290127 00000100:00000001:2.0:1788880653.950885:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880653.950888:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880653.950889:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306285311 00000800:00000200:3.0:1788880653.950896:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880653.950898:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008f790e4a 00000100:00000001:0.0:1788880653.950900:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306290127 00000400:00000010:1.0:1788880653.950902:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008f790e4a. 00000800:00000010:3.0:1788880653.950903:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000088d6bc1. 00000100:00000001:0.0:1788880653.950905:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306285311 00000100:00000001:0.0:1788880653.950909:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306290127 00000100:00000001:1.0:1788880653.950909:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:3.0:1788880653.950913:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880653.950915:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306285311 00000100:00000200:1.0:1788880653.950917:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880669 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880653.950920:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306290127 00000800:00000200:3.0:1788880653.950924:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880653.950926:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1788880653.950931:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000088d6bc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880653.950932:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880653.950939:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880653.950940:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880653.950943:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880653.950945:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 17046 usecs (18 jiffies) 00000100:00000001:1.0:1788880653.950948:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880653.950949:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306285074 00000100:00000001:3.0:1788880653.950953:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306290127 00000100:00000040:1.0:1788880653.950955:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880669 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880653.950957:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880653.950957:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306285074 00000100:00000001:0.0:1788880653.950961:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880653.950961:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306290127 00000100:00000001:0.0:1788880653.950967:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880653.950969:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306285074 00000100:00000001:0.0:1788880653.950971:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880653.950974:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880653.950975:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880653.950979:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880653.950981:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880653.950998:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880653.951001:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f0a0057b. 00000400:00000200:1.0:1788880653.951005:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880653.951008:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880653.951011:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005c796512 00000400:00000010:1.0:1788880653.951013:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005c796512. 00000100:00000001:1.0:1788880653.951015:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880653.951024:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880653.951062:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306290127 00000100:00000001:3.0:1788880653.951064:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306285074 00000100:00000001:3.0:1788880653.951066:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306290127 00000100:00000001:3.0:1788880653.951068:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880653.951070:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880653.951072:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880653.951073:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880653.951077:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880653.951078:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880653.951080:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880653.951081:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880653.951082:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880653.951084:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880653.951921:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880653.951926:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880653.951932:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880653.951934:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880653.951951:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880653.951954:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000088d6bc1. 00000400:00000200:1.0:1788880653.951958:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880653.951961:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880653.951963:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000097c86a00 00000400:00000010:1.0:1788880653.951965:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000097c86a00. 00000100:00000001:1.0:1788880653.951967:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880653.951969:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880653.951975:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880653.951977:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880653.951983:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880653.951985:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880653.952871:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880653.952882:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880653.952885:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880653.952887:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880653.952893:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880653.952904:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0a780 00000400:00000200:1.0:1788880653.952911:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xbad [1] + 224 00000800:00000001:1.0:1788880653.952916:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880653.952919:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880653.952921:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880653.952924:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880653.952936:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880653.952940:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880653.952949:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880653.952966:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880653.952971:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880653.952974:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880653.952976:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880653.952978:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880653.952981:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880653.952986:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a800 00000400:00000200:1.0:1788880653.952991:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xbb5 [1] + 224 00000800:00000001:1.0:1788880653.952994:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880653.952999:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880653.953001:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880653.953005:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880653.953008:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880653.953008:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:1.0:1788880653.953012:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880653.953014:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880653.953022:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:1.0:1788880653.953023:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788880653.953026:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880653.953029:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a7a7e63a. 00000400:00000200:2.0:1788880653.953035:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880653.953038:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880653.953041:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880653.953047:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880653.953048:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880653.953054:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880653.953059:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880653.953060:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880653.953068:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880653.953070:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880653.953073:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880653.953075:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880653.953078:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880653.953080:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880653.953082:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880653.953088:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=31 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880653.953091:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=31 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880653.953094:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880653.953095:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880653.953097:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880653.953099:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880653.953102:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880653.953105:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880653.953124:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880653.953127:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880653.953128:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880653.953132:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880653.953137:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780674944:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880653.953142:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880653.953144:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880653.953146:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c09231ec x1875776780674944/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880653.953152:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880653.953154:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880653.953157:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000784e3b9. 02000000:00000001:2.0:1788880653.953160:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880653.953161:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880653.953163:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880653.953165:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880653.953167:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 02000000:00000001:2.0:1788880653.953170:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880653.953172:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880653.953173:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c09231ec. 00000100:00000001:2.0:1788880653.953176:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880653.953178:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880653.953179:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880653.953184:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880653.953186:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880653.953187:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880653.953189:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880653.953391:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880653.953394:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880653.953397:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880653.953399:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880653.953400:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dfa9de1d. 00000400:00000200:0.0:1788880653.953403:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880653.953404:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788880653.953407:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880653.953411:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880653.953430:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880653.953435:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880653.953437:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880653.953438:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880653.953439:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880653.953440:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880653.953442:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880653.953444:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880653.953448:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=21 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880653.953450:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=21 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880653.953452:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880653.953453:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880653.953455:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880653.953458:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880653.953461:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880653.953462:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880653.953464:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880653.953465:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880653.953468:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880653.953474:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880653.953476:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880653.953477:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880653.953479:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880653.953485:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780675072:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880653.953488:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880653.953491:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880653.953492:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780675072/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880653.953506:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880653.953507:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880653.953509:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000c9708566. 02000000:00000001:0.0:1788880653.953512:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880653.953513:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880653.953514:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880653.953516:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880653.953517:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 02000000:00000001:0.0:1788880653.953519:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880653.953521:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880653.953522:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:0.0:1788880653.953524:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880653.953525:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880653.953527:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880653.953530:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880653.953531:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880653.953532:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880653.953534:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880653.953867:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880653.953874:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880653.953877:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880653.953878:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880653.953884:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880653.953891:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a880 00000400:00000200:1.0:1788880653.953902:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xbbd [1] + 224 00000800:00000001:1.0:1788880653.953907:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880653.953911:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880653.953913:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880653.953916:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880653.953919:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880653.953922:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880653.953930:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880653.953950:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880653.953955:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880653.953958:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880653.954073:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880653.954082:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880653.954086:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880653.954087:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880653.954089:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:3.0:1788880653.954092:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880653.954094:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880653.954103:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880653.954107:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880653.954115:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880653.954121:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880653.954122:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880653.954124:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880653.954126:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880653.954127:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880653.954128:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880653.954130:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880653.954134:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=31 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880653.954136:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=138 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880653.954138:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880653.954139:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880653.954141:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880653.954144:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880653.954150:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880653.954151:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880653.954153:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880653.954154:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880653.954155:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880653.954157:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880653.954163:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880653.954165:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880653.954171:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880653.954175:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880653.954182:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780675200:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880653.954186:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880653.954187:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880653.954189:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000094565713 x1875776780675200/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880669 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880653.954195:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880653.954197:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880653.954198:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000fa0f7569. 02000000:00000001:3.0:1788880653.954211:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880653.954212:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880653.954213:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880653.954215:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880653.954216:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:3.0:1788880653.954219:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880653.954220:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880653.954221:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000094565713. 00000100:00000001:3.0:1788880653.954223:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880653.954225:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880653.954226:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880653.954230:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880653.954231:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880653.954307:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880653.954309:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880654.014910:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880654.014917:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880654.014923:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880654.014924:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880654.014979:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880654.014983:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880654.014987:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880655.038478:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880655.038486:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880655.038488:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880655.038491:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880655.038495:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880655.038497:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880655.038499:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880655.038502:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880655.038506:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880655.038507:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880655.038932:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880655.038935:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880655.038937:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880656.062365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880656.062373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880656.062378:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880656.062379:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880656.062443:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880656.062450:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880656.062454:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880657.086345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880657.086352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880657.086359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880657.086378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880657.086444:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880657.086448:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880657.086451:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880658.110348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880658.110355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880658.110361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880658.110364:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880658.110366:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880658.110369:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880658.110371:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880658.110374:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880658.110376:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880658.110377:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880658.110439:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880658.110444:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880658.110446:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880659.070340:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306290127, at 4306290176 00000100:00000040:3.0:1788880659.070340:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306290127, at 4306290176 00000100:00000040:1.0:1788880659.070345:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306290127, at 4306290176 00000100:00000010:2.0:1788880659.070357:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ca0fac21. 00000100:00000010:1.0:1788880659.070359:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a0959d87. 00000100:00000010:3.0:1788880659.070361:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001e508d7a. 00000100:00000040:0.0:1788880659.070456:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880659.070461:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880659.070467:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b8846eac. 00000020:00000040:0.0:1788880659.070472:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880659.070476:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880659.070479:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880659.070484:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880659.070491:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b016e4f4. 00000100:00000001:0.0:1788880659.070497:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880659.070511:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880659.070538:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880659.070540:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880659.070546:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b8846eac] to pc [ptlrpcd_00_01+1] req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880659.070565:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880659.070567:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880659.070570:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880659.070573:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000802a821. 00000020:00000040:0.0:1788880659.070575:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880659.070578:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880659.070580:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880659.070582:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880659.070585:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006df406c6. 00000100:00000001:0.0:1788880659.070589:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880659.070590:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880659.070593:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880659.070594:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880659.070595:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880659.070600:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880659.070601:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880659.070606:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:1.0:1788880659.070609:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880659.070613:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880659.070623:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000802a821] to pc [ptlrpcd_00_02+2] req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880659.070625:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880659.070628:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880659.070634:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880659.070638:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880659.070641:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880659.070645:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780675328:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880659.070648:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880659.070650:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:0.0:1788880659.070652:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880659.070655:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880659.070656:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1788880659.070657:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880659.070660:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:0.0:1788880659.070661:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000a2475c3c. 02000000:00000001:1.0:1788880659.070662:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880659.070664:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:0.0:1788880659.070667:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880659.070667:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880659.070673:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:3.0:1788880659.070673:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880659.070674:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000edcbc44b. 02000000:00000001:0.0:1788880659.070676:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880659.070680:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880659.070680:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880659.070686:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000030024c2c. 00000100:00000001:3.0:1788880659.070689:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880659.070691:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880659.070695:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880659.070697:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880659.070698:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788880659.070700:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b75bbc07. 02000000:00000001:3.0:1788880659.070704:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880659.070705:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788880659.070710:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780675456:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880659.070719:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:1.0:1788880659.070720:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005377d7ad. 00000100:00000001:0.0:1788880659.070721:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880659.070723:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880659.070724:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880659.070727:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880659.070729:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780675328, portal 25 02000000:00000001:3.0:1788880659.070730:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880659.070731:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a2475c3c] to pc [ptlrpcd_00_03+3] req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880659.070735:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000010:3.0:1788880659.070736:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f8a9d8a8. 00000100:00000001:1.0:1788880659.070738:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364096 : -126822860187520 : ffff8ca7c0781880) 02000000:00000001:3.0:1788880659.070741:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880659.070747:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880659.070749:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000100:00000040:1.0:1788880659.070754:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880675 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880659.070755:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880659.070755:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000040:0.0:1788880659.070774:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000200:3.0:1788880659.070774:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780675456, portal 10 00000100:00000001:3.0:1788880659.070779:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880659.070782:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849370368 : -126822860181248 : ffff8ca7c0783100) 00000100:00000040:0.0:1788880659.070783:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11627) 00000100:00000001:1.0:1788880659.070785:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880659.070792:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880659.070793:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880659.070797:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1788880659.070798:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000007206624. 00000100:00000001:0.0:1788880659.070801:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880659.070805:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:1.0:1788880659.070805:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780675328, offset 0 00000100:00000001:3.0:1788880659.070806:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880659.070812:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880659.070813:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880659.070818:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880659.070819:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ad0499ab. 00000100:00000200:3.0:1788880659.070825:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780675456, offset 0 00000100:00000001:0.0:1788880659.070828:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880659.070830:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880659.070834:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880659.070835:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880659.070835:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880659.070837:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880659.070840:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880659.070844:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:0.0:1788880659.070847:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780675584:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880659.070852:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880659.070856:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880659.070859:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880659.070862:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [350:350:256:4294967295] 192.168.204.111@tcp LPNI seq info [350:350:8:4294967295] 00000400:00000200:1.0:1788880659.070875:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880659.070882:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880659.070885:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880659.070889:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000400:00000200:3.0:1788880659.070890:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880659.070896:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880659.070900:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880659.070901:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880659.070906:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880659.070910:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [351:351:255:4294967295] 192.168.204.111@tcp LPNI seq info [351:351:7:4294967295] 00000800:00000200:1.0:1788880659.070915:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880659.070921:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:1.0:1788880659.070921:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880659.070927:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880659.070932:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880659.070933:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880659.070937:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880659.070937:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b899fe8d. 00000100:00000001:1.0:1788880659.070941:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880659.070942:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:3.0:1788880659.070946:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880659.070948:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306295247 02000000:00000010:0.0:1788880659.070951:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e9fa1855. 00000800:00000200:3.0:1788880659.070954:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880659.070955:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306290127 02000000:00000001:0.0:1788880659.070956:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880659.070960:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b899fe8d type 1, nob 320 niov 1 nkiov 1 00000400:00000010:0.0:1788880659.070963:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dfa9de1d. 00000100:00000001:1.0:1788880659.070963:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306295247 00000100:00000001:3.0:1788880659.070966:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880659.070969:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000010:1.0:1788880659.070971:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000100:00000040:3.0:1788880659.070971:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 252335 usecs (253 jiffies) 00000100:00000200:0.0:1788880659.070974:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780675584, portal 10 00000100:00000001:3.0:1788880659.070977:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306290429 00000100:00000001:1.0:1788880659.070978:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880659.070981:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306295247 00000100:00000001:1.0:1788880659.070985:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306295247 00000100:00000001:3.0:1788880659.070986:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306290429 00000100:00000001:3.0:1788880659.070990:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306295247 00000100:00000010:1.0:1788880659.070991:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000100:00000001:3.0:1788880659.070992:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306290429 00000100:00000001:0.0:1788880659.070993:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880659.070997:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849361408 : -126822860190208 : ffff8ca7c0780e00) 00000100:00000001:1.0:1788880659.070997:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880659.071005:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306295247 00000100:00000001:3.0:1788880659.071005:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306295247 00000100:00000040:0.0:1788880659.071006:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880659.071010:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306290429 00000800:00000200:2.0:1788880659.071015:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:3.0:1788880659.071018:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000625b9f8b. 00000800:00000010:2.0:1788880659.071021:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000100:00000001:0.0:1788880659.071024:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880659.071025:0:201287:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000040:0.0:1788880659.071029:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880659.071032:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880659.071034:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e0ace8ca. 00000100:00000001:1.0:1788880659.071035:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306290429 00000400:00000200:2.0:1788880659.071035:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880659.071037:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880659.071040:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780675584, offset 0 00000100:00000001:1.0:1788880659.071043:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880659.071043:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880659.071049:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880659.071049:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880659.071050:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880659.071056:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880659.071057:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880659.071059:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880659.071060:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880659.071062:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880659.071066:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880659.071068:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880659.071068:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880659.071071:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880659.071072:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880659.071073:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880659.071076:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880659.071077:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880659.071078:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880659.071081:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880659.071081:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880659.071082:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880659.071086:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880659.071087:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880659.071089:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [352:352:254:4294967295] 192.168.204.111@tcp LPNI seq info [352:352:6:4294967295] 00000100:00000001:1.0:1788880659.071093:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880659.071102:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880659.071109:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880659.071110:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000007206624 00000800:00000010:0.0:1788880659.071113:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f0a0057b. 00000400:00000010:2.0:1788880659.071114:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000007206624. 00000800:00000200:0.0:1788880659.071121:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880659.071123:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:0.0:1788880659.071128:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:2.0:1788880659.071130:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880675 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880659.071133:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f0a0057b type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880659.071144:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880659.071149:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880675 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880659.071150:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880659.071156:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 16920 usecs (17 jiffies) 00000100:00000001:0.0:1788880659.071161:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306290193 00000100:00000001:2.0:1788880659.071163:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880659.071165:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306295247 00000100:00000001:0.0:1788880659.071169:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306290193 00000100:00000001:0.0:1788880659.071170:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306295247 00000100:00000001:0.0:1788880659.071173:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306290193 00000100:00000001:0.0:1788880659.071174:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306295247 00000100:00000001:2.0:1788880659.071180:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880659.071183:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306290193 00000100:00000001:0.0:1788880659.071187:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306295247 00000100:00000001:0.0:1788880659.071190:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880659.071192:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880659.071195:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880659.071197:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788880659.071197:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880659.071202:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:2.0:1788880659.071202:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b899fe8d. 00000100:00000001:0.0:1788880659.071205:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880659.071209:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880659.071209:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880659.071212:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880659.071214:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880659.071217:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880659.071221:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880659.071261:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ad0499ab 00000400:00000010:2.0:1788880659.071264:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ad0499ab. 00000100:00000001:2.0:1788880659.071269:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880659.071273:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880659.071281:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880659.071283:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880659.071289:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880659.071291:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880659.071347:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880659.071350:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f0a0057b. 00000400:00000200:2.0:1788880659.071354:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880659.071357:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880659.071359:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e0ace8ca 00000400:00000010:2.0:1788880659.071361:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e0ace8ca. 00000100:00000001:2.0:1788880659.071364:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880659.071372:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880659.071379:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880659.071381:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880659.071386:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880659.071388:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880659.072992:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880659.073003:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880659.073007:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880659.073009:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880659.073016:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880659.073028:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0a900 00000400:00000200:1.0:1788880659.073036:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xbc5 [1] + 224 00000800:00000001:1.0:1788880659.073041:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880659.073047:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880659.073049:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880659.073052:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880659.073057:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880659.073061:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880659.073071:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880659.073088:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880659.073094:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880659.073096:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880659.073113:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880659.073125:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880659.073130:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880659.073132:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880659.073134:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b75bbc07. 00000400:00000200:3.0:1788880659.073137:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005377d7ad 00000400:00000010:3.0:1788880659.073139:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005377d7ad. 00000100:00000001:3.0:1788880659.073142:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880659.073146:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880659.073161:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880659.073170:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880659.073171:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880659.073174:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880659.073176:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880659.073178:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880659.073180:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880659.073182:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880659.073188:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=37 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880659.073191:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=37 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880659.073194:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880659.073195:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880659.073197:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880659.073199:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880659.073201:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880659.073205:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880659.073211:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880659.073213:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880659.073215:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880659.073226:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880659.073253:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780675328:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880659.073265:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880659.073266:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880659.073269:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b8846eac x1875776780675328/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880659.073274:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880659.073277:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880659.073279:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000edcbc44b. 02000000:00000001:3.0:1788880659.073282:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880659.073284:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880659.073289:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880659.073291:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880659.073293:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b016e4f4. 02000000:00000001:3.0:1788880659.073297:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880659.073298:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880659.073300:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b8846eac. 00000100:00000001:3.0:1788880659.073303:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880659.073304:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880659.073306:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880659.073310:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880659.073312:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880659.073314:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880659.073315:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880659.073740:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880659.073755:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880659.073768:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880659.073778:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880659.073783:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880659.073791:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0aa00 00000400:00000200:1.0:1788880659.073797:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xbd5 [1] + 224 00000800:00000001:1.0:1788880659.073802:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880659.073805:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880659.073807:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880659.073810:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880659.073814:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880659.073817:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880659.073827:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880659.073847:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880659.073852:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880659.073855:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880659.073857:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880659.073859:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880659.073863:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880659.073869:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0a980 00000400:00000200:1.0:1788880659.073875:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xbc9 [1] + 224 00000800:00000001:1.0:1788880659.073879:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880659.073885:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880659.073886:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880659.073887:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880659.073891:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880659.073895:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:0.0:1788880659.073896:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880659.073899:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880659.073900:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dfa9de1d. 00000400:00000200:0.0:1788880659.073904:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880659.073906:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788880659.073909:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880659.073910:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880659.073914:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880659.073917:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880659.073938:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880659.073942:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880659.073960:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880659.073963:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880659.073965:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880659.073967:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880659.073970:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880659.073972:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880659.073977:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880659.073984:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=37 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880659.073988:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=144 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880659.073991:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880659.073992:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880659.073993:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880659.074000:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:0.0:1788880659.074002:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:1.0:1788880659.074006:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00080000:0.0:1788880659.074010:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880659.074014:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880659.074019:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:0.0:1788880659.074020:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880659.074025:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000001:3.0:1788880659.074025:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:0.0:1788880659.074027:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:3.0:1788880659.074029:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880659.074031:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000010:3.0:1788880659.074033:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000100:00000001:0.0:1788880659.074034:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880659.074039:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000100:00000040:0.0:1788880659.074041:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880659.074042:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880659.074047:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880659.074053:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:3.0:1788880659.074054:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880659.074056:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880659.074059:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880659.074067:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880659.074085:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880659.074091:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880659.074092:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880659.074094:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00100000:0.0:1788880659.074096:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780675584:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880659.074096:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880659.074099:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880659.074101:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880659.074104:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880659.074105:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880659.074109:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00001000:3.0:1788880659.074112:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=27 v=5 (5 5 5 5) 00000100:00000040:0.0:1788880659.074115:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000a2475c3c x1875776780675584/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880659.074118:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=27 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880659.074125:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880659.074127:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880659.074130:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00010000:00000001:3.0:1788880659.074131:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:0.0:1788880659.074134:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00010000:00000001:3.0:1788880659.074135:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880659.074137:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e9fa1855. 00000100:00000001:3.0:1788880659.074140:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:0.0:1788880659.074142:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:3.0:1788880659.074143:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000020:00000001:0.0:1788880659.074146:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880659.074147:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000020:00000040:0.0:1788880659.074149:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880659.074150:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880659.074154:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880659.074156:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000030024c2c. 00000100:00000040:3.0:1788880659.074158:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880659.074161:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880659.074163:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880659.074164:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000a2475c3c. 00000100:00000001:0.0:1788880659.074173:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880659.074175:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880659.074176:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880659.074181:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880659.074182:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880659.074185:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880659.074188:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788880659.074190:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880659.074191:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880659.074196:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880659.074198:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880659.074206:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780675456:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880659.074211:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880659.074213:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880659.074215:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000802a821 x1875776780675456/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880675 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880659.074257:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880659.074259:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880659.074261:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f8a9d8a8. 02000000:00000001:3.0:1788880659.074264:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880659.074265:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880659.074266:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880659.074268:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880659.074270:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006df406c6. 02000000:00000001:3.0:1788880659.074272:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880659.074273:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880659.074274:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000802a821. 00000100:00000001:3.0:1788880659.074277:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880659.074278:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880659.074280:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880659.074283:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880659.074285:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880659.074286:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880659.074288:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880659.134560:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880659.134568:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880659.134573:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880659.134574:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880659.134634:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880659.134638:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880659.134641:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880660.158369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880660.158376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880660.158382:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880660.158384:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880660.158458:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880660.158465:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880660.158469:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880661.182337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880661.182343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880661.182355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880661.182356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880661.182413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880661.182417:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880661.182420:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880662.206350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880662.206356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880662.206362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880662.206363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880662.206407:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880662.206411:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880662.206414:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880663.230632:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880663.230639:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880663.230644:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880663.230645:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880663.230756:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880663.230759:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880663.230762:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880664.190397:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306295247, at 4306295296 00000100:00000040:0.0:1788880664.190403:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306295247, at 4306295296 00000100:00000040:2.0:1788880664.190403:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306295247, at 4306295296 00000100:00000010:1.0:1788880664.190424:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000010:2.0:1788880664.190429:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000010:0.0:1788880664.190432:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b63100a0. 00000100:00000040:3.0:1788880664.190479:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306295247, at 4306295296 00000100:00000010:3.0:1788880664.190491:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000625b9f8b. 00000100:00000040:1.0:1788880664.190492:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880664.190498:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880664.190504:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000663051e5. 00000020:00000040:1.0:1788880664.190509:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880664.190514:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880664.190517:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880664.190524:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880664.190532:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:1.0:1788880664.190540:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880664.190569:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880664.190579:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880664.190581:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880664.190588:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000663051e5] to pc [ptlrpcd_00_00+0] req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880664.190623:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880664.190626:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880664.190630:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880664.190633:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c27d4622. 00000020:00000040:1.0:1788880664.190635:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880664.190638:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880664.190639:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880664.190641:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880664.190644:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:1.0:1788880664.190647:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880664.190651:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880664.190667:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880664.190669:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880664.190673:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c27d4622] to pc [ptlrpcd_00_01+1] req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880664.190684:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880664.190687:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880664.190690:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880664.190695:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 00000100:00000001:3.0:1788880664.190695:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:1.0:1788880664.190699:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880664.190702:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880664.190704:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880664.190706:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880664.190727:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:1.0:1788880664.190730:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880664.190734:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880664.190741:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880664.190742:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880664.190746:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_00_02+2] req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880664.190778:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880664.190780:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880664.190784:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788880664.190793:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11632) 00000100:00000001:3.0:1788880664.190793:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880664.190801:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880664.190802:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880664.190807:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880664.190810:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880664.190812:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880664.190818:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880664.190829:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880664.190836:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788880664.190839:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:3.0:1788880664.190839:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880664.190841:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880664.190843:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880664.190845:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880664.190845:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880664.190849:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880664.190861:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880664.190862:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880664.190866:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880664.190866:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880664.190872:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880664.190873:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880664.190883:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788880664.190884:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780675840:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880664.190884:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880664.190889:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880664.190894:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880664.190896:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880664.190903:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880664.190905:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880664.190909:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880664.190919:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880664.190925:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000010:0.0:1788880664.190927:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e8a109c2. 00000100:00000001:1.0:1788880664.190929:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880664.190934:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880664.190937:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880664.190941:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880664.190949:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880664.190952:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780675968:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880664.190957:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:0.0:1788880664.190959:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dfa9de1d. 02000000:00000001:1.0:1788880664.190963:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880664.190967:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880664.190968:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 02000000:00000001:3.0:1788880664.190971:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880664.190975:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780675840, portal 10 02000000:00000001:1.0:1788880664.190980:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00100000:3.0:1788880664.190981:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780675712:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880664.190982:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000010:1.0:1788880664.190987:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000edcbc44b. 00000100:00000001:3.0:1788880664.190988:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880664.190989:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938049664 : -126823771501952 : ffff8ca78a268a80) 02000000:00000001:1.0:1788880664.190994:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880664.190996:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880664.191002:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880664.191005:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880664.191005:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000744a548a. 02000000:00000001:3.0:1788880664.191007:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880664.191014:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b4722d49. 00000400:00000010:1.0:1788880664.191019:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a131edae. 02000000:00000001:3.0:1788880664.191020:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880664.191023:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:1.0:1788880664.191029:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780675968, portal 10 00000100:00000040:0.0:1788880664.191030:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880664.191031:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000100:00000001:1.0:1788880664.191040:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000010:0.0:1788880664.191043:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 00000100:00000001:1.0:1788880664.191044:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 00000400:00000010:3.0:1788880664.191044:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000200:0.0:1788880664.191052:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780675840, offset 0 00000100:00000200:3.0:1788880664.191055:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780675712, portal 25 00000100:00000001:3.0:1788880664.191059:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:1.0:1788880664.191062:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880664.191064:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938053248 : -126823771498368 : ffff8ca78a269880) 00000400:00000200:0.0:1788880664.191065:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880664.191075:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880680 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880664.191083:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880664.191096:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880664.191111:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880664.191123:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880664.191124:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788880664.191135:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788880664.191135:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880664.191135:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:1.0:1788880664.191146:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005377d7ad. 00000100:00000040:3.0:1788880664.191146:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880664.191154:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ad0499ab. 00000400:00000200:0.0:1788880664.191155:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [353:353:256:4294967295] 192.168.204.111@tcp LPNI seq info [353:353:8:4294967295] 00000100:00000200:1.0:1788880664.191156:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780675968, offset 0 00000100:00000200:3.0:1788880664.191166:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780675712, offset 0 00000400:00000200:0.0:1788880664.191177:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:1.0:1788880664.191185:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880664.191191:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880664.191197:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880664.191199:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000400:00000200:1.0:1788880664.191202:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880664.191211:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1788880664.191219:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880664.191225:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880664.191231:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788880664.191265:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880664.191274:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880664.191274:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880664.191284:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [354:354:255:4294967295] 192.168.204.111@tcp LPNI seq info [354:354:7:4294967295] 00000100:00000001:0.0:1788880664.191296:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880664.191300:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880664.191305:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 249926 usecs (250 jiffies) 00000800:00000200:1.0:1788880664.191311:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880664.191314:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306295547 00000400:00000200:3.0:1788880664.191315:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880664.191324:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000100:00000010:0.0:1788880664.191325:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d2bf5f. 00000400:00000200:3.0:1788880664.191325:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880664.191334:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880664.191335:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000800:00000200:1.0:1788880664.191343:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880664.191344:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880664.191345:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306295547 00000400:00000200:3.0:1788880664.191356:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:0.0:1788880664.191358:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e245be24. 00000400:00000200:3.0:1788880664.191361:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [355:355:254:4294967295] 192.168.204.111@tcp LPNI seq info [355:355:6:4294967295] 00000800:00000200:1.0:1788880664.191367:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880664.191369:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000800:00000200:1.0:1788880664.191377:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880664.191378:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306295547 00000400:00000200:3.0:1788880664.191384:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:0.0:1788880664.191390:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a682425c. 00000100:00000001:1.0:1788880664.191393:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880664.191396:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000800:00000200:3.0:1788880664.191402:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880664.191403:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 16790 usecs (17 jiffies) 00000100:00000001:0.0:1788880664.191405:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306295547 00000800:00000010:3.0:1788880664.191408:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000100:00000010:0.0:1788880664.191416:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000455e6647. 00000100:00000001:1.0:1788880664.191416:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306295314 00000100:00000001:1.0:1788880664.191424:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306295547 00000800:00000200:3.0:1788880664.191425:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880664.191427:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880664.191431:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880664.191440:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880664.191440:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306295314 00000100:00000001:0.0:1788880664.191443:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880664.191445:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306295547 00000800:00000200:3.0:1788880664.191450:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880664.191451:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880664.191452:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306295314 00000100:00000001:1.0:1788880664.191456:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306295547 00000100:00000001:0.0:1788880664.191459:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880664.191460:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306295314 00000100:00000001:0.0:1788880664.191464:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:3.0:1788880664.191465:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880664.191467:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306295547 00000100:00000001:0.0:1788880664.191473:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880664.191476:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880664.191478:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880664.191479:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880664.191482:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880664.191485:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880664.191489:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880664.191489:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306300368 00000100:00000001:1.0:1788880664.191491:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880664.191501:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880664.191505:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306295547 00000100:00000001:3.0:1788880664.191523:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306300368 00000100:00000001:3.0:1788880664.191530:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306295547 00000100:00000001:3.0:1788880664.191535:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306300368 00000100:00000001:3.0:1788880664.191538:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306295547 00000100:00000001:3.0:1788880664.191541:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306300368 00000100:00000001:3.0:1788880664.191543:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306295547 00000100:00000001:3.0:1788880664.191546:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880664.191549:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880664.191551:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880664.191553:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880664.191557:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880664.191559:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880664.191562:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880664.191564:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880664.191567:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880664.191569:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880664.191575:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000100:00000001:3.0:1788880664.191576:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880664.191585:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880664.191596:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880664.191605:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8884ddc 00000400:00000010:2.0:1788880664.191608:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8884ddc. 00000100:00000001:2.0:1788880664.191614:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880664.191619:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880664.191630:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880664.191633:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880664.191641:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880664.191642:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880664.191657:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880664.191670:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000400:00000200:2.0:1788880664.191674:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880664.191677:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880664.191680:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005377d7ad 00000400:00000010:2.0:1788880664.191681:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005377d7ad. 00000100:00000001:2.0:1788880664.191683:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880664.191686:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880664.191703:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880664.191706:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880664.191712:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880664.191714:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880664.191760:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880664.191763:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f92e72a2. 00000400:00000200:2.0:1788880664.191768:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880664.191771:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880664.191774:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ad0499ab 00000400:00000010:2.0:1788880664.191776:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ad0499ab. 00000100:00000001:2.0:1788880664.191779:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880664.191782:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880680 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880664.191789:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880664.191791:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880680 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880664.191797:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880664.191798:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880664.193894:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880664.193910:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880664.193915:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880664.193917:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880664.193927:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880664.193943:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ab80 00000400:00000200:1.0:1788880664.193951:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xbe1 [1] + 224 00000800:00000001:1.0:1788880664.193958:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880664.193961:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880664.193964:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880664.193966:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880664.193971:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880664.193982:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880664.193991:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880664.194012:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880664.194019:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880664.194022:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880664.194033:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880664.194036:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880664.194041:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880664.194049:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ab00 00000400:00000200:1.0:1788880664.194053:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xbdd [1] + 224 00000800:00000001:1.0:1788880664.194057:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880664.194065:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880664.194067:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880664.194070:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880664.194073:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880664.194078:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880664.194081:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880664.194090:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880664.194094:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880664.194100:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880664.194105:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880664.194108:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000744a548a. 00000400:00000200:0.0:1788880664.194117:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a131edae 00000100:00000001:1.0:1788880664.194117:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000010:0.0:1788880664.194123:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a131edae. 00000100:00000001:0.0:1788880664.194128:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:1.0:1788880664.194134:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880664.194138:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880664.194140:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880664.194160:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880664.194164:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:0.0:1788880664.194168:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:3.0:1788880664.194171:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880664.194173:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880664.194175:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dfa9de1d. 00000400:00000200:3.0:1788880664.194180:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:3.0:1788880664.194184:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:3.0:1788880664.194188:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880664.194193:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880664.194197:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880664.194205:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880664.194208:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880664.194212:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880664.194224:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880664.194226:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880664.194228:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880664.194263:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880664.194266:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880664.194270:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880664.194274:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880664.194282:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880664.194286:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00001000:3.0:1788880664.194286:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=32 v=5 (5 5 5 5) 00000100:00000001:0.0:1788880664.194291:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880664.194294:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=32 v=5 (1 1 1 1) 02000000:00000001:0.0:1788880664.194297:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880664.194300:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880664.194304:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880664.194308:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00001000:0.0:1788880664.194310:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=42 v=5 (5 5 5 5) 00010000:00000001:3.0:1788880664.194313:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880664.194323:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00001000:0.0:1788880664.194324:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=149 v=5 (1 0 0 0) 00000100:00000040:3.0:1788880664.194328:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880664.194331:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880664.194333:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880664.194335:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880664.194337:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880664.194345:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:3.0:1788880664.194346:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880664.194355:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00010000:00080000:0.0:1788880664.194359:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880664.194360:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880664.194364:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880664.194373:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880664.194392:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780675840:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880664.194400:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880664.194402:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880664.194404:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880664.194410:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c27d4622 x1875776780675840/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880664.194411:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880664.194416:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880664.194419:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:0.0:1788880664.194425:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000010:3.0:1788880664.194428:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e8a109c2. 00000100:00000001:0.0:1788880664.194432:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000001:3.0:1788880664.194433:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880664.194437:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880664.194437:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880664.194440:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880664.194443:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880664.194446:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 00000100:00000040:0.0:1788880664.194450:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880664.194453:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880664.194455:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880664.194457:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c27d4622. 00000100:00000001:3.0:1788880664.194464:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880664.194469:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880664.194473:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880664.194483:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880664.194488:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880664.194488:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880664.194491:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880664.194492:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880664.194504:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880664.194507:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880664.194511:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880664.194523:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780675968:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880664.194530:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880664.194533:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880664.194536:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780675968/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880664.194544:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880664.194546:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880664.194549:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000edcbc44b. 02000000:00000001:0.0:1788880664.194554:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880664.194563:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880664.194565:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880664.194568:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880664.194570:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:0.0:1788880664.194573:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880664.194575:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880664.194577:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000001:0.0:1788880664.194580:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880664.194581:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880664.194583:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880664.194598:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880664.194599:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880664.194601:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880664.194603:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880664.194981:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880664.194997:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880664.195001:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880664.195003:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880664.195010:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880664.195025:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0aa80 00000400:00000200:1.0:1788880664.195032:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xbe9 [1] + 224 00000800:00000001:1.0:1788880664.195037:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880664.195044:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880664.195047:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880664.195050:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880664.195056:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880664.195059:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880664.195068:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880664.195088:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880664.195097:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880664.195100:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880664.195125:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880664.195129:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880664.195133:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880664.195135:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880664.195136:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:3.0:1788880664.195140:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880664.195142:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880664.195145:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880664.195149:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880664.195157:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880664.195163:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880664.195165:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880664.195167:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880664.195168:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880664.195170:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880664.195171:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880664.195173:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880664.195176:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=42 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880664.195179:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=42 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880664.195188:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880664.195189:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880664.195191:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880664.195192:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880664.195194:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880664.195197:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880664.195204:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880664.195206:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880664.195208:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880664.195210:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880664.195217:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780675712:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880664.195221:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880664.195223:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880664.195225:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000663051e5 x1875776780675712/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880680 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880664.195231:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880664.195256:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880664.195258:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b4722d49. 02000000:00000001:3.0:1788880664.195261:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880664.195262:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880664.195263:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880664.195265:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880664.195266:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 02000000:00000001:3.0:1788880664.195269:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880664.195270:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880664.195271:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000663051e5. 00000100:00000001:3.0:1788880664.195274:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880664.195275:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880664.195276:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880664.195281:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880664.195282:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880664.195284:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880664.195291:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880664.254375:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880664.254383:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880664.254385:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880664.254389:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880664.254393:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880664.254396:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880664.254397:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880664.254399:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880664.254403:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880664.254405:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880664.254475:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880664.254479:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880664.254482:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880665.278355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880665.278362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880665.278368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880665.278369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880665.278425:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880665.278429:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880665.278431:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880666.302400:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880666.302406:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880666.302412:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880666.302413:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880666.302478:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880666.302481:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880666.302484:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880667.326352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880667.326360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880667.326367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880667.326370:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880667.326372:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880667.326375:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880667.326378:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880667.326380:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880667.326382:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880667.326384:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880667.326440:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880667.326444:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880667.326446:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880668.350699:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880668.350706:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880668.350711:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880668.350712:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880668.353643:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880668.353649:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880668.353653:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880669.310346:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306300368, at 4306300416 00000100:00000040:3.0:1788880669.310360:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306300368, at 4306300416 00000100:00000010:2.0:1788880669.310365:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a682425c. 00000100:00000010:3.0:1788880669.310380:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000455e6647. 00000100:00000040:1.0:1788880669.310406:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306300368, at 4306300416 00000100:00000040:0.0:1788880669.310424:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880669.310431:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880669.310431:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e245be24. 00000100:00000010:0.0:1788880669.310437:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000a2475c3c. 00000020:00000040:0.0:1788880669.310443:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880669.310448:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880669.310451:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880669.310456:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880669.310463:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 00000100:00000001:0.0:1788880669.310470:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880669.310479:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880669.310490:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880669.310492:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880669.310498:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a2475c3c] to pc [ptlrpcd_00_03+3] req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880669.310538:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880669.310540:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880669.310544:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880669.310546:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000937cf312. 00000020:00000040:0.0:1788880669.310548:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880669.310550:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880669.310552:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880669.310553:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880669.310556:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000004e0f6846. 00000100:00000001:0.0:1788880669.310559:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880669.310574:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880669.310581:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880669.310583:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880669.310587:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000937cf312] to pc [ptlrpcd_00_00+0] req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880669.310596:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:2.0:1788880669.310600:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:2.0:1788880669.310603:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880669.310608:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880669.310610:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880669.310611:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880669.310615:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880669.310616:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880669.310621:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000007f91037. 00000100:00000001:2.0:1788880669.310621:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880669.310625:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:0.0:1788880669.310626:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880669.310629:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880669.310631:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880669.310633:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880669.310637:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880669.310640:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880669.310642:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c93f60a2. 00000100:00000001:0.0:1788880669.310646:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880669.310648:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880669.310654:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880669.310657:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880669.310662:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880669.310681:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880669.310688:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880669.310690:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880669.310698:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880669.310705:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880669.310706:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880669.310709:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880669.310710:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000007f91037] to pc [ptlrpcd_00_01+1] req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880669.310714:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880669.310719:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880669.310724:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880669.310731:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880669.310732:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780676096:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880669.310735:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880669.310741:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880669.310741:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880669.310748:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880669.310749:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11638) 02000000:00000001:2.0:1788880669.310753:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880669.310757:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880669.310762:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e8a109c2. 02000000:00000001:2.0:1788880669.310765:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880669.310767:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306300368, at 4306300416 00000400:00000010:2.0:1788880669.310771:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a7a7e63a. 00000100:00000010:0.0:1788880669.310773:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d2bf5f. 00000400:00000010:2.0:1788880669.310779:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 02000000:00000001:3.0:1788880669.310782:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:2.0:1788880669.310787:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780676096, portal 25 02000000:00000001:3.0:1788880669.310788:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880669.310795:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880669.310800:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880669.310801:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880669.310801:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880669.310805:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849361408 : -126822860190208 : ffff8ca7c0780e00) 00000100:00000001:0.0:1788880669.310806:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00100000:3.0:1788880669.310812:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780676224:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880669.310818:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880669.310819:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880685 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880669.310834:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880669.310845:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880669.310852:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880669.310864:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880669.310865:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880669.310865:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880669.310869:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880669.310874:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000010b36294. 02000000:00000001:0.0:1788880669.310875:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880669.310875:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880669.310878:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880669.310880:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c3a5d4ae. 02000000:00000001:0.0:1788880669.310882:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880669.310886:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780676096, offset 0 00000100:00100000:0.0:1788880669.310887:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780676352:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880669.310897:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880669.310901:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880669.310902:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880669.310904:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880669.310910:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880669.310912:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:3.0:1788880669.310915:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 02000000:00000010:0.0:1788880669.310917:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a581a743. 00000400:00000200:2.0:1788880669.310917:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880669.310925:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880669.310929:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:3.0:1788880669.310931:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000400:00000010:0.0:1788880669.310932:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 00000400:00000200:2.0:1788880669.310938:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:3.0:1788880669.310944:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780676224, portal 10 00000400:00000010:0.0:1788880669.310945:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000400:00000200:2.0:1788880669.310945:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880669.310952:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788880669.310954:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:0.0:1788880669.310955:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780676352, portal 10 00000100:00000001:3.0:1788880669.310959:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849367680 : -126822860183936 : ffff8ca7c0782680) 00000100:00000001:0.0:1788880669.310961:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880669.310963:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849371264 : -126822860180352 : ffff8ca7c0783480) 00000400:00000200:2.0:1788880669.310966:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [356:356:256:4294967295] 192.168.204.111@tcp LPNI seq info [356:356:8:4294967295] 00000100:00000040:0.0:1788880669.310972:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880669.310988:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788880669.310990:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880669.311010:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1788880669.311010:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880669.311015:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880669.311018:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e9812fb0. 00000800:00000010:2.0:1788880669.311018:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000011e1ab42. 00000100:00000200:0.0:1788880669.311025:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780676352, offset 0 00000100:00000001:1.0:1788880669.311036:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880669.311042:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880669.311042:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1788880669.311043:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:3.0:1788880669.311051:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880669.311058:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880669.311058:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000097c86a00. 00000800:00000200:2.0:1788880669.311059:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880669.311067:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000011e1ab42 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880669.311069:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:3.0:1788880669.311075:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780676224, offset 0 00000400:00000200:0.0:1788880669.311079:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880669.311085:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880669.311087:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880669.311090:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880669.311094:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880669.311097:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306305487 00000400:00000200:0.0:1788880669.311102:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880669.311108:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [357:357:255:4294967295] 192.168.204.111@tcp LPNI seq info [357:357:7:4294967295] 00000100:00000010:2.0:1788880669.311108:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000043ac658c. 00000100:00000001:2.0:1788880669.311116:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880669.311122:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880669.311123:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306305487 00000100:00000010:2.0:1788880669.311130:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ea9971ca. 00000100:00000001:2.0:1788880669.311134:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:3.0:1788880669.311141:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:0.0:1788880669.311143:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880669.311144:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306305487 00000800:00000010:0.0:1788880669.311156:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ac392bfe. 00000100:00000010:2.0:1788880669.311157:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ca0fac21. 00000400:00000200:3.0:1788880669.311158:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880669.311167:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:0.0:1788880669.311168:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880669.311171:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880669.311175:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306305487 00000800:00000200:0.0:1788880669.311182:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:2.0:1788880669.311182:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c99cc67. 00000400:00000200:3.0:1788880669.311184:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788880669.311189:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ac392bfe type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880669.311194:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880669.311195:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880669.311196:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880669.311201:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:0.0:1788880669.311204:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 16667 usecs (17 jiffies) 00000100:00000001:2.0:1788880669.311204:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880669.311207:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [358:358:254:4294967295] 192.168.204.111@tcp LPNI seq info [358:358:6:4294967295] 00000800:00000010:1.0:1788880669.311209:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000011e1ab42. 00000100:00000001:0.0:1788880669.311211:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306300433 00000100:00000001:2.0:1788880669.311212:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880669.311221:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306305487 00000400:00000200:1.0:1788880669.311222:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880669.311222:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880669.311224:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306300433 00000100:00000001:0.0:1788880669.311230:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306305487 00000100:00000001:2.0:1788880669.311230:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880669.311260:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880669.311285:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306300434 00000100:00000001:0.0:1788880669.311289:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306305487 00000100:00000001:0.0:1788880669.311290:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306300434 00000100:00000001:0.0:1788880669.311292:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306305487 00000100:00000001:0.0:1788880669.311299:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880669.311301:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880669.311303:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880669.311304:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880669.311308:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880669.311309:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880669.311311:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880669.311312:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880669.311313:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880669.311315:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880669.311391:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880669.311400:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880669.311401:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c3a5d4ae 00000400:00000010:1.0:1788880669.311407:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c3a5d4ae. 00000800:00000010:3.0:1788880669.311407:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000446e4ede. 00000100:00000001:1.0:1788880669.311417:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:3.0:1788880669.311419:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:1.0:1788880669.311426:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880685 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880669.311432:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880669.311442:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000446e4ede type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880669.311456:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880669.311457:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880669.311464:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 247566 usecs (248 jiffies) 00000100:00000040:1.0:1788880669.311465:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880685 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880669.311472:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306300665 00000100:00000001:3.0:1788880669.311481:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306305487 00000100:00000001:1.0:1788880669.311487:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880669.311488:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306300665 00000100:00000001:1.0:1788880669.311494:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880669.311496:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306305487 00000100:00000001:3.0:1788880669.311501:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306300665 00000100:00000001:3.0:1788880669.311504:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306305487 00000100:00000001:3.0:1788880669.311506:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306300665 00000100:00000001:3.0:1788880669.311518:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306305487 00000800:00000200:1.0:1788880669.311519:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880669.311526:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ac392bfe. 00000400:00000200:1.0:1788880669.311532:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880669.311537:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880669.311538:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880669.311546:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880669.311547:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e9812fb0 00000400:00000010:1.0:1788880669.311555:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e9812fb0. 00000100:00000001:3.0:1788880669.311557:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880669.311576:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880669.311577:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880669.311582:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880669.311586:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880669.311590:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880669.311598:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880669.311599:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880669.311604:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880669.311611:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880669.311611:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880669.311618:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880669.311631:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880669.311633:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880669.311676:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880669.311679:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000446e4ede. 00000400:00000200:1.0:1788880669.311689:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880669.311693:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880669.311696:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000097c86a00 00000400:00000010:1.0:1788880669.311697:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000097c86a00. 00000100:00000001:1.0:1788880669.311700:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880669.311704:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880669.311711:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880669.311713:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880669.311718:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880669.311720:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880669.315501:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880669.315524:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880669.315528:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880669.315530:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880669.315539:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880669.315553:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0ac00 00000400:00000200:1.0:1788880669.315571:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xbf5 [1] + 224 00000800:00000001:1.0:1788880669.315578:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880669.315582:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880669.315584:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880669.315588:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880669.315595:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880669.315600:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880669.315611:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880669.315671:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880669.315675:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880669.315694:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880669.315702:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880669.315706:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880669.315708:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880669.315720:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880669.315724:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880669.315730:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ad00 00000400:00000200:1.0:1788880669.315736:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc01 [1] + 224 00000800:00000001:1.0:1788880669.315740:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880669.315742:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880669.315744:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880669.315747:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880669.315750:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880669.315753:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880669.315761:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880669.315775:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880669.315779:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880669.315782:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880669.315784:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880669.315785:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880669.315788:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880669.315794:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ac80 00000400:00000200:1.0:1788880669.315799:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xbfd [1] + 224 00000800:00000001:1.0:1788880669.315802:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880669.315809:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880669.315811:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880669.315812:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880669.315816:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880669.315817:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880669.315823:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:0.0:1788880669.315824:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:1.0:1788880669.315827:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880669.315828:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880669.315832:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000400:00000200:0.0:1788880669.315838:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:0.0:1788880669.315842:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000040:1.0:1788880669.315843:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880669.315846:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880669.315853:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880669.315870:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880669.315886:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880669.315892:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880669.315895:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880669.315971:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880669.315973:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880669.315975:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a7a7e63a. 00000400:00000200:2.0:1788880669.315978:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880669.315980:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880669.315983:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880669.315987:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880669.315994:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880669.316009:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880669.316011:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880669.316014:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880669.316016:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880669.316019:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880669.316021:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880669.316024:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880669.316029:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=47 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880669.316034:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=47 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880669.316036:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880669.316038:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880669.316040:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880669.316043:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880669.316047:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880669.316054:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:2.0:1788880669.316054:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880669.316058:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880669.316062:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880669.316065:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880669.316068:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880669.316068:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880669.316071:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880669.316074:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:0.0:1788880669.316075:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880669.316077:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880669.316083:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=47 v=5 (5 5 5 5) 00000100:00000040:2.0:1788880669.316083:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880669.316089:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=154 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880669.316095:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880669.316097:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880669.316101:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00100000:2.0:1788880669.316102:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780676096:192.168.204.111@tcp:400: 00010000:00080000:0.0:1788880669.316108:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880669.316111:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880669.316116:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880669.316121:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000a2475c3c x1875776780676096/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880669.316143:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00010000:00000001:0.0:1788880669.316145:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880669.316147:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880669.316150:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000010:2.0:1788880669.316151:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e8a109c2. 00000100:00000040:0.0:1788880669.316153:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880669.316157:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880669.316158:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880669.316158:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880669.316162:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:0.0:1788880669.316165:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:2.0:1788880669.316166:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880669.316173:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880669.316178:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000020:00000001:2.0:1788880669.316178:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880669.316182:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:2.0:1788880669.316183:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 00000100:00000040:0.0:1788880669.316186:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880669.316189:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880669.316193:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00100000:0.0:1788880669.316198:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780676352:192.168.204.111@tcp:400: 00000100:00000010:2.0:1788880669.316198:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000a2475c3c. 00000100:00000001:2.0:1788880669.316205:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880669.316206:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880669.316210:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880669.316210:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880669.316214:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000007f91037 x1875776780676352/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880669.316215:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880669.316224:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880669.316224:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1788880669.316226:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880669.316228:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880669.316229:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a581a743. 00000100:00000001:3.0:1788880669.316256:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880669.316260:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880669.316261:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000001:3.0:1788880669.316263:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880669.316265:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880669.316267:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880669.316269:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:3.0:1788880669.316273:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880669.316274:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880669.316278:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880669.316290:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880669.316298:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880669.316304:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880669.316304:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880669.316306:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880669.316307:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880669.316309:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880669.316310:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880669.316311:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880669.316315:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=37 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880669.316317:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=37 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880669.316319:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880669.316320:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880669.316322:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880669.316329:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880669.316331:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880669.316332:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880669.316334:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880669.316335:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880669.316337:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880669.316343:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880669.316344:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880669.316345:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880669.316348:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880669.316353:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780676224:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880669.316356:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880669.316357:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880669.316359:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000937cf312 x1875776780676224/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880685 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880669.316363:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880669.316364:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880669.316366:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000010b36294. 02000000:00000001:3.0:1788880669.316368:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880669.316369:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880669.316370:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880669.316372:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880669.316373:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000004e0f6846. 02000000:00000001:3.0:1788880669.316375:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880669.316376:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880669.316377:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000937cf312. 00000100:00000001:3.0:1788880669.316380:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880669.316381:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880669.316382:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880669.316386:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880669.316387:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880669.316393:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880669.316395:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880669.316632:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880669.316635:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880669.316636:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880669.316638:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880669.316640:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c93f60a2. 02000000:00000001:0.0:1788880669.316643:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880669.316644:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880669.316645:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000007f91037. 00000100:00000001:0.0:1788880669.316648:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880669.316649:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880669.316657:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880669.316660:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880669.316662:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880669.316664:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880669.316665:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880669.374355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880669.374361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880669.374367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880669.374368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880669.374430:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880669.374434:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880669.374436:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880670.398434:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880670.398449:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880670.398460:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880670.398462:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880670.398529:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880670.398536:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880670.398539:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880671.422437:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880671.422447:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880671.422454:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880671.422455:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880671.422547:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880671.422555:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880671.422560:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880672.446318:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880672.446323:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880672.446328:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880672.446329:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880672.446382:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880672.446386:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880672.446388:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880673.470380:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880673.470391:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880673.470399:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880673.470401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880673.470501:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880673.470509:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880673.470513:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880674.430331:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306305487, at 4306305536 00000100:00000040:1.0:1788880674.430331:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306305487, at 4306305536 00000100:00000040:2.0:1788880674.430331:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306305487, at 4306305536 00000100:00000010:0.0:1788880674.430357:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000043ac658c. 00000100:00000010:2.0:1788880674.430365:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ca0fac21. 00000100:00000040:3.0:1788880674.430428:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880674.430436:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880674.430447:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880674.430453:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880674.430459:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880674.430473:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880674.430479:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880674.430491:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000446e4ede. 00000100:00000001:3.0:1788880674.430499:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880674.430512:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880674.430524:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880674.430526:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880674.430532:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_02+2] req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880674.430554:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880674.430556:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880674.430559:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880674.430562:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000020:00000040:3.0:1788880674.430564:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880674.430565:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880674.430567:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880674.430568:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880674.430571:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f92e72a2. 00000100:00000001:2.0:1788880674.430574:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880674.430574:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880674.430580:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880674.430583:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880674.430590:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880674.430599:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880674.430607:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880674.430609:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880674.430611:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880674.430612:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880674.430619:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1788880674.430620:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_03+3] req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880674.430624:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880674.430630:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880674.430637:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780676480:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880674.430638:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880674.430644:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880674.430646:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880674.430651:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:2.0:1788880674.430655:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:3.0:1788880674.430655:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 02000000:00000001:2.0:1788880674.430659:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880674.430660:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:2.0:1788880674.430665:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880674.430666:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880674.430670:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880674.430672:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880674.430674:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000064b136ca. 02000000:00000010:3.0:1788880674.430677:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b899fe8d. 02000000:00000001:2.0:1788880674.430678:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880674.430684:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880674.430689:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a7a7e63a. 00000100:00000040:3.0:1788880674.430690:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:2.0:1788880674.430700:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000001:3.0:1788880674.430701:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880674.430705:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880674.430709:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_00+0] req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880674.430711:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780676480, portal 25 00000100:00000001:2.0:1788880674.430717:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880674.430726:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000040:2.0:1788880674.430733:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880690 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880674.430736:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880674.430740:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880674.430745:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880674.430751:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880674.430755:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880674.430757:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880674.430759:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11643) 00000100:00000001:0.0:1788880674.430766:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880674.430766:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880674.430771:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001e963647. 00000100:00000200:2.0:1788880674.430776:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780676480, offset 0 00000100:00000040:0.0:1788880674.430782:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880674.430784:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306305487, at 4306305536 00000400:00000200:2.0:1788880674.430792:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788880674.430796:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c99cc67. 00000100:00000001:0.0:1788880674.430812:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880674.430814:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880674.430815:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880674.430816:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880674.430828:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880674.430830:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880674.430830:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880674.430831:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788880674.430840:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880674.430844:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788880674.430844:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880674.430851:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880674.430856:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00100000:0.0:1788880674.430867:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780676608:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880674.430868:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880674.430883:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:2.0:1788880674.430884:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [359:359:256:4294967295] 192.168.204.111@tcp LPNI seq info [359:359:8:4294967295] 00000100:00000001:0.0:1788880674.430886:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880674.430889:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880674.430897:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880674.430898:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880674.430901:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880674.430903:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880674.430907:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:3.0:1788880674.430910:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880674.430918:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780676736:192.168.204.111@tcp:400: 02000000:00000010:0.0:1788880674.430919:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000072dc05f6. 00000400:00000200:2.0:1788880674.430924:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880674.430930:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880674.430932:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:2.0:1788880674.430938:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880674.430939:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:0.0:1788880674.430941:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 02000000:00000001:3.0:1788880674.430944:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880674.430945:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000781b5970. 02000000:00000001:3.0:1788880674.430950:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:0.0:1788880674.430952:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000800:00000200:2.0:1788880674.430957:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000010:3.0:1788880674.430957:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000082966425. 02000000:00000001:3.0:1788880674.430962:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880674.430969:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000800:00000200:2.0:1788880674.430973:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:0.0:1788880674.430978:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780676608, portal 10 00000800:00000200:2.0:1788880674.430979:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000781b5970 type 1, nob 320 niov 1 nkiov 1 00000400:00000010:3.0:1788880674.430979:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000001:0.0:1788880674.430988:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880674.430993:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000200:3.0:1788880674.430993:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780676736, portal 10 00000100:00000001:3.0:1788880674.431001:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880674.431003:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880674.431006:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000001:2.0:1788880674.431011:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306310607 00000100:00000040:0.0:1788880674.431012:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880674.431020:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788880674.431023:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ca0fac21. 00000100:00000001:2.0:1788880674.431037:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:2.0:1788880674.431047:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306310607 00000100:00000001:0.0:1788880674.431056:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:2.0:1788880674.431057:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c61ceeab. 00000100:00000001:3.0:1788880674.431064:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880674.431066:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:0.0:1788880674.431069:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880674.431072:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306310607 00000100:00000040:3.0:1788880674.431075:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880674.431079:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f34d0caa. 00000100:00000010:2.0:1788880674.431081:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a3bf26bf. 00000400:00000010:3.0:1788880674.431082:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eab8d071. 00000100:00000200:0.0:1788880674.431089:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780676608, offset 0 00000100:00000001:2.0:1788880674.431090:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000200:3.0:1788880674.431090:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780676736, offset 0 00000100:00000001:2.0:1788880674.431110:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306310607 00000400:00000200:0.0:1788880674.431116:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880674.431119:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000077b36062. 00000100:00000010:1.0:1788880674.431125:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ea9971ca. 00000100:00000001:2.0:1788880674.431126:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:0.0:1788880674.431133:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880674.431133:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880674.431141:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880674.431147:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880674.431148:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880674.431155:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880674.431167:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:0.0:1788880674.431170:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880674.431171:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880674.431177:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880674.431178:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880674.431181:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880674.431187:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880674.431187:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880674.431197:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [360:360:255:4294967295] 192.168.204.111@tcp LPNI seq info [360:360:7:4294967295] 00000100:00000001:2.0:1788880674.431203:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880674.431214:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880674.431222:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880674.431261:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880674.431265:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880674.431267:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c93f60a2. 00000800:00000200:0.0:1788880674.431281:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880674.431285:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880674.431292:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000781b5970. 00000800:00000200:0.0:1788880674.431295:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880674.431303:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:0.0:1788880674.431304:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c93f60a2 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880674.431315:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880674.431318:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:0.0:1788880674.431322:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 245276 usecs (246 jiffies) 00000100:00000001:0.0:1788880674.431328:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306305782 00000100:00000001:0.0:1788880674.431331:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306310607 00000100:00000001:0.0:1788880674.431333:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306305782 00000100:00000001:0.0:1788880674.431335:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306310607 00000100:00000001:0.0:1788880674.431336:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306305782 00000100:00000001:0.0:1788880674.431345:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306310607 00000100:00000001:0.0:1788880674.431347:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306305782 00000100:00000001:0.0:1788880674.431350:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306310607 00000100:00000001:0.0:1788880674.431352:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880674.431356:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880674.431359:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880674.431361:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880674.431368:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880674.431372:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880674.431374:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880674.431376:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880674.431377:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880674.431380:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880674.431446:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880674.431455:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880674.431459:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880674.431462:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880674.431477:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880674.431481:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880674.431492:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [361:361:254:4294967295] 192.168.204.111@tcp LPNI seq info [361:361:6:4294967295] 00000400:00000200:3.0:1788880674.431498:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880674.431502:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880674.431505:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000088d6bc1. 00000800:00000200:3.0:1788880674.431508:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880674.431512:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880674.431515:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000088d6bc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880674.431518:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880674.431521:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 16570 usecs (17 jiffies) 00000100:00000001:3.0:1788880674.431523:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306305554 00000100:00000001:3.0:1788880674.431525:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306310607 00000100:00000001:3.0:1788880674.431526:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306305554 00000100:00000001:3.0:1788880674.431528:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306310607 00000100:00000001:3.0:1788880674.431528:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306305554 00000100:00000001:3.0:1788880674.431529:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306310607 00000100:00000001:3.0:1788880674.431530:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306305554 00000100:00000001:3.0:1788880674.431531:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306310607 00000100:00000001:3.0:1788880674.431532:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880674.431534:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880674.431535:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880674.431536:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880674.431539:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880674.431540:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880674.431541:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880674.431542:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880674.431543:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880674.431544:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880674.431598:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001e963647 00000400:00000010:1.0:1788880674.431601:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001e963647. 00000100:00000001:1.0:1788880674.431605:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880674.431608:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880690 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880674.431625:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880674.431628:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880690 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880674.431635:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880674.431637:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880674.431654:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880674.431657:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c93f60a2. 00000400:00000200:1.0:1788880674.431661:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880674.431665:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880674.431668:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f34d0caa 00000400:00000010:1.0:1788880674.431670:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f34d0caa. 00000100:00000001:1.0:1788880674.431672:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880674.431676:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880674.431681:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880674.431683:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880674.431686:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880674.431687:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880674.431722:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880674.431724:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000088d6bc1. 00000400:00000200:1.0:1788880674.431728:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880674.431732:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880674.431734:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eab8d071 00000400:00000010:1.0:1788880674.431736:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eab8d071. 00000100:00000001:1.0:1788880674.431739:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880674.431742:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880674.431749:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880674.431751:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880674.431768:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880674.431770:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880674.434167:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880674.434183:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880674.434187:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880674.434189:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880674.434199:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880674.434213:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0ad80 00000400:00000200:1.0:1788880674.434226:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xc0d [1] + 224 00000800:00000001:1.0:1788880674.434256:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880674.434261:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880674.434263:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880674.434268:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880674.434275:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880674.434280:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880674.434290:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880674.434311:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880674.434319:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880674.434323:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880674.434324:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880674.434326:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880674.434330:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880674.434336:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ae00 00000400:00000200:1.0:1788880674.434342:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc15 [1] + 224 00000800:00000001:1.0:1788880674.434346:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880674.434353:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880674.434355:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880674.434359:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880674.434360:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880674.434365:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880674.434366:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:2.0:1788880674.434373:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880674.434377:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880674.434380:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a7a7e63a. 00000100:00000200:1.0:1788880674.434381:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880674.434386:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:2.0:1788880674.434391:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:2.0:1788880674.434396:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:1.0:1788880674.434400:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880674.434405:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880674.434420:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880674.434426:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880674.434428:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880674.434430:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880674.434432:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880674.434434:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880674.434435:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880674.434438:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880674.434441:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880674.434442:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880674.434447:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880674.434448:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=52 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880674.434453:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=52 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880674.434473:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880674.434476:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880674.434479:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880674.434481:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880674.434484:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880674.434488:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880674.434494:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880674.434498:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880674.434500:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880674.434504:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880674.434510:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780676480:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880674.434513:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880674.434515:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880674.434517:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880674.434520:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000001:0.0:1788880674.434522:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:2.0:1788880674.434524:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780676480/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880674.434526:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880674.434529:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000400:00000200:0.0:1788880674.434535:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:0.0:1788880674.434538:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:2.0:1788880674.434538:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880674.434542:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880674.434544:0:201288:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000010:2.0:1788880674.434546:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000064b136ca. 00000100:00000200:0.0:1788880674.434551:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880674.434552:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880674.434556:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880674.434559:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880674.434564:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880674.434567:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000446e4ede. 00000100:00000200:0.0:1788880674.434569:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880674.434573:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880674.434577:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880674.434580:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000010:2.0:1788880674.434580:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:0.0:1788880674.434584:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880674.434586:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880674.434587:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880674.434589:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880674.434590:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880674.434592:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880674.434593:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880674.434597:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880674.434599:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880674.434604:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00001000:0.0:1788880674.434605:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=42 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880674.434608:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880674.434611:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=42 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880674.434612:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880674.434622:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880674.434623:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880674.434626:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880674.434630:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880674.434633:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880674.434636:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880674.434638:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880674.434641:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880674.434642:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880674.434646:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880674.434652:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880674.434654:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880674.434655:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880674.434658:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880674.434664:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780676608:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880674.434667:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880674.434669:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880674.434670:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780676608/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880674.434674:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880674.434675:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880674.434676:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000072dc05f6. 02000000:00000001:0.0:1788880674.434678:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880674.434679:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880674.434680:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880674.434681:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000800:00000001:1.0:1788880674.434710:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880674.434714:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880674.434715:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880674.434717:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880674.434720:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880674.434725:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ae80 00000400:00000200:1.0:1788880674.434729:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc19 [1] + 224 00000800:00000001:1.0:1788880674.434731:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880674.434735:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880674.434736:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880674.434738:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880674.434740:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880674.434742:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880674.434747:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880674.434759:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880674.434762:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880674.434763:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880674.434791:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880674.434793:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f92e72a2. 00000100:00000001:3.0:1788880674.434798:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880674.434801:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880674.434804:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880674.434805:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:0.0:1788880674.434807:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880674.434811:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880674.434820:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880674.434825:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880674.434826:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880674.434828:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880674.434829:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880674.434853:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880674.434857:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880674.434860:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:3.0:1788880674.434865:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880674.434867:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880674.434877:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880674.434885:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880674.434895:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880674.434902:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880674.434904:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880674.434906:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880674.434915:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880674.434917:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880674.434919:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880674.434921:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880674.434928:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=52 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880674.434932:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=9 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880674.434935:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880674.434937:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880674.434939:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880674.434943:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880674.434950:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880674.434953:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880674.434955:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880674.434957:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880674.434958:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880674.434961:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880674.434967:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880674.434976:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880674.434987:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880674.434991:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880674.434998:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780676736:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880674.435003:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880674.435004:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880674.435006:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780676736/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880690 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880674.435010:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880674.435012:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880674.435014:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000082966425. 02000000:00000001:3.0:1788880674.435016:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880674.435017:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880674.435019:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880674.435021:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880674.435023:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b899fe8d. 02000000:00000001:3.0:1788880674.435027:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880674.435028:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880674.435030:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:3.0:1788880674.435033:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880674.435034:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880674.435035:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880674.435040:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880674.435042:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880674.435044:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880674.435046:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880674.494321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880674.494331:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880674.494334:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880674.494336:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880674.494341:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880674.494344:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880674.494362:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880674.494364:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880674.494369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880674.494370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880674.494429:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880674.494432:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880674.494434:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880675.518333:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880675.518340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880675.518345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880675.518346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880675.518404:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880675.518409:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880675.518412:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880676.542370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880676.542380:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880676.542389:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880676.542390:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880676.542467:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880676.542472:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880676.542475:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880677.566350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880677.566356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880677.566361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880677.566365:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880677.566367:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880677.566371:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880677.566374:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880677.566376:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880677.566378:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880677.566380:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880677.566444:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880677.566448:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880677.566450:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880678.590375:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880678.590397:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880678.590406:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880678.590406:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880678.590490:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880678.590494:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880678.590496:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880679.550339:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306310607, at 4306310656 00000100:00000040:1.0:1788880679.550339:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306310607, at 4306310656 00000100:00000010:0.0:1788880679.550355:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ca0fac21. 00000100:00000040:2.0:1788880679.550356:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306310607, at 4306310656 00000100:00000010:1.0:1788880679.550357:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c61ceeab. 00000100:00000040:3.0:1788880679.550362:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306310607, at 4306310656 00000100:00000010:2.0:1788880679.550384:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a3bf26bf. 00000100:00000010:3.0:1788880679.550387:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000077b36062. 00000100:00000040:2.0:1788880679.550426:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880679.550432:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880679.550440:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880679.550446:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880679.550450:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880679.550453:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880679.550458:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880679.550465:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000100:00000001:2.0:1788880679.550472:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880679.550483:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880679.550492:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880679.550495:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880679.550501:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_01+1] req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880679.550525:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880679.550532:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880679.550535:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880679.550539:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 00000020:00000040:2.0:1788880679.550541:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880679.550544:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880679.550544:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880679.550545:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880679.550549:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 00000100:00000001:0.0:1788880679.550550:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880679.550553:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880679.550557:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880679.550558:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880679.550563:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880679.550568:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:0.0:1788880679.550576:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880679.550586:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880679.550588:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880679.550590:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880679.550594:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_02+2] req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880679.550596:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880679.550602:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880679.550611:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880679.550615:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788880679.550622:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780676864:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880679.550624:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880679.550638:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880679.550645:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880679.550649:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880679.550652:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880679.550655:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000afd3911b. 02000000:00000001:0.0:1788880679.550658:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880679.550661:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880679.550663:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 00000100:00000040:2.0:1788880679.550665:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880679.550670:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000400:00000010:0.0:1788880679.550672:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000010:2.0:1788880679.550673:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000020:00000040:2.0:1788880679.550677:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000200:0.0:1788880679.550679:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780676864, portal 25 00000100:00000001:2.0:1788880679.550681:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880679.550682:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880679.550687:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 02000000:00000001:2.0:1788880679.550687:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880679.550693:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880679.550694:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788880679.550701:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 02000000:00000010:2.0:1788880679.550701:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006aca3ff7. 00000100:00000040:0.0:1788880679.550708:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880695 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880679.550710:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880679.550711:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880679.550723:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880679.550724:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:2.0:1788880679.550727:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880679.550729:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880679.550736:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880679.550740:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880679.550742:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880679.550745:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880679.550746:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880679.550747:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f34d0caa. 00000100:00000001:3.0:1788880679.550753:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880679.550756:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780676864, offset 0 00000100:00000040:1.0:1788880679.550759:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880679.550759:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880679.550766:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880679.550767:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880679.550781:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_03+3] req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880679.550783:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880679.550798:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880679.550805:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880679.550818:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880679.550829:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880679.550839:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [362:362:256:4294967295] 192.168.204.111@tcp LPNI seq info [362:362:8:4294967295] 00000100:00000001:1.0:1788880679.550855:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880679.550859:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880679.550864:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:1.0:1788880679.550864:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880679.550869:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880679.550869:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880679.550878:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880679.550880:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880679.550880:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000010:0.0:1788880679.550884:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c93f60a2. 00000100:00100000:1.0:1788880679.550892:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780676992:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880679.550894:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000800:00000200:0.0:1788880679.550905:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880679.550906:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880679.550909:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11648) 02000000:00000001:1.0:1788880679.550910:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880679.550913:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880679.550916:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880679.550919:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f8c34a6a. 02000000:00000001:1.0:1788880679.550922:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880679.550923:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:1.0:1788880679.550926:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b884f446. 00000800:00000200:0.0:1788880679.550929:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c93f60a2 type 1, nob 320 niov 1 nkiov 1 00000400:00000010:1.0:1788880679.550930:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000007206624. 00000100:00000200:1.0:1788880679.550935:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780676992, portal 10 00000100:00000001:1.0:1788880679.550939:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880679.550940:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938435456 : -126823771116160 : ffff8ca78a2c6d80) 00000100:00000001:0.0:1788880679.550943:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880679.550946:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880679.550949:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306315727 00000100:00000001:1.0:1788880679.550955:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:0.0:1788880679.550956:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e64ede4. 00000100:00000040:1.0:1788880679.550957:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880679.550961:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009822ffbc. 00000100:00000001:0.0:1788880679.550964:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000200:1.0:1788880679.550964:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780676992, offset 0 00000100:00000001:0.0:1788880679.550973:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306315727 00000400:00000200:1.0:1788880679.550976:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880679.550978:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000535177f1. 00000100:00000001:0.0:1788880679.550981:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880679.550984:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306315727 00000400:00000200:1.0:1788880679.550985:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:0.0:1788880679.550987:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003b9d7f95. 00000400:00000200:1.0:1788880679.550991:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880679.550992:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:1.0:1788880679.550997:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880679.550998:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306315727 00000400:00000200:1.0:1788880679.551002:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:0.0:1788880679.551007:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d2bf5f. 00000400:00000200:1.0:1788880679.551009:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880679.551012:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788880679.551018:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [363:363:255:4294967295] 192.168.204.111@tcp LPNI seq info [363:363:7:4294967295] 00000100:00000001:0.0:1788880679.551019:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880679.551024:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880679.551029:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880679.551033:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788880679.551034:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880679.551034:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880679.551044:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00100000:3.0:1788880679.551044:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000800:00000010:2.0:1788880679.551046:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c93f60a2. 00000100:00000001:0.0:1788880679.551047:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880679.551047:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880679.551053:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880679.551058:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880679.551059:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880679.551061:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880679.551066:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880679.551068:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880679.551071:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880679.551076:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000400:00000200:2.0:1788880679.551080:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000200:1.0:1788880679.551087:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880679.551089:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f34d0caa 00000800:00000200:1.0:1788880679.551096:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:2.0:1788880679.551097:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f34d0caa. 00000800:00000200:1.0:1788880679.551099:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880679.551106:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880679.551107:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000040:1.0:1788880679.551110:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 243008 usecs (244 jiffies) 00000100:00000001:1.0:1788880679.551113:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306310900 00000100:00000001:1.0:1788880679.551116:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306315727 00000100:00000001:1.0:1788880679.551123:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306310900 00000100:00000001:1.0:1788880679.551124:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306315727 00000100:00000200:2.0:1788880679.551127:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880695 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880679.551127:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880679.551128:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306310900 00000100:00000001:3.0:1788880679.551134:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880679.551135:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306315727 00000100:00000001:1.0:1788880679.551143:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306310900 00000100:00000001:3.0:1788880679.551143:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880679.551150:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880679.551153:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306315727 00000100:00000001:3.0:1788880679.551154:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880679.551164:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880679.551168:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880679.551169:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880679.551172:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880679.551178:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880695 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880679.551183:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880679.551194:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880679.551196:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880679.551198:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880679.551209:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880679.551210:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880679.551211:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880679.551217:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880679.551218:0:201278:0:(events.c:68:request_out_callback()) Process leaving 02000000:00000001:3.0:1788880679.551218:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880679.551224:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880679.551226:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880679.551228:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880679.551264:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780677120:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880679.551269:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880679.551275:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880679.551277:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880679.551280:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880679.551285:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000060f4a118. 02000000:00000001:3.0:1788880679.551287:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880679.551294:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000400:00000010:3.0:1788880679.551302:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000200:3.0:1788880679.551308:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780677120, portal 10 00000100:00000001:3.0:1788880679.551312:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880679.551314:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 00000100:00000040:3.0:1788880679.551320:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880679.551333:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880679.551337:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000010:2.0:1788880679.551338:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000100:00000040:3.0:1788880679.551341:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880679.551344:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000400:00000200:2.0:1788880679.551346:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:3.0:1788880679.551347:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780677120, offset 0 00000400:00000200:2.0:1788880679.551356:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880679.551360:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880679.551381:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009822ffbc 00000400:00000200:3.0:1788880679.551385:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:2.0:1788880679.551386:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009822ffbc. 00000100:00000001:2.0:1788880679.551392:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880679.551398:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880679.551406:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880679.551408:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880679.551414:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880679.551416:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:3.0:1788880679.551433:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880679.551438:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880679.551442:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880679.551445:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880679.551448:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [364:364:255:4294967295] 192.168.204.111@tcp LPNI seq info [364:364:7:4294967295] 00000400:00000200:3.0:1788880679.551458:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880679.551463:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880679.551468:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b899fe8d. 00000800:00000200:3.0:1788880679.551474:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880679.551479:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880679.551482:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b899fe8d type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880679.551496:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880679.551500:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 16460 usecs (17 jiffies) 00000100:00000001:3.0:1788880679.551503:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306310674 00000100:00000001:3.0:1788880679.551506:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306315727 00000100:00000001:3.0:1788880679.551508:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306310674 00000100:00000001:3.0:1788880679.551510:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306315727 00000100:00000001:3.0:1788880679.551513:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306310674 00000100:00000001:3.0:1788880679.551516:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306315727 00000100:00000001:3.0:1788880679.551518:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306310674 00000100:00000001:3.0:1788880679.551520:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306315727 00000100:00000001:3.0:1788880679.551529:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880679.551531:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880679.551534:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880679.551538:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880679.551601:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880679.551605:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b899fe8d. 00000400:00000200:1.0:1788880679.551610:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880679.551616:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880679.551622:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000400:00000010:1.0:1788880679.551624:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000100:00000001:1.0:1788880679.551628:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880679.551638:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880679.551647:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880679.551649:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880679.551655:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880679.551656:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880679.552219:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880679.552227:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880679.552230:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880679.552261:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880679.552268:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880679.552280:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0af00 00000400:00000200:1.0:1788880679.552288:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xc25 [1] + 224 00000800:00000001:1.0:1788880679.552294:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880679.552300:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880679.552303:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880679.552306:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880679.552310:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880679.552313:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880679.552321:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880679.552334:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880679.552339:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880679.552342:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880679.552349:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880679.552352:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880679.552356:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880679.552357:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880679.552358:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000400:00000200:0.0:1788880679.552361:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:0.0:1788880679.552362:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:0.0:1788880679.552364:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880679.552376:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880679.552381:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880679.552386:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880679.552387:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880679.552389:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880679.552391:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880679.552392:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880679.552394:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880679.552396:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880679.552401:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=57 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880679.552403:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=57 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880679.552405:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880679.552406:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880679.552408:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880679.552414:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880679.552416:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880679.552418:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880679.552423:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880679.552424:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880679.552426:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880679.552428:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880679.552431:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780676864:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880679.552434:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880679.552435:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880679.552436:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780676864/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880679.552440:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880679.552441:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880679.552443:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000afd3911b. 02000000:00000001:0.0:1788880679.552445:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880679.552446:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880679.552448:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880679.552449:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880679.552451:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 02000000:00000001:0.0:1788880679.552453:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880679.552454:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880679.552456:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:0.0:1788880679.552458:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880679.552458:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880679.552459:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880679.552462:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880679.552463:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880679.552464:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880679.552465:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880679.553057:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880679.553064:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880679.553067:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880679.553070:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880679.553075:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880679.553081:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b000 00000400:00000200:2.0:1788880679.553085:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc35 [1] + 224 00000800:00000001:2.0:1788880679.553089:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880679.553091:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880679.553092:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880679.553095:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880679.553098:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880679.553105:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880679.553111:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880679.553124:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880679.553128:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880679.553131:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880679.553132:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880679.553133:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880679.553137:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880679.553141:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0af80 00000400:00000200:2.0:1788880679.553144:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc2d [1] + 224 00000800:00000001:2.0:1788880679.553147:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880679.553150:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880679.553154:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880679.553158:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880679.553159:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880679.553163:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:3.0:1788880679.553163:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880679.553166:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:2.0:1788880679.553171:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880679.553172:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880679.553175:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880679.553179:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880679.553180:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880679.553184:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880679.553186:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880679.553211:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880679.553213:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880679.553252:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880679.553255:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880679.553258:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880679.553258:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880679.553261:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788880679.553263:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880679.553263:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880679.553265:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880679.553267:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880679.553275:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880679.553281:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=57 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880679.553285:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=14 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880679.553288:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880679.553290:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880679.553290:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880679.553292:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880679.553294:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880679.553298:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00010000:00080000:3.0:1788880679.553298:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880679.553301:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880679.553304:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b884f446. 00000400:00000200:1.0:1788880679.553308:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000007206624 00000400:00000010:1.0:1788880679.553312:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000007206624. 00010000:00000001:3.0:1788880679.553312:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880679.553316:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880679.553317:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880679.553321:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880679.553323:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880679.553328:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880679.553331:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880679.553339:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880679.553341:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880679.553353:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880679.553355:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880679.553358:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880679.553359:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880679.553362:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880679.553363:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880679.553364:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880679.553375:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880679.553379:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880679.553382:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880679.553383:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880679.553389:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=47 v=5 (5 5 5 5) 00000100:00100000:3.0:1788880679.553396:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780677120:192.168.204.111@tcp:400: 00000100:00001000:1.0:1788880679.553397:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=47 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880679.553403:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880679.553405:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880679.553405:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880679.553409:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880679.553409:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00010000:00000001:1.0:1788880679.553414:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880679.553414:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780677120/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880679.553418:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880679.553420:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880679.553423:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880679.553425:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880679.553428:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880679.553431:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:1.0:1788880679.553432:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880679.553433:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000060f4a118. 02000000:00000001:3.0:1788880679.553439:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880679.553441:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880679.553443:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000020:00000040:3.0:1788880679.553444:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880679.553447:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000020:00000001:3.0:1788880679.553448:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880679.553449:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1788880679.553451:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006aca3ff7. 00000100:00000040:1.0:1788880679.553455:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880679.553456:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880679.553459:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880679.553461:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000100:00000001:3.0:1788880679.553466:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00100000:1.0:1788880679.553468:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780676992:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880679.553469:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880679.553473:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880679.553475:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880679.553478:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880679.553479:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880679.553480:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880679.553483:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880679.553484:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880679.553490:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000175072aa x1875776780676992/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880695 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880679.553498:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880679.553500:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880679.553502:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f8c34a6a. 02000000:00000001:1.0:1788880679.553505:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880679.553506:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880679.553507:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880679.553509:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880679.553510:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 02000000:00000001:1.0:1788880679.553512:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880679.553513:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880679.553514:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000175072aa. 00000100:00000001:1.0:1788880679.553517:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880679.553518:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880679.553519:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880679.553523:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880679.553524:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880679.553527:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880679.553528:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880679.614291:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880679.614298:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880679.614304:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880679.614306:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880679.614349:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880679.614358:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880679.614384:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880680.638324:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880680.638331:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880680.638337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880680.638338:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880680.638428:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880680.638435:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880680.638463:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880681.662300:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880681.662306:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880681.662311:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880681.662312:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880681.662363:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880681.662367:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880681.662369:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880682.686363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880682.686369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880682.686374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880682.686374:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880682.686433:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880682.686437:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880682.686438:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880683.710342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880683.710363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880683.710369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880683.710370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880683.710424:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880683.710428:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880683.710431:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880684.670398:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306315727, at 4306315776 00000100:00000040:1.0:1788880684.670398:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306315727, at 4306315776 00000100:00000040:3.0:1788880684.670405:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306315727, at 4306315776 00000100:00000040:2.0:1788880684.670410:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306315727, at 4306315776 00000100:00000010:1.0:1788880684.670419:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000535177f1. 00000100:00000010:0.0:1788880684.670421:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e64ede4. 00000100:00000010:3.0:1788880684.670427:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d2bf5f. 00000100:00000010:2.0:1788880684.670429:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003b9d7f95. 00000100:00000040:3.0:1788880684.670464:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880684.670470:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880684.670481:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880684.670485:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880684.670489:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880684.670492:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880684.670497:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880684.670505:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000088542ca3. 00000100:00000001:3.0:1788880684.670512:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880684.670523:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880684.670533:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880684.670536:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880684.670542:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_00+0] req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880684.670571:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880684.670572:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880684.670575:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880684.670578:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880684.670581:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880684.670583:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880684.670589:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000100:00000040:0.0:1788880684.670592:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:3.0:1788880684.670595:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880684.670599:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880684.670604:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880684.670605:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880684.670610:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880684.670616:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880684.670626:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000010:3.0:1788880684.670627:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b899fe8d. 00000100:00000001:0.0:1788880684.670630:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880684.670636:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880684.670642:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880684.670644:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880684.670649:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880684.670654:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780677248:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880684.670655:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880684.670659:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880684.670666:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880684.670668:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880684.670672:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880684.670679:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000006c62949. 02000000:00000001:0.0:1788880684.670684:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880684.670685:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880684.670688:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880684.670693:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 00000100:00000040:3.0:1788880684.670694:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_01+1] req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880684.670701:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000200:0.0:1788880684.670711:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780677248, portal 25 00000100:00000001:0.0:1788880684.670714:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880684.670715:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000040:0.0:1788880684.670720:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880700 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880684.670721:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880684.670730:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880684.670743:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880684.670747:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000100:00000001:2.0:1788880684.670749:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:3.0:1788880684.670753:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880684.670756:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880684.670765:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880684.670770:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880684.670775:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880684.670776:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:0.0:1788880684.670783:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e9812fb0. 02000000:00000001:3.0:1788880684.670783:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:2.0:1788880684.670784:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880684.670794:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780677248, offset 0 02000000:00000001:3.0:1788880684.670794:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880684.670808:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f92e72a2. 00000400:00000200:0.0:1788880684.670809:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880684.670810:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880684.670817:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880684.670818:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880684.670825:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1788880684.670828:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880684.670829:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880684.670836:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880684.670837:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00100000:2.0:1788880684.670851:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780677376:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880684.670853:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880684.670855:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880684.670861:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880684.670864:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880684.670865:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880684.670871:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880684.670872:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_02+2] req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880684.670873:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880684.670876:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880684.670885:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [365:365:256:4294967295] 192.168.204.111@tcp LPNI seq info [365:365:8:4294967295] 02000000:00000001:2.0:1788880684.670893:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880684.670900:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880684.670910:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000009e54fce. 00000400:00000200:0.0:1788880684.670916:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788880684.670920:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880684.670934:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880684.670934:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a7a7e63a. 00000100:00000001:3.0:1788880684.670940:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880684.670944:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000010:0.0:1788880684.670946:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000004e0f6846. 00000400:00000010:2.0:1788880684.670947:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000040:3.0:1788880684.670949:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880684.670961:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880684.670962:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780677376, portal 10 00000100:00000040:3.0:1788880684.670962:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11653) 00000800:00000200:0.0:1788880684.670965:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880684.670967:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880684.670975:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880684.670975:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880684.670981:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000800:00000200:0.0:1788880684.670986:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:1.0:1788880684.670991:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880684.670994:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000004e0f6846 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:2.0:1788880684.671001:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880684.671003:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880684.671011:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880684.671012:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880684.671022:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306320847 00000100:00000001:1.0:1788880684.671029:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000010:0.0:1788880684.671031:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e64ede4. 00000100:00000001:1.0:1788880684.671033:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880684.671041:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880684.671042:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:2.0:1788880684.671043:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880684.671046:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880684.671051:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880684.671053:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306320847 00000100:00100000:1.0:1788880684.671057:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780677504:192.168.204.111@tcp:400: 00000100:00000010:0.0:1788880684.671060:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e6f9f09c. 00000100:00000040:2.0:1788880684.671061:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880684.671068:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880684.671074:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880684.671076:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306320847 00000400:00000010:2.0:1788880684.671077:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cc594c. 02000000:00000001:1.0:1788880684.671078:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880684.671082:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880684.671086:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e245be24. 02000000:00000001:1.0:1788880684.671089:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880684.671090:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000200:2.0:1788880684.671090:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780677376, offset 0 02000000:00000010:1.0:1788880684.671094:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b31f2b6a. 02000000:00000001:1.0:1788880684.671100:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880684.671105:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b884f446. 00000100:00000001:0.0:1788880684.671106:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306320847 00000400:00000200:2.0:1788880684.671114:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880684.671115:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000455e6647. 00000400:00000010:1.0:1788880684.671122:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000007206624. 00000100:00000001:0.0:1788880684.671125:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000200:1.0:1788880684.671131:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780677504, portal 10 00000100:00000001:0.0:1788880684.671134:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880684.671135:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:3.0:1788880684.671135:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:2.0:1788880684.671137:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880684.671139:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880684.671141:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000800:00000010:3.0:1788880684.671146:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000004e0f6846. 00000100:00000001:0.0:1788880684.671151:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880684.671155:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880684.671157:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:1.0:1788880684.671158:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880684.671159:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880684.671166:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880684.671172:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880684.671175:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880684.671179:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880684.671182:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880684.671185:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880684.671190:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880684.671193:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:1.0:1788880684.671196:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880684.671197:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880684.671199:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880684.671203:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000994ad42e. 00000400:00000200:2.0:1788880684.671209:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:1.0:1788880684.671211:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780677504, offset 0 00000400:00000200:1.0:1788880684.671224:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880684.671228:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [366:366:255:4294967295] 192.168.204.111@tcp LPNI seq info [366:366:7:4294967295] 00000400:00000200:2.0:1788880684.671301:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880684.671308:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880684.671310:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e9812fb0 00000400:00000200:1.0:1788880684.671313:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880684.671314:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000400:00000010:3.0:1788880684.671314:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e9812fb0. 00000400:00000200:1.0:1788880684.671322:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880684.671327:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880684.671330:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:1.0:1788880684.671332:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1788880684.671337:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880684.671340:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:3.0:1788880684.671346:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880700 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880684.671351:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [367:367:254:4294967295] 192.168.204.111@tcp LPNI seq info [367:367:6:4294967295] 00000800:00000200:2.0:1788880684.671361:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880684.671370:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880684.671381:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880684.671389:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880684.671391:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880684.671392:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 240770 usecs (241 jiffies) 00000100:00000001:2.0:1788880684.671398:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306316018 00000100:00000040:3.0:1788880684.671401:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880700 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880684.671405:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880684.671407:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306320847 00000100:00000001:2.0:1788880684.671418:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306316018 00000800:00000010:1.0:1788880684.671419:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000100:00000001:3.0:1788880684.671420:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880684.671424:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306320847 00000100:00000001:3.0:1788880684.671427:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880684.671429:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306316018 00000800:00000200:1.0:1788880684.671434:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880684.671436:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306320847 00000100:00000001:2.0:1788880684.671442:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306316018 00000800:00000200:1.0:1788880684.671445:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880684.671449:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306320847 00000800:00000200:1.0:1788880684.671451:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880684.671454:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880684.671459:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880684.671461:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880684.671466:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 16330 usecs (17 jiffies) 00000100:00000001:2.0:1788880684.671468:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880684.671474:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306315794 00000800:00000200:3.0:1788880684.671474:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880684.671475:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880684.671482:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306320847 00000800:00000010:3.0:1788880684.671482:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000100:00000001:2.0:1788880684.671488:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880684.671490:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306315794 00000100:00000001:2.0:1788880684.671492:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:3.0:1788880684.671492:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880684.671498:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306320847 00000100:00000001:2.0:1788880684.671501:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880684.671502:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306315794 00000100:00000001:2.0:1788880684.671506:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880684.671507:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880684.671509:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306320847 00000100:00000001:2.0:1788880684.671511:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880684.671519:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306315794 00000100:00000001:2.0:1788880684.671521:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880684.671521:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cc594c 00000100:00000001:1.0:1788880684.671524:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306320847 00000400:00000010:3.0:1788880684.671527:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cc594c. 00000100:00000001:1.0:1788880684.671531:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880684.671535:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880684.671538:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880684.671541:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880684.671547:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880684.671557:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880684.671560:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880684.671562:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880684.671565:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880684.671570:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880684.671571:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880684.671576:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880684.671582:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880684.671583:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880684.671589:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880684.671590:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880684.671623:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880684.671625:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:3.0:1788880684.671628:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880684.671630:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880684.671632:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000994ad42e 00000400:00000010:3.0:1788880684.671633:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000994ad42e. 00000100:00000001:3.0:1788880684.671635:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880684.671637:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880684.671646:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880684.671647:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880684.671651:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880684.671652:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880684.672724:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880684.672738:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880684.672741:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880684.672742:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880684.672750:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880684.672762:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0b080 00000400:00000200:1.0:1788880684.672769:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xc3d [1] + 224 00000800:00000001:1.0:1788880684.672776:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880684.672787:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880684.672789:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880684.672793:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880684.672799:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880684.672804:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880684.672813:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880684.672830:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880684.672837:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880684.672840:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880684.672842:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880684.672843:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880684.672846:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880684.672851:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b180 00000400:00000200:1.0:1788880684.672855:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc4d [1] + 224 00000800:00000001:1.0:1788880684.672857:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880684.672861:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880684.672862:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880684.672865:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880684.672876:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880684.672883:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880684.672890:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880684.672893:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880684.672896:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880684.672904:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880684.672908:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880684.672911:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000100:00000001:1.0:1788880684.672912:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:0.0:1788880684.672919:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:0.0:1788880684.672922:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:0.0:1788880684.672925:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880684.672931:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880684.672940:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880684.672948:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880684.672950:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880684.672952:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880684.672954:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880684.672955:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880684.672957:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880684.672958:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880684.672963:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=62 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880684.672966:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=62 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880684.672967:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880684.672968:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880684.672970:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880684.672972:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880684.672979:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880684.672981:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880684.672985:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880684.672990:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880684.672996:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880684.672998:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880684.672999:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880684.673004:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880684.673009:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780677248:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880684.673013:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880684.673015:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880684.673017:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780677248/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880684.673022:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880684.673024:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880684.673026:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000006c62949. 02000000:00000001:0.0:1788880684.673028:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880684.673030:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880684.673031:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880684.673032:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880684.673034:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000088542ca3. 02000000:00000001:0.0:1788880684.673038:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880684.673039:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880684.673041:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:0.0:1788880684.673043:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880684.673044:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880684.673046:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880684.673049:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880684.673051:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880684.673052:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880684.673053:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880684.673059:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880684.673079:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880684.673088:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880684.673091:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880684.673099:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880684.673104:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880684.673107:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:3.0:1788880684.673113:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880684.673116:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788880684.673119:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b100 00000400:00000010:3.0:1788880684.673121:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b884f446. 00000400:00000200:3.0:1788880684.673128:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000007206624 00000400:00000010:3.0:1788880684.673131:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000007206624. 00000100:00000001:3.0:1788880684.673135:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880684.673141:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc45 [1] + 224 00000100:00000200:3.0:1788880684.673147:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880684.673148:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880684.673155:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880684.673162:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880684.673171:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880684.673177:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880684.673183:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880684.673183:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880684.673266:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880684.673298:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880684.673302:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880684.673306:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880684.673309:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880684.673311:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880684.673313:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880684.673315:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880684.673320:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880684.673323:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:3.0:1788880684.673328:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=62 v=5 (5 5 5 5) 00000800:00000001:2.0:1788880684.673329:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880684.673335:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880684.673342:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:3.0:1788880684.673344:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=19 v=5 (1 1 0 0) 00000400:00000001:0.0:1788880684.673348:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880684.673349:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880684.673351:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880684.673354:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000001:0.0:1788880684.673360:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00010000:00080000:3.0:1788880684.673362:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880684.673364:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a7a7e63a. 00000400:00000200:0.0:1788880684.673373:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00010000:00000001:3.0:1788880684.673374:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880684.673377:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:0.0:1788880684.673381:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880684.673386:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880684.673388:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880684.673392:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880684.673398:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880684.673400:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880684.673405:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880684.673409:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880684.673421:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880684.673424:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880684.673426:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880684.673428:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880684.673432:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880684.673432:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880684.673436:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880684.673436:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880684.673438:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880684.673441:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880684.673443:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880684.673447:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=52 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880684.673454:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=52 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880684.673460:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00100000:3.0:1788880684.673462:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780677504:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880684.673464:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880684.673468:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880684.673472:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880684.673472:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880684.673476:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880684.673477:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880684.673478:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880684.673482:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000040:3.0:1788880684.673483:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780677504/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880684.673485:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880684.673494:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880684.673499:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880684.673502:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880684.673506:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000010:3.0:1788880684.673507:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b31f2b6a. 00000100:00000001:0.0:1788880684.673511:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880684.673514:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880684.673515:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:0.0:1788880684.673519:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880684.673519:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880684.673522:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880684.673528:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880684.673531:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f92e72a2. 00000100:00100000:0.0:1788880684.673535:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780677376:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880684.673537:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880684.673539:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880684.673541:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:3.0:1788880684.673548:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880684.673549:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880684.673551:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880684.673552:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880684.673555:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880684.673559:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780677376/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880700 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880684.673562:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880684.673565:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880684.673571:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880684.673572:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880684.673574:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880684.673575:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000009e54fce. 02000000:00000001:0.0:1788880684.673579:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880684.673582:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880684.673582:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880684.673584:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880684.673588:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880684.673589:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b899fe8d. 02000000:00000001:0.0:1788880684.673592:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880684.673593:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880684.673594:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:0.0:1788880684.673597:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880684.673598:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880684.673601:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880684.673604:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880684.673606:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880684.673608:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880684.673609:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880684.734373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880684.734383:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880684.734385:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880684.734388:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880684.734394:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880684.734396:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880684.734397:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880684.734399:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880684.734402:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880684.734403:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880684.734452:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880684.734456:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880684.734458:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880685.758353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880685.758360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880685.758366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880685.758367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880685.758429:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880685.758433:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880685.758436:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880686.782320:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880686.782326:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880686.782331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880686.782332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880686.782373:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880686.782377:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880686.782379:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880687.806326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880687.806331:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880687.806335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880687.806337:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880687.806339:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880687.806341:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880687.806344:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880687.806345:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880687.806346:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880687.806348:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880687.806406:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880687.806410:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880687.806412:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880688.830313:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880688.830320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880688.830326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880688.830327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880688.830370:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880688.830374:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880688.830377:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880689.790333:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880689.790333:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306320847, at 4306320896 00000100:00000040:3.0:1788880689.790333:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306320847, at 4306320896 00000100:00000040:1.0:1788880689.790334:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306320847, at 4306320896 00000100:00000001:0.0:1788880689.790345:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880689.790349:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000455e6647. 00000100:00000010:2.0:1788880689.790351:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e245be24. 00000100:00000010:0.0:1788880689.790353:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000007f91037. 00000100:00000010:1.0:1788880689.790354:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e6f9f09c. 00000020:00000040:0.0:1788880689.790358:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880689.790363:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880689.790366:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880689.790369:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880689.790388:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 00000100:00000001:0.0:1788880689.790393:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880689.790401:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880689.790408:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880689.790410:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880689.790414:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000007f91037] to pc [ptlrpcd_00_03+3] req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880689.790433:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880689.790435:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880689.790438:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880689.790439:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000009ee4c6bf. 00000020:00000040:0.0:1788880689.790441:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880689.790443:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880689.790444:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880689.790445:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880689.790447:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000ac392bfe. 00000100:00000001:3.0:1788880689.790447:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880689.790451:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880689.790451:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880689.790454:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880689.790455:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:3.0:1788880689.790461:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880689.790463:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880689.790464:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880689.790467:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880689.790469:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880689.790473:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000009ee4c6bf] to pc [ptlrpcd_00_00+0] req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880689.790480:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880689.790482:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880689.790486:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880689.790488:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880689.790490:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880689.790493:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880689.790496:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788880689.790496:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780677632:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880689.790501:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880689.790503:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880689.790505:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:0.0:1788880689.790507:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000451fe31a. 00000100:00000001:3.0:1788880689.790507:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880689.790508:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:0.0:1788880689.790510:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880689.790513:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880689.790513:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:1.0:1788880689.790516:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880689.790518:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880689.790519:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880689.790523:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880689.790523:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880689.790529:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000030024c2c. 02000000:00000010:3.0:1788880689.790530:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f8a9d8a8. 00000100:00000001:1.0:1788880689.790531:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880689.790533:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880689.790534:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880689.790535:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880689.790536:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880689.790539:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880689.790540:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880689.790544:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880689.790544:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880689.790545:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000100:00000001:0.0:1788880689.790548:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880689.790551:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880689.790551:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880689.790555:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000040:0.0:1788880689.790556:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000451fe31a] to pc [ptlrpcd_00_01+1] req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880689.790558:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780677760:192.168.204.111@tcp:400: 00000100:00000200:3.0:1788880689.790564:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780677632, portal 25 00000100:00000001:1.0:1788880689.790568:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880689.790570:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1788880689.790573:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880689.790574:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849371264 : -126822860180352 : ffff8ca7c0783480) 00000100:00000001:0.0:1788880689.790577:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880689.790577:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880689.790581:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880689.790582:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880689.790583:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880705 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880689.790586:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000010:1.0:1788880689.790587:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a4a3549f. 00000100:00000001:2.0:1788880689.790587:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880689.790590:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880689.790591:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11658) 02000000:00000001:1.0:1788880689.790592:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880689.790593:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880689.790600:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880689.790601:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008ffd4b4e. 00000100:00000040:0.0:1788880689.790604:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306320847, at 4306320896 00000400:00000010:1.0:1788880689.790609:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000100:00000010:0.0:1788880689.790612:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e64ede4. 00000100:00000001:3.0:1788880689.790613:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:1.0:1788880689.790616:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780677760, portal 10 00000100:00000001:1.0:1788880689.790622:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880689.790622:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880689.790624:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880689.790626:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849358720 : -126822860192896 : ffff8ca7c0780380) 02000000:00000001:2.0:1788880689.790627:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880689.790630:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880689.790632:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880689.790633:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880689.790637:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880689.790638:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780677888:192.168.204.111@tcp:400: 00000400:00000010:3.0:1788880689.790638:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007839347f. 00000100:00000001:2.0:1788880689.790645:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000200:3.0:1788880689.790645:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780677632, offset 0 00000100:00000001:1.0:1788880689.790654:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:2.0:1788880689.790655:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880689.790657:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880689.790658:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880689.790659:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880689.790663:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000087acc4de. 02000000:00000001:2.0:1788880689.790666:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880689.790668:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880689.790670:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a7a7e63a. 00000400:00000010:1.0:1788880689.790673:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000400:00000200:3.0:1788880689.790674:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:2.0:1788880689.790675:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000023cb0718. 00000100:00000200:1.0:1788880689.790680:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780677760, offset 0 00000100:00000200:2.0:1788880689.790681:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780677888, portal 10 00000400:00000200:3.0:1788880689.790682:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880689.790686:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788880689.790688:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880689.790689:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849366784 : -126822860184832 : ffff8ca7c0782300) 00000400:00000200:1.0:1788880689.790694:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880689.790694:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1788880689.790697:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880689.790701:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880689.790709:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788880689.790709:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [368:368:256:4294967295] 192.168.204.111@tcp LPNI seq info [368:368:8:4294967295] 00000100:00000040:2.0:1788880689.790715:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880689.790718:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000200:2.0:1788880689.790722:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780677888, offset 0 00000400:00000200:3.0:1788880689.790726:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:2.0:1788880689.790730:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880689.790735:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880689.790738:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:3.0:1788880689.790741:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f92e72a2. 00000400:00000200:1.0:1788880689.790743:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:3.0:1788880689.790748:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880689.790749:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880689.790754:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880689.790758:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788880689.790758:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880689.790763:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f92e72a2 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880689.790766:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [369:369:255:4294967295] 192.168.204.111@tcp LPNI seq info [369:369:7:4294967295] 00000400:00000200:1.0:1788880689.790776:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880689.790777:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880689.790782:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880689.790782:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306325967 00000400:00000200:2.0:1788880689.790784:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:3.0:1788880689.790788:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005869c8be. 00000400:00000200:2.0:1788880689.790791:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:1.0:1788880689.790792:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000055d7d92. 00000100:00000001:3.0:1788880689.790793:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:2.0:1788880689.790795:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880689.790800:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306325967 00000400:00000200:2.0:1788880689.790801:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:1.0:1788880689.790803:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:3.0:1788880689.790806:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000dc3cc638. 00000400:00000200:2.0:1788880689.790808:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880689.790810:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:1.0:1788880689.790815:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880689.790817:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [370:370:254:4294967295] 192.168.204.111@tcp LPNI seq info [370:370:6:4294967295] 00000100:00000001:3.0:1788880689.790817:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306325967 00000800:00000200:1.0:1788880689.790821:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000055d7d92 type 1, nob 320 niov 1 nkiov 1 00000100:00000010:3.0:1788880689.790822:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e83229b9. 00000100:00000001:3.0:1788880689.790827:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880689.790829:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880689.790830:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306325967 00000100:00000040:1.0:1788880689.790836:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 238571 usecs (239 jiffies) 00000100:00000010:3.0:1788880689.790836:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e62b7132. 00000400:00000200:2.0:1788880689.790837:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880689.790843:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306321135 00000100:00000001:3.0:1788880689.790846:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880689.790848:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306325967 00000800:00000200:2.0:1788880689.790849:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880689.790852:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880689.790854:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306321135 00000800:00000010:2.0:1788880689.790855:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003139f6ba. 00000100:00000001:3.0:1788880689.790856:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880689.790860:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306325967 00000100:00000001:3.0:1788880689.790862:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880689.790863:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880689.790864:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880689.790867:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306321135 00000800:00000010:0.0:1788880689.790870:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f92e72a2. 00000100:00000001:3.0:1788880689.790870:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880689.790871:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306325967 00000800:00000200:2.0:1788880689.790873:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880689.790878:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880689.790878:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306321135 00000800:00000200:2.0:1788880689.790879:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003139f6ba type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880689.790881:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880689.790885:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306325967 00000100:00000001:2.0:1788880689.790885:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880689.790886:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880689.790886:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:2.0:1788880689.790889:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 16203 usecs (17 jiffies) 00000100:00000001:1.0:1788880689.790892:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880689.790894:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007839347f 00000100:00000001:2.0:1788880689.790894:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306320913 00000100:00000001:3.0:1788880689.790894:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880689.790897:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880689.790897:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:0.0:1788880689.790898:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007839347f. 00000100:00000001:3.0:1788880689.790902:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880689.790904:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880689.790904:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306325967 00000100:00000001:3.0:1788880689.790909:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880689.790910:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880705 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880689.790910:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306320913 00000100:00000001:1.0:1788880689.790911:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880689.790916:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306325967 00000100:00000001:1.0:1788880689.790917:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880689.790922:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306320913 00000100:00000001:0.0:1788880689.790924:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880689.790925:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880689.790927:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306325967 00000100:00000001:1.0:1788880689.790928:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:0.0:1788880689.790929:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880705 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880689.790932:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306320913 00000100:00000001:1.0:1788880689.790934:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880689.790937:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880689.790938:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306325967 00000100:00000001:0.0:1788880689.790942:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880689.790942:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880689.790943:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880689.790946:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880689.790948:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880689.790948:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880689.790953:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880689.790955:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880689.790958:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880689.790959:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880689.790961:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880689.790962:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:0.0:1788880689.790964:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880689.790965:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880689.790969:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000055d7d92. 00000100:00000001:2.0:1788880689.790969:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880689.790975:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880689.790978:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880689.790981:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000400:00000010:0.0:1788880689.790982:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:0.0:1788880689.790984:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880689.790986:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880689.790991:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880689.790992:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880689.790996:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880689.790997:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880689.791023:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880689.791024:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003139f6ba. 00000400:00000200:0.0:1788880689.791026:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880689.791028:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880689.791029:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:0.0:1788880689.791031:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:0.0:1788880689.791033:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880689.791034:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880689.791039:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880689.791041:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880689.791045:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880689.791046:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880689.792425:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880689.792435:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880689.792438:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880689.792440:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880689.792447:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880689.792456:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0b200 00000400:00000200:1.0:1788880689.792463:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xc55 [1] + 224 00000800:00000001:1.0:1788880689.792468:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880689.792484:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880689.792486:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880689.792490:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880689.792494:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880689.792498:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880689.792508:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880689.792524:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880689.792536:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880689.792539:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880689.792545:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880689.792548:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880689.792552:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880689.792554:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880689.792555:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:3.0:1788880689.792626:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880689.792628:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880689.792631:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880689.792635:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880689.792645:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880689.792650:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880689.792654:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880689.792657:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880689.792658:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880689.792662:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880689.792662:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880689.792666:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788880689.792667:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880689.792674:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880689.792682:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b280 00000400:00000200:2.0:1788880689.792687:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc59 [1] + 224 00000800:00000001:2.0:1788880689.792690:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880689.792694:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880689.792696:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880689.792702:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880689.792702:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880689.792705:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880689.792708:0:201278:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1788880689.792708:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880689.792715:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880689.792716:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=67 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880689.792723:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=67 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880689.792727:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880689.792729:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880689.792733:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880689.792734:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880689.792744:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880689.792747:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880689.792751:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880689.792765:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880689.792767:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880689.792768:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880689.792770:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880689.792771:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880689.792776:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:3.0:1788880689.792779:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780677632:192.168.204.111@tcp:400: 00000800:00000001:2.0:1788880689.792781:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880689.792784:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880689.792786:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880689.792789:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880689.792795:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880689.792797:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b300 00000100:00000001:1.0:1788880689.792800:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880689.792800:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880689.792805:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880689.792805:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc65 [1] + 224 00000100:00000040:3.0:1788880689.792809:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000007f91037 x1875776780677632/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880689.792814:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880689.792817:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880689.792820:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000001:1.0:1788880689.792821:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:2.0:1788880689.792821:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880689.792824:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008ffd4b4e. 00000400:00000200:2.0:1788880689.792826:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:3.0:1788880689.792827:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000200:1.0:1788880689.792831:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 02000000:00000010:3.0:1788880689.792832:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f8a9d8a8. 00000400:00000010:1.0:1788880689.792836:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000400:00000200:2.0:1788880689.792837:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:3.0:1788880689.792839:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880689.792843:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000001:3.0:1788880689.792843:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880689.792845:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000040:3.0:1788880689.792846:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000200:2.0:1788880689.792851:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880689.792852:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880689.792856:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 00000100:00000200:1.0:1788880689.792857:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880689.792862:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880689.792866:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880689.792869:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000007f91037. 00000100:00000040:2.0:1788880689.792871:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880689.792876:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880689.792879:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880689.792884:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880689.792889:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880689.792891:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880689.792893:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880689.792897:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880689.792898:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880689.792902:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880689.792904:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880689.792907:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880689.792908:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880689.792912:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880689.792916:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880689.792918:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880689.792919:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880689.792921:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880689.792923:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880689.792924:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880689.792927:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880689.792929:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880689.792930:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880689.792931:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a7a7e63a. 00000400:00000200:0.0:1788880689.792935:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000023cb0718 00000100:00001000:1.0:1788880689.792937:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=57 v=5 (5 5 5 5) 00000400:00000010:0.0:1788880689.792942:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000023cb0718. 00000100:00001000:1.0:1788880689.792944:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=57 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880689.792945:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880689.792949:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880689.792949:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880689.792952:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880689.792957:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000200:0.0:1788880689.792961:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880689.792961:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880689.792967:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788880689.792969:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880689.792970:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880689.792971:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880689.792973:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880689.792976:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880689.792976:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880689.792978:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880689.792980:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880689.792980:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880689.792985:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880689.792989:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=67 v=5 (5 5 5 5) 00000100:00000040:1.0:1788880689.792989:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880689.792994:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=24 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880689.793000:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880689.793002:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880689.793004:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880689.793008:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00010000:00080000:0.0:1788880689.793009:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880689.793014:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880689.793017:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880689.793018:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880689.793023:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880689.793024:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880689.793025:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000040:1.0:1788880689.793027:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880689.793028:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880689.793033:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880689.793052:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880689.793054:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880689.793056:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1788880689.793057:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780677760:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880689.793060:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880689.793067:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880689.793070:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880689.793075:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000009ee4c6bf x1875776780677760/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880689.793079:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780677888:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880689.793081:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880689.793084:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880689.793087:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000451fe31a x1875776780677888/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880705 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880689.793092:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880689.793093:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880689.793094:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000087acc4de. 02000000:00000001:0.0:1788880689.793096:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880689.793097:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880689.793098:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880689.793101:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880689.793102:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000030024c2c. 02000000:00000001:0.0:1788880689.793106:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880689.793108:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880689.793110:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000010:0.0:1788880689.793111:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000451fe31a. 02000000:00000001:1.0:1788880689.793114:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880689.793116:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880689.793118:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880689.793118:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a4a3549f. 00000100:00000001:0.0:1788880689.793123:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880689.793124:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880689.793127:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880689.793128:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000020:00000040:1.0:1788880689.793129:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880689.793130:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880689.793134:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880689.793135:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880689.793137:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000ac392bfe. 00000100:00000001:0.0:1788880689.793138:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880689.793141:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880689.793144:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880689.793145:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000009ee4c6bf. 00000100:00000001:1.0:1788880689.793149:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880689.793150:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880689.793152:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880689.793155:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880689.793156:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880689.793158:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880689.793160:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880689.854298:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880689.854303:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880689.854308:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880689.854309:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880689.854356:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880689.854360:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880689.854362:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880690.878336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880690.878342:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880690.878349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880690.878350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880690.878408:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880690.878412:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880690.878415:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880691.902388:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880691.902397:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880691.902406:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880691.902407:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880691.902488:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880691.902497:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880691.902501:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880692.926323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880692.926328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880692.926332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880692.926333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880692.926391:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880692.926395:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880692.926398:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880693.950301:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880693.950307:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880693.950327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880693.950328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880693.950388:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880693.950392:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880693.950395:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880694.910356:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306325967, at 4306326016 00000100:00000040:1.0:1788880694.910356:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306325967, at 4306326016 00000100:00000010:1.0:1788880694.910372:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000dc3cc638. 00000100:00000010:0.0:1788880694.910383:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005869c8be. 00000100:00000040:3.0:1788880694.910385:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306325967, at 4306326016 00000100:00000010:3.0:1788880694.910403:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e62b7132. 00000100:00000040:2.0:1788880694.910495:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880694.910501:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880694.910506:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002084a2b4. 00000020:00000040:2.0:1788880694.910510:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880694.910514:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880694.910516:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880694.910520:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880694.910526:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008eccda10. 00000100:00000001:2.0:1788880694.910536:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880694.910546:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880694.910556:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880694.910558:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880694.910563:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002084a2b4] to pc [ptlrpcd_00_02+2] req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880694.910592:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880694.910595:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880694.910598:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880694.910600:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 00000020:00000040:2.0:1788880694.910601:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880694.910603:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880694.910604:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880694.910605:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880694.910607:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000001:2.0:1788880694.910609:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880694.910612:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880694.910620:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880694.910621:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880694.910623:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_03+3] req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880694.910630:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880694.910631:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880694.910634:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880694.910637:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000884e74b8. 00000020:00000040:2.0:1788880694.910639:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880694.910640:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880694.910641:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880694.910646:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880694.910650:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f9812869. 00000100:00000001:3.0:1788880694.910651:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880694.910653:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880694.910657:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880694.910659:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880694.910666:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880694.910670:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880694.910672:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880694.910678:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000884e74b8] to pc [ptlrpcd_00_00+0] req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880694.910684:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880694.910699:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880694.910701:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880694.910701:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880694.910708:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880694.910711:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:3.0:1788880694.910712:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880694.910715:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11663) 02000000:00000001:3.0:1788880694.910719:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880694.910726:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880694.910727:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880694.910735:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880694.910736:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306325967, at 4306326016 00000100:00100000:3.0:1788880694.910738:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780678144:192.168.204.111@tcp:400: 00000100:00000010:2.0:1788880694.910749:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e83229b9. 00000100:00000001:0.0:1788880694.910750:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880694.910752:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:0.0:1788880694.910759:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880694.910762:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880694.910768:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880694.910772:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:3.0:1788880694.910774:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880694.910775:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880694.910784:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880694.910786:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880694.910788:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880694.910789:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008cdf44b0. 00000100:00100000:0.0:1788880694.910792:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780678272:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880694.910799:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880694.910799:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880694.910804:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880694.910805:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880694.910808:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880694.910812:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:3.0:1788880694.910813:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000100:00000040:1.0:1788880694.910814:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880694.910816:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880694.910824:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880694.910828:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:3.0:1788880694.910829:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 02000000:00000010:0.0:1788880694.910831:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000057e92cbc. 00000100:00000001:1.0:1788880694.910833:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880694.910839:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880694.910840:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:3.0:1788880694.910844:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780678144, portal 10 02000000:00000001:1.0:1788880694.910845:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880694.910848:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 00000100:00000001:3.0:1788880694.910852:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1788880694.910854:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880694.910856:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000400:00000010:0.0:1788880694.910859:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00100000:1.0:1788880694.910862:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780678016:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880694.910868:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000200:0.0:1788880694.910869:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780678272, portal 10 00000100:00000040:3.0:1788880694.910871:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880694.910874:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880694.910876:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1788880694.910882:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880694.910883:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938423808 : -126823771127808 : ffff8ca78a2c4000) 02000000:00000001:1.0:1788880694.910890:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880694.910899:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000937f3d66. 00000100:00000001:3.0:1788880694.910901:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880694.910905:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880694.910907:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880694.910919:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880694.910923:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008ffd4b4e. 00000400:00000010:3.0:1788880694.910926:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cd783db0. 00000400:00000010:1.0:1788880694.910932:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000100:00000200:3.0:1788880694.910938:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780678144, offset 0 00000100:00000001:0.0:1788880694.910939:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:1.0:1788880694.910940:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780678016, portal 25 00000100:00000001:1.0:1788880694.910945:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880694.910946:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880694.910949:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938429184 : -126823771122432 : ffff8ca78a2c5500) 00000400:00000010:0.0:1788880694.910951:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000200:0.0:1788880694.910957:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780678272, offset 0 00000100:00000040:1.0:1788880694.910959:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880710 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880694.910969:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880694.910972:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880694.910977:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880694.910980:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000400:00000200:0.0:1788880694.910984:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880694.910988:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880694.911006:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880694.911009:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880694.911011:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880694.911015:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [371:371:256:4294967295] 192.168.204.111@tcp LPNI seq info [371:371:8:4294967295] 00000400:00000200:0.0:1788880694.911025:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880694.911031:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880694.911036:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000800:00000200:0.0:1788880694.911042:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880694.911059:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880694.911060:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880694.911069:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000100:00000200:1.0:1788880694.911070:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780678016, offset 0 00000400:00000200:3.0:1788880694.911089:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880694.911098:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880694.911101:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880694.911107:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880694.911111:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880694.911113:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880694.911118:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:0.0:1788880694.911122:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 16078 usecs (17 jiffies) 00000400:00000200:3.0:1788880694.911127:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [372:372:255:4294967295] 192.168.204.111@tcp LPNI seq info [372:372:7:4294967295] 00000100:00000001:0.0:1788880694.911131:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306326033 00000100:00000010:0.0:1788880694.911140:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e64ede4. 00000100:00000001:0.0:1788880694.911148:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000400:00000200:3.0:1788880694.911152:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880694.911156:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306326033 00000800:00000200:3.0:1788880694.911163:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880694.911164:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000751d0eed. 00000400:00000200:1.0:1788880694.911166:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880694.911173:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000400:00000200:1.0:1788880694.911174:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:3.0:1788880694.911174:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b899fe8d. 00000100:00000001:0.0:1788880694.911184:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306326033 00000400:00000200:1.0:1788880694.911186:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:0.0:1788880694.911188:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a682425c. 00000800:00000200:3.0:1788880694.911193:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880694.911195:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000400:00000200:1.0:1788880694.911195:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880694.911207:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880694.911209:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306326033 00000800:00000200:3.0:1788880694.911216:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:0.0:1788880694.911217:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e602733. 00000400:00000200:1.0:1788880694.911225:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [373:373:254:4294967295] 192.168.204.111@tcp LPNI seq info [373:373:6:4294967295] 00000800:00000200:3.0:1788880694.911226:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b899fe8d type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880694.911227:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880694.911264:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880694.911265:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880694.911269:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880694.911275:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880694.911276:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 236411 usecs (237 jiffies) 00000100:00000001:0.0:1788880694.911278:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880694.911285:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880694.911288:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880694.911289:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306326254 00000800:00000010:1.0:1788880694.911290:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000800:00000200:1.0:1788880694.911307:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880694.911308:0:201288:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, simply extend its deadline from 4306326033 00000100:00000001:0.0:1788880694.911309:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880694.911318:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306326254 00000800:00000200:1.0:1788880694.911319:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880694.911321:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:1.0:1788880694.911327:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880694.911330:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880694.911331:0:201288:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, simply extend its deadline from 4306326033 00000100:00000001:1.0:1788880694.911332:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880694.911335:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880694.911346:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880694.911347:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306326254 00000100:00000001:1.0:1788880694.911348:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306331088 00000100:00000001:0.0:1788880694.911349:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880694.911366:0:201288:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, simply extend its deadline from 4306326033 00000100:00000001:1.0:1788880694.911368:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306326254 00000100:00000001:0.0:1788880694.911374:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880694.911379:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880694.911382:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306331088 00000100:00000001:3.0:1788880694.911383:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306326254 00000100:00000001:1.0:1788880694.911391:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306326254 00000100:00000001:3.0:1788880694.911395:0:201288:0:(niobuf.c:685:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, simply extend its deadline from 4306326033 00000100:00000001:1.0:1788880694.911398:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306331088 00000100:00000001:3.0:1788880694.911407:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880694.911408:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306326254 00000100:00000001:3.0:1788880694.911420:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880694.911421:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306331088 00000100:00000001:1.0:1788880694.911429:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306326254 00000100:00000001:3.0:1788880694.911432:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880694.911434:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880694.911437:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880694.911438:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880694.911448:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880694.911451:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880694.911453:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880694.911457:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880694.911458:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880694.911460:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880694.911464:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880694.911465:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880694.911468:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880694.911472:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880694.911474:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880694.911476:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880694.911484:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880694.911486:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880694.911581:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880694.911584:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000400:00000200:2.0:1788880694.911589:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880694.911594:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880694.911599:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:2.0:1788880694.911601:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:2.0:1788880694.911606:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880694.911609:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880694.911617:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880694.911619:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880694.911623:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880694.911624:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880694.911634:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880694.911641:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b899fe8d. 00000400:00000200:2.0:1788880694.911645:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880694.911648:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880694.911651:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cd783db0 00000400:00000010:2.0:1788880694.911652:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cd783db0. 00000100:00000001:2.0:1788880694.911655:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880694.911658:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880694.911665:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880694.911667:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880694.911673:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880694.911674:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880694.911730:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880694.911733:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000400:00000200:2.0:1788880694.911737:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880694.911740:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880694.911743:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:2.0:1788880694.911744:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:2.0:1788880694.911746:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880694.911749:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880710 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880694.911756:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880694.911757:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880710 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880694.911762:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880694.911763:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880694.913823:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880694.913834:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880694.913837:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880694.913839:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880694.913848:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880694.913860:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b400 00000400:00000200:1.0:1788880694.913867:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc6d [1] + 224 00000800:00000001:1.0:1788880694.913873:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880694.913877:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880694.913880:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880694.913883:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880694.913888:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880694.913893:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880694.913904:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880694.913918:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880694.913926:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880694.913931:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880694.913934:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880694.913936:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880694.913939:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880694.913942:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880694.913943:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880694.913949:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:1.0:1788880694.913952:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b480 00000400:00000001:3.0:1788880694.913953:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880694.913956:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:3.0:1788880694.913960:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880694.913961:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880694.913964:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788880694.913969:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc71 [1] + 224 00000100:00000200:3.0:1788880694.913970:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880694.913980:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1788880694.913986:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880694.913989:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880694.913994:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880694.914000:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880694.914003:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880694.914005:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880694.914008:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880694.914011:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:1.0:1788880694.914012:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880694.914014:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880694.914018:0:201280:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1788880694.914018:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880694.914024:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880694.914027:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=62 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880694.914035:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=62 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880694.914038:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880694.914039:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880694.914040:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880694.914042:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880694.914045:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880694.914047:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880694.914049:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880694.914052:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880694.914059:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880694.914071:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880694.914113:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880694.914118:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880694.914122:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880694.914156:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880694.914159:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880694.914163:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880694.914164:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880694.914166:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000400:00000200:0.0:1788880694.914170:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:0.0:1788880694.914172:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:0.0:1788880694.914174:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880694.914179:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880694.914183:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880694.914187:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880694.914188:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880694.914195:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880694.914201:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880694.914205:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780678144:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880694.914216:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880694.914218:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880694.914218:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880694.914222:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880694.914224:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880694.914225:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880694.914226:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880694.914228:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880694.914231:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:0.0:1788880694.914256:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880694.914261:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=72 v=5 (5 5 5 5) 00000800:00000001:2.0:1788880694.914265:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880694.914265:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880694.914266:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=29 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880694.914272:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880694.914272:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880694.914276:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880694.914278:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00010000:00000001:0.0:1788880694.914280:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:3.0:1788880694.914280:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780678144/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00080000:0.0:1788880694.914288:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880694.914293:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0b380 00000100:00000001:3.0:1788880694.914295:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880694.914298:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880694.914301:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008cdf44b0. 00010000:00000001:0.0:1788880694.914303:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880694.914305:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880694.914307:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000020:00000001:3.0:1788880694.914308:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:0.0:1788880694.914310:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000020:00000040:3.0:1788880694.914312:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000200:2.0:1788880694.914313:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xc79 [1] + 224 00000020:00000001:3.0:1788880694.914316:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880694.914319:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 00000100:00000001:0.0:1788880694.914322:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000800:00000001:2.0:1788880694.914323:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788880694.914324:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880694.914325:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880694.914327:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880694.914329:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 00000800:00000001:2.0:1788880694.914332:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880694.914332:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:0.0:1788880694.914333:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880694.914335:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880694.914338:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880694.914339:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880694.914348:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880694.914349:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880694.914350:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880694.914350:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880694.914355:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880694.914355:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880694.914356:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880694.914359:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880694.914360:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880694.914363:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880694.914369:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880694.914385:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880694.914387:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780678272:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880694.914396:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880694.914401:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880694.914405:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000884e74b8 x1875776780678272/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880694.914412:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880694.914412:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788880694.914414:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880694.914418:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000057e92cbc. 00000800:00000001:2.0:1788880694.914420:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:0.0:1788880694.914421:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880694.914424:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000800:00000001:2.0:1788880694.914425:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:0.0:1788880694.914426:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880694.914431:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880694.914433:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f9812869. 02000000:00000001:0.0:1788880694.914435:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880694.914437:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880694.914439:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000884e74b8. 00000100:00000001:1.0:1788880694.914439:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880694.914442:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880694.914443:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880694.914445:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880694.914448:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880694.914449:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880694.914452:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880694.914453:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:1.0:1788880694.914454:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008ffd4b4e. 00000100:00000001:0.0:1788880694.914456:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880694.914459:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000100:00000001:0.0:1788880694.914460:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880694.914464:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880694.914464:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:1.0:1788880694.914470:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880694.914476:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880694.914485:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880694.914492:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880694.914493:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880694.914494:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880694.914496:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880694.914497:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880694.914498:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880694.914500:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880694.914503:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=72 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880694.914505:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=72 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880694.914507:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880694.914508:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880694.914510:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880694.914511:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880694.914513:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880694.914516:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880694.914534:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880694.914535:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880694.914537:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880694.914540:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880694.914545:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780678016:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880694.914548:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880694.914549:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880694.914550:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002084a2b4 x1875776780678016/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880710 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880694.914554:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880694.914555:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880694.914556:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000937f3d66. 02000000:00000001:1.0:1788880694.914558:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880694.914559:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880694.914560:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880694.914561:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880694.914562:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008eccda10. 02000000:00000001:1.0:1788880694.914563:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880694.914564:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880694.914565:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002084a2b4. 00000100:00000001:1.0:1788880694.914566:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880694.914567:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880694.914568:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880694.914570:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880694.914571:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880694.914572:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880694.914573:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880694.974341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880694.974348:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880694.974351:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880694.974354:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880694.974359:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880694.974378:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880694.974380:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880694.974382:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880694.974386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880694.974388:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880694.974448:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880694.974451:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880694.974454:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880695.998370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880695.998380:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880695.998388:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880695.998389:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880695.998456:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880695.998479:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880695.998484:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880697.022318:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880697.022328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880697.022336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880697.022337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880697.022425:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880697.022436:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880697.022440:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880698.046368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880698.046374:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880698.046383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880698.046386:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880698.046389:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880698.046392:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880698.046395:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880698.046399:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880698.046401:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880698.046404:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880698.046507:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880698.046518:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880698.046521:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880699.070391:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880699.070399:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880699.070406:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880699.070408:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880699.070503:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880699.070510:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880699.070515:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880700.030333:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306331088, at 4306331136 00000100:00000040:1.0:1788880700.030333:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880700.030333:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306331088, at 4306331136 00000100:00000040:3.0:1788880700.030342:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306331088, at 4306331136 00000100:00000001:1.0:1788880700.030343:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880700.030348:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e64ede4. 00000100:00000010:2.0:1788880700.030349:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a682425c. 00000100:00000010:1.0:1788880700.030351:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002b387587. 00000100:00000010:3.0:1788880700.030352:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e602733. 00000020:00000040:1.0:1788880700.030357:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880700.030361:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880700.030363:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880700.030367:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880700.030372:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5b428a0. 00000100:00000001:1.0:1788880700.030379:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880700.030388:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880700.030410:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880700.030413:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880700.030420:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002b387587] to pc [ptlrpcd_00_01+1] req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880700.030448:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880700.030451:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880700.030455:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880700.030459:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:1.0:1788880700.030459:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e2be23b2. 00000020:00000040:1.0:1788880700.030462:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880700.030465:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880700.030466:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880700.030467:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880700.030468:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880700.030469:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880700.030470:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:0.0:1788880700.030475:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880700.030483:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880700.030485:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880700.030488:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880700.030489:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 02000000:00000001:0.0:1788880700.030493:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880700.030494:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880700.030497:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880700.030500:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880700.030502:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880700.030509:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780678400:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880700.030511:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880700.030513:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880700.030514:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880700.030519:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e2be23b2] to pc [ptlrpcd_00_02+2] req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880700.030520:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880700.030523:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880700.030526:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880700.030532:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cadd275. 02000000:00000001:0.0:1788880700.030534:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880700.030539:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 00000400:00000010:0.0:1788880700.030545:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000200:0.0:1788880700.030550:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780678400, portal 25 00000100:00000001:1.0:1788880700.030551:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880700.030552:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:1.0:1788880700.030555:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880700.030556:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938057728 : -126823771493888 : ffff8ca78a26aa00) 00000100:00000001:1.0:1788880700.030563:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000040:0.0:1788880700.030565:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880716 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880700.030568:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002980f9ff. 00000100:00000001:3.0:1788880700.030571:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:1.0:1788880700.030574:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880700.030576:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880700.030578:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880700.030579:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880700.030580:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880700.030581:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880700.030582:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880700.030583:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880700.030586:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880700.030590:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880700.030591:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:1.0:1788880700.030591:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000400:00000010:0.0:1788880700.030603:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 00000100:00000001:1.0:1788880700.030603:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880700.030604:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880700.030608:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880700.030609:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780678400, offset 0 00000100:00000040:1.0:1788880700.030612:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:3.0:1788880700.030612:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880700.030616:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880700.030621:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880700.030623:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880700.030630:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:3.0:1788880700.030631:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780678528:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880700.030632:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880700.030639:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:1.0:1788880700.030639:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002980f9ff] to pc [ptlrpcd_00_03+3] req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880700.030639:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880700.030643:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880700.030647:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:3.0:1788880700.030647:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880700.030653:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880700.030655:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880700.030659:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000010:3.0:1788880700.030659:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f8a9d8a8. 00000100:00000001:1.0:1788880700.030660:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880700.030665:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880700.030665:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880700.030668:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788880700.030671:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:3.0:1788880700.030673:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000400:00000200:0.0:1788880700.030677:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [374:374:256:4294967295] 192.168.204.111@tcp LPNI seq info [374:374:8:4294967295] 00000100:00000040:1.0:1788880700.030677:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11668) 00000100:00000001:2.0:1788880700.030679:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880700.030680:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000001:2.0:1788880700.030683:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880700.030686:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:3.0:1788880700.030687:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780678528, portal 10 00000100:00000040:1.0:1788880700.030692:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306331088, at 4306331136 00000100:00000040:2.0:1788880700.030692:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880700.030697:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880700.030698:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000010:1.0:1788880700.030700:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000751d0eed. 00000100:00000001:3.0:1788880700.030701:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055936 : -126823771495680 : ffff8ca78a26a300) 00000800:00000200:0.0:1788880700.030710:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880700.030713:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880700.030715:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000010:0.0:1788880700.030716:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000100:00000001:2.0:1788880700.030718:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880700.030722:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880700.030725:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880700.030726:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:2.0:1788880700.030729:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880700.030729:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00100000:2.0:1788880700.030735:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780678656:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880700.030736:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880700.030737:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880700.030739:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:0.0:1788880700.030741:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880700.030741:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:3.0:1788880700.030747:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007fe20b6f. 02000000:00000001:2.0:1788880700.030751:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880700.030752:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780678528, offset 0 02000000:00000001:2.0:1788880700.030755:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880700.030759:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880700.030760:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000014e021ca. 00000400:00000200:3.0:1788880700.030762:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880700.030763:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880700.030765:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306336207 00000100:00000010:0.0:1788880700.030768:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e64ede4. 00000400:00000010:2.0:1788880700.030769:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000edc7a681. 00000400:00000200:3.0:1788880700.030772:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880700.030773:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000010:2.0:1788880700.030773:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000006bff322d. 00000100:00000200:2.0:1788880700.030778:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780678656, portal 10 00000100:00000001:0.0:1788880700.030779:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306336207 00000400:00000200:3.0:1788880700.030780:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880700.030782:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880700.030784:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938061312 : -126823771490304 : ffff8ca78a26b800) 00000400:00000200:3.0:1788880700.030786:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:0.0:1788880700.030791:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000a4f85af. 00000400:00000200:3.0:1788880700.030791:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1788880700.030795:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880700.030797:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:3.0:1788880700.030799:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880700.030801:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306336207 00000400:00000200:3.0:1788880700.030807:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [375:375:255:4294967295] 192.168.204.111@tcp LPNI seq info [375:375:7:4294967295] 00000100:00000001:2.0:1788880700.030808:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:0.0:1788880700.030809:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009a4dee8c. 00000100:00000001:0.0:1788880700.030813:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000040:2.0:1788880700.030814:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880700.030817:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ac9b6c42. 00000100:00000001:0.0:1788880700.030819:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306336207 00000100:00000200:2.0:1788880700.030820:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780678656, offset 0 00000100:00000010:0.0:1788880700.030823:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000506ae1c0. 00000100:00000001:0.0:1788880700.030827:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:2.0:1788880700.030827:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880700.030831:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880700.030831:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880700.030837:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880700.030840:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880700.030843:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880700.030846:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880700.030847:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880700.030849:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:3.0:1788880700.030849:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b899fe8d. 00000400:00000200:2.0:1788880700.030850:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880700.030853:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880700.030857:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880700.030858:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880700.030860:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:3.0:1788880700.030861:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880700.030862:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880700.030863:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [376:376:254:4294967295] 192.168.204.111@tcp LPNI seq info [376:376:6:4294967295] 00000100:00000001:0.0:1788880700.030865:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880700.030868:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880700.030871:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880700.030873:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:3.0:1788880700.030873:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880700.030878:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880700.030879:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b899fe8d type 1, nob 320 niov 1 nkiov 1 00000800:00000010:1.0:1788880700.030880:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000100:00000001:3.0:1788880700.030886:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880700.030889:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:2.0:1788880700.030891:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880700.030893:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 234300 usecs (235 jiffies) 00000800:00000010:2.0:1788880700.030896:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5d1e8ed. 00000400:00000200:1.0:1788880700.030899:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880700.030899:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306331371 00000800:00000200:2.0:1788880700.030905:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880700.030906:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306336207 00000400:00000200:1.0:1788880700.030908:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8884ddc 00000100:00000001:3.0:1788880700.030911:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306331371 00000400:00000010:1.0:1788880700.030913:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8884ddc. 00000800:00000200:2.0:1788880700.030916:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880700.030917:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306336207 00000100:00000001:1.0:1788880700.030920:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:2.0:1788880700.030922:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5d1e8ed type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880700.030922:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306331371 00000100:00000200:1.0:1788880700.030927:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880716 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880700.030927:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306336207 00000100:00000001:2.0:1788880700.030933:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880700.030933:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306331371 00000100:00000001:3.0:1788880700.030939:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306336207 00000100:00000040:2.0:1788880700.030940:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 15970 usecs (16 jiffies) 00000100:00000001:2.0:1788880700.030946:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306331152 00000100:00000001:3.0:1788880700.030946:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880700.030947:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880700.030950:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880700.030953:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880716 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880700.030955:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306336207 00000100:00000001:3.0:1788880700.030956:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880700.030959:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306331152 00000100:00000001:2.0:1788880700.030963:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306336207 00000100:00000001:3.0:1788880700.030966:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880700.030967:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306331152 00000100:00000001:2.0:1788880700.030971:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306336207 00000100:00000001:2.0:1788880700.030974:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306331152 00000100:00000001:2.0:1788880700.030984:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306336207 00000100:00000001:1.0:1788880700.030987:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880700.030987:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880700.030989:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880700.030992:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880700.030993:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880700.030995:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880700.031000:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880700.031002:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880700.031004:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880700.031005:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880700.031006:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880700.031008:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880700.031015:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880700.031019:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b899fe8d. 00000400:00000200:1.0:1788880700.031023:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880700.031033:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880700.031036:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007fe20b6f 00000400:00000010:1.0:1788880700.031038:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007fe20b6f. 00000100:00000001:1.0:1788880700.031042:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880700.031045:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880700.031053:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880700.031055:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880700.031062:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880700.031064:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880700.031070:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880700.031072:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880700.031074:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880700.031076:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880700.031078:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880700.031081:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880700.031081:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880700.031085:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5d1e8ed. 00000400:00000200:1.0:1788880700.031090:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880700.031094:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880700.031096:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ac9b6c42 00000400:00000010:1.0:1788880700.031098:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ac9b6c42. 00000100:00000001:1.0:1788880700.031101:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880700.031104:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880700.031110:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880700.031113:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880700.031119:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880700.031120:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880700.032676:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880700.032688:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880700.032691:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880700.032693:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880700.032699:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880700.032707:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b580 00000400:00000200:1.0:1788880700.032712:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc8d [1] + 224 00000800:00000001:1.0:1788880700.032728:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880700.032732:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880700.032734:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880700.032736:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880700.032739:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880700.032743:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880700.032749:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880700.032759:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880700.032764:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880700.032765:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880700.032851:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880700.032853:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880700.032858:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880700.032860:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880700.032861:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:3.0:1788880700.032864:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880700.032866:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880700.032869:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880700.032872:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880700.032880:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880700.032881:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880700.032888:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880700.032890:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880700.032892:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880700.032897:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880700.032904:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0b500 00000400:00000200:2.0:1788880700.032908:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xc85 [1] + 224 00000800:00000001:2.0:1788880700.032912:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880700.032916:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880700.032918:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880700.032924:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880700.032931:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880700.032932:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880700.032935:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880700.032937:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880700.032939:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880700.032943:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880700.032952:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880700.032955:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880700.032963:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880700.032963:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880700.032973:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=68 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880700.032989:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=68 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880700.032991:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880700.032992:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880700.032995:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880700.032998:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:3.0:1788880700.032998:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880700.033001:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880700.033003:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880700.033006:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880700.033010:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880700.033013:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880700.033015:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880700.033020:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880700.033022:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880700.033026:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880700.033030:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880700.033032:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880700.033036:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000400:00000200:0.0:1788880700.033042:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000100:00000001:3.0:1788880700.033042:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880700.033045:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:3.0:1788880700.033047:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880700.033049:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880700.033050:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880700.033056:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880700.033058:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880700.033080:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880700.033082:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780678528:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880700.033092:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880700.033094:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880700.033095:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880700.033096:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880700.033099:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880700.033100:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880700.033101:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880700.033102:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880700.033104:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880700.033107:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=78 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880700.033109:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=78 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880700.033111:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880700.033112:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880700.033113:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880700.033115:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880700.033116:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880700.033119:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880700.033134:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880700.033136:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880700.033137:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880700.033140:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880700.033146:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780678400:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880700.033149:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880700.033151:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880700.033153:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002b387587 x1875776780678400/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880700.033158:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880700.033160:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880700.033162:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cadd275. 02000000:00000001:0.0:1788880700.033167:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880700.033169:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880700.033171:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880700.033173:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880700.033174:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5b428a0. 02000000:00000001:0.0:1788880700.033178:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880700.033181:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880700.033183:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002b387587. 00000100:00000040:3.0:1788880700.033183:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e2be23b2 x1875776780678528/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880700.033194:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880700.033195:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000800:00000001:1.0:1788880700.033197:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880700.033198:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880700.033198:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880700.033202:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f8a9d8a8. 00000100:00000001:0.0:1788880700.033203:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880700.033206:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880700.033209:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000800:00000001:1.0:1788880700.033210:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880700.033211:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880700.033214:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880700.033214:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000800:00000001:1.0:1788880700.033215:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880700.033217:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880700.033219:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880700.033256:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880700.033264:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b600 00000400:00000200:1.0:1788880700.033269:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xc95 [1] + 224 00000800:00000001:1.0:1788880700.033274:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000040:3.0:1788880700.033274:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880700.033278:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880700.033280:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 00000800:00000001:1.0:1788880700.033281:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880700.033282:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880700.033285:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000200:1.0:1788880700.033286:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000010:3.0:1788880700.033287:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e2be23b2. 00000100:00000001:3.0:1788880700.033291:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000400:00000200:1.0:1788880700.033294:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880700.033294:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880700.033295:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880700.033300:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880700.033300:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880700.033303:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880700.033304:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880700.033305:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880700.033313:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880700.033322:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880700.033333:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880700.033337:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880700.033339:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880700.033391:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880700.033393:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880700.033396:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880700.033397:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880700.033398:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000edc7a681. 00000400:00000200:2.0:1788880700.033401:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000006bff322d 00000400:00000010:2.0:1788880700.033402:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000006bff322d. 00000100:00000001:2.0:1788880700.033403:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880700.033406:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880700.033412:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880700.033416:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880700.033417:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880700.033418:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880700.033419:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880700.033420:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880700.033421:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880700.033422:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880700.033425:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=78 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880700.033427:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=35 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880700.033430:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880700.033431:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880700.033432:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880700.033435:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880700.033443:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880700.033452:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880700.033453:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880700.033455:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880700.033456:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880700.033459:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880700.033465:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880700.033467:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880700.033468:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880700.033471:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880700.033478:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780678656:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880700.033481:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880700.033483:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880700.033485:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002980f9ff x1875776780678656/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880716 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880700.033491:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880700.033493:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880700.033495:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000014e021ca. 02000000:00000001:2.0:1788880700.033498:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880700.033499:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880700.033500:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880700.033502:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880700.033503:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 02000000:00000001:2.0:1788880700.033506:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880700.033507:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880700.033508:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002980f9ff. 00000100:00000001:2.0:1788880700.033510:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880700.033512:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880700.033513:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880700.033516:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880700.033517:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880700.033519:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880700.033525:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880700.094326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880700.094333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880700.094337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880700.094338:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880700.094388:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880700.094391:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880700.094393:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880701.118290:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880701.118298:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880701.118304:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880701.118305:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880701.118354:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880701.118357:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880701.118359:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880702.142344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880702.142351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880702.142357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880702.142359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880702.142373:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880702.142376:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880702.142393:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880703.166337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880703.166343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880703.166348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880703.166349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880703.166392:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880703.166409:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880703.166411:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880704.190740:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880704.190749:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880704.190756:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880704.190758:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880704.190837:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880704.190841:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880704.190844:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880705.150375:0:187007:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306336207, at 4306336256 00000100:00000040:1.0:1788880705.150375:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306336207, at 4306336256 00000100:00000040:2.0:1788880705.150375:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306336207, at 4306336256 00000100:00000010:1.0:1788880705.150397:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000a4f85af. 00000100:00000010:0.0:1788880705.150399:0:187007:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e64ede4. 00000100:00000010:2.0:1788880705.150404:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009a4dee8c. 00000100:00000040:1.0:1788880705.150417:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880705.150423:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880705.150431:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002980f9ff. 00000020:00000040:1.0:1788880705.150436:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880705.150442:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880705.150446:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880705.150453:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880705.150455:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306336207, at 4306336256 02000000:00000010:1.0:1788880705.150463:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000010:3.0:1788880705.150465:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000506ae1c0. 00000100:00000001:1.0:1788880705.150473:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880705.150490:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880705.150501:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880705.150504:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880705.150535:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002980f9ff] to pc [ptlrpcd_00_00+0] req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880705.150564:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880705.150567:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880705.150571:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880705.150575:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e2be23b2. 00000020:00000040:1.0:1788880705.150578:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880705.150581:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880705.150583:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880705.150585:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880705.150587:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880705.150589:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:2.0:1788880705.150593:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880705.150594:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880705.150597:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880705.150600:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880705.150601:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880705.150605:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880705.150606:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880705.150618:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880705.150620:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880705.150624:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880705.150628:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e2be23b2] to pc [ptlrpcd_00_01+1] req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880705.150629:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880705.150638:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880705.150642:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880705.150645:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880705.150648:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880705.150651:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780678784:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880705.150653:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880705.150660:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880705.150661:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880705.150665:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002b387587. 00000020:00000040:1.0:1788880705.150668:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:2.0:1788880705.150668:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880705.150673:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880705.150674:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880705.150675:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880705.150679:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880705.150679:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880705.150685:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5b428a0. 00000100:00000001:1.0:1788880705.150689:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880705.150695:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000010:2.0:1788880705.150698:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000744e90e2. 02000000:00000001:2.0:1788880705.150703:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880705.150711:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000edc7a681. 00000100:00000001:0.0:1788880705.150719:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880705.150720:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000400:00000010:2.0:1788880705.150720:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000006bff322d. 00000100:00000001:1.0:1788880705.150723:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880705.150730:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880705.150732:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002b387587] to pc [ptlrpcd_00_02+2] req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880705.150734:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780678784, portal 25 00000100:00000001:0.0:1788880705.150735:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880705.150743:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880705.150747:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938061312 : -126823771490304 : ffff8ca78a26b800) 00000100:00000040:0.0:1788880705.150753:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880705.150761:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880721 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880705.150766:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880705.150779:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880705.150782:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880705.150787:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880705.150787:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880705.150795:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880705.150797:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880705.150797:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880705.150801:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880705.150808:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11673) 00000100:00000040:3.0:1788880705.150809:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880705.150812:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880705.150814:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:2.0:1788880705.150818:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a3b2cf4a. 02000000:00000001:0.0:1788880705.150821:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880705.150823:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880705.150825:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780678784, offset 0 00000100:00000001:1.0:1788880705.150829:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880705.150831:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880705.150846:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780678912:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880705.150849:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:2.0:1788880705.150851:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880705.150853:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880705.150863:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880705.150863:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880705.150868:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880705.150873:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880705.150873:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880705.150889:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880705.150891:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880705.150899:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00100000:3.0:1788880705.150899:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780679040:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880705.150900:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880705.150908:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880705.150912:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880705.150912:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000010:0.0:1788880705.150913:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000bef544e8. 02000000:00000001:3.0:1788880705.150915:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880705.150919:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1788880705.150923:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880705.150923:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880705.150929:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000010:3.0:1788880705.150930:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000082966425. 02000000:00000001:3.0:1788880705.150933:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880705.150941:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [377:377:256:4294967295] 192.168.204.111@tcp LPNI seq info [377:377:8:4294967295] 00000400:00000010:3.0:1788880705.150943:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000400:00000010:0.0:1788880705.150947:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 00000400:00000010:3.0:1788880705.150948:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000200:3.0:1788880705.150953:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780679040, portal 10 00000100:00000001:3.0:1788880705.150956:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880705.150959:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938057728 : -126823771493888 : ffff8ca78a26aa00) 00000400:00000010:0.0:1788880705.150960:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000400:00000200:2.0:1788880705.150964:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:0.0:1788880705.150974:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780678912, portal 10 00000100:00000040:3.0:1788880705.150976:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880705.150980:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880705.150987:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000010:2.0:1788880705.150990:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e5360691. 00000100:00000001:0.0:1788880705.150992:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055936 : -126823771495680 : ffff8ca78a26a300) 00000100:00000001:3.0:1788880705.150995:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880705.151001:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880705.151004:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:3.0:1788880705.151005:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8336eb2. 00000100:00000040:0.0:1788880705.151010:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880705.151012:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780679040, offset 0 00000800:00000200:2.0:1788880705.151022:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880705.151028:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880705.151029:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e5360691 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880705.151046:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880705.151048:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880705.151050:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880705.151052:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:0.0:1788880705.151059:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880705.151059:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880705.151060:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306341327 00000400:00000200:3.0:1788880705.151071:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:2.0:1788880705.151072:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a3bf26bf. 00000400:00000010:0.0:1788880705.151074:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e0ace8ca. 00000400:00000200:3.0:1788880705.151076:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880705.151080:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000200:0.0:1788880705.151086:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780678912, offset 0 00000400:00000200:3.0:1788880705.151087:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [378:378:255:4294967295] 192.168.204.111@tcp LPNI seq info [378:378:7:4294967295] 00000100:00000001:2.0:1788880705.151088:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306341327 00000100:00000010:2.0:1788880705.151094:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000007e595eb4. 00000100:00000001:2.0:1788880705.151098:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:2.0:1788880705.151102:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306341327 00000400:00000200:3.0:1788880705.151106:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:2.0:1788880705.151111:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000668fd8a. 00000100:00000001:2.0:1788880705.151117:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:3.0:1788880705.151121:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880705.151122:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306341327 00000400:00000200:0.0:1788880705.151126:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880705.151129:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005c6a0f3f. 00000800:00000010:3.0:1788880705.151131:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000001:2.0:1788880705.151133:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880705.151136:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880705.151137:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:3.0:1788880705.151137:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880705.151141:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880705.151144:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e5360691. 00000100:00000001:2.0:1788880705.151147:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880705.151148:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788880705.151148:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880705.151153:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:3.0:1788880705.151160:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880705.151164:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880705.151167:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880705.151169:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880705.151172:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880705.151175:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788880705.151177:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 15853 usecs (16 jiffies) 00000100:00000001:2.0:1788880705.151180:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880705.151182:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880705.151185:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306336272 00000100:00000001:2.0:1788880705.151191:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880705.151195:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880705.151197:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880705.151200:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880705.151204:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880705.151208:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880705.151215:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [379:379:254:4294967295] 192.168.204.111@tcp LPNI seq info [379:379:6:4294967295] 00000100:00000001:3.0:1788880705.151255:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306341327 00000100:00000001:3.0:1788880705.151260:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306336273 00000400:00000200:0.0:1788880705.151262:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880705.151262:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306341327 00000100:00000001:3.0:1788880705.151273:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306336273 00000100:00000001:3.0:1788880705.151280:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306341327 00000400:00000200:1.0:1788880705.151282:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a3b2cf4a 00000800:00000200:0.0:1788880705.151286:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880705.151287:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306336273 00000400:00000010:1.0:1788880705.151289:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a3b2cf4a. 00000100:00000001:3.0:1788880705.151295:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306341327 00000800:00000010:0.0:1788880705.151298:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b016e4f4. 00000100:00000001:1.0:1788880705.151300:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880705.151300:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880705.151308:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880705.151311:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880721 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880705.151312:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880705.151318:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880705.151321:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880705.151328:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880705.151331:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880705.151332:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880705.151340:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880721 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880705.151342:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:0.0:1788880705.151348:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880705.151349:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880705.151354:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880705.151360:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b016e4f4 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880705.151362:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880705.151378:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880705.151379:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880705.151381:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000040:0.0:1788880705.151389:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 232216 usecs (233 jiffies) 00000100:00000001:0.0:1788880705.151396:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306336490 00000100:00000001:0.0:1788880705.151399:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306341327 00000100:00000001:0.0:1788880705.151402:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306336490 00000800:00000200:1.0:1788880705.151402:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880705.151408:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000100:00000001:0.0:1788880705.151411:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306341327 00000400:00000200:1.0:1788880705.151413:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880705.151421:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306336490 00000100:00000001:0.0:1788880705.151427:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306341327 00000400:00000200:1.0:1788880705.151428:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880705.151431:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306336490 00000100:00000001:0.0:1788880705.151433:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306341327 00000100:00000001:0.0:1788880705.151436:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880705.151438:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880705.151445:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880705.151450:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880705.151451:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8336eb2 00000400:00000010:1.0:1788880705.151455:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8336eb2. 00000100:00000001:0.0:1788880705.151458:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880705.151463:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880705.151467:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880705.151470:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880705.151484:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880705.151488:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880705.151489:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880705.151494:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880705.151495:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880705.151502:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880705.151515:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880705.151517:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880705.151550:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880705.151553:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b016e4f4. 00000400:00000200:1.0:1788880705.151558:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880705.151562:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880705.151564:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e0ace8ca 00000400:00000010:1.0:1788880705.151566:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e0ace8ca. 00000100:00000001:1.0:1788880705.151568:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880705.151571:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880705.151579:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880705.151581:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880705.151587:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880705.151589:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880705.152819:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880705.152832:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880705.152835:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880705.152837:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880705.152845:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880705.152859:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0b680 00000400:00000200:2.0:1788880705.152866:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xc9d [1] + 224 00000800:00000001:2.0:1788880705.152873:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880705.152892:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880705.152894:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880705.152898:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880705.152902:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880705.152907:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880705.152917:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880705.152939:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880705.152946:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880705.152950:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880705.152978:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880705.152982:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880705.152987:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880705.152989:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880705.152991:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000edc7a681. 00000400:00000200:0.0:1788880705.152996:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000006bff322d 00000400:00000010:0.0:1788880705.152998:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000006bff322d. 00000100:00000001:0.0:1788880705.153002:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880705.153007:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880705.153014:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880705.153020:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880705.153021:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880705.153022:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880705.153024:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880705.153026:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880705.153027:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880705.153029:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880705.153034:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=83 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880705.153040:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=83 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880705.153042:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880705.153043:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880705.153045:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880705.153047:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880705.153049:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880705.153051:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880705.153062:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880705.153065:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880705.153068:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880705.153081:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880705.153089:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780678784:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880705.153094:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880705.153097:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880705.153100:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002980f9ff x1875776780678784/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880705.153106:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880705.153109:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880705.153111:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000744e90e2. 02000000:00000001:0.0:1788880705.153115:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880705.153118:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880705.153119:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880705.153122:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880705.153124:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 02000000:00000001:0.0:1788880705.153128:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880705.153130:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000800:00000001:1.0:1788880705.153131:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000010:0.0:1788880705.153132:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002980f9ff. 00000100:00000001:0.0:1788880705.153138:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880705.153139:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880705.153141:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880705.153145:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880705.153146:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880705.153149:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880705.153150:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880705.153152:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:1.0:1788880705.153154:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880705.153156:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880705.153164:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880705.153176:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b780 00000400:00000200:1.0:1788880705.153184:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xca5 [1] + 224 00000800:00000001:1.0:1788880705.153189:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880705.153194:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880705.153196:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880705.153203:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880705.153208:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880705.153210:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880705.153217:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880705.153230:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880705.153260:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880705.153264:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880705.153341:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880705.153344:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880705.153349:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880705.153351:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880705.153353:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:3.0:1788880705.153357:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000800:00000001:1.0:1788880705.153359:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000010:3.0:1788880705.153360:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880705.153365:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:1.0:1788880705.153367:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880705.153371:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1788880705.153371:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880705.153376:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880705.153384:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000200:3.0:1788880705.153388:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880705.153393:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b700 00000400:00000200:1.0:1788880705.153402:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xca9 [1] + 224 00000100:00000001:3.0:1788880705.153404:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880705.153406:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880705.153407:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880705.153410:0:201287:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:1.0:1788880705.153412:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880705.153413:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000200:1.0:1788880705.153415:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880705.153416:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880705.153420:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880705.153421:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:3.0:1788880705.153425:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880705.153428:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880705.153433:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880705.153433:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=83 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880705.153441:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=40 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880705.153448:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880705.153451:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880705.153455:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880705.153455:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880705.153464:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880705.153475:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880705.153477:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880705.153479:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000800:00000001:1.0:1788880705.153482:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880705.153485:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880705.153502:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880705.153505:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880705.153508:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880705.153509:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880705.153511:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000400:00000200:0.0:1788880705.153514:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:0.0:1788880705.153516:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:0.0:1788880705.153518:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880705.153521:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880705.153541:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880705.153547:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880705.153549:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880705.153551:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880705.153552:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880705.153553:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880705.153554:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880705.153555:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880705.153558:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=73 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880705.153560:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=73 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880705.153561:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880705.153562:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880705.153563:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880705.153565:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880705.153566:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880705.153567:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880705.153568:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880705.153569:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880705.153571:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880705.153575:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880705.153576:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880705.153576:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880705.153578:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880705.153582:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780678912:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880705.153584:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880705.153585:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880705.153587:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e2be23b2 x1875776780678912/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880705.153595:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880705.153598:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880705.153599:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000001:0.0:1788880705.153601:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880705.153602:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880705.153603:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000bef544e8. 02000000:00000001:0.0:1788880705.153606:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880705.153607:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880705.153607:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000040:3.0:1788880705.153613:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880705.153618:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880705.153620:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880705.153621:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880705.153623:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880705.153629:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880705.153632:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 00000100:00100000:3.0:1788880705.153637:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780679040:192.168.204.111@tcp:400: 02000000:00000001:0.0:1788880705.153638:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880705.153641:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880705.153644:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e2be23b2. 00000100:00000001:3.0:1788880705.153645:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880705.153649:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880705.153650:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880705.153653:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880705.153656:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002b387587 x1875776780679040/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880721 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880705.153657:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880705.153664:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880705.153666:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880705.153670:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880705.153671:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880705.153672:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880705.153674:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880705.153675:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000082966425. 02000000:00000001:3.0:1788880705.153679:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880705.153680:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880705.153681:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880705.153683:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880705.153684:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5b428a0. 02000000:00000001:3.0:1788880705.153686:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880705.153687:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880705.153688:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002b387587. 00000100:00000001:3.0:1788880705.153690:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880705.153691:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880705.153692:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880705.153694:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880705.153695:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880705.153697:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880705.153698:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880705.214340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880705.214349:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880705.214351:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880705.214370:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880705.214374:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880705.214376:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880705.214378:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880705.214380:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880705.214384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880705.214386:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880705.214469:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880705.214475:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880705.214479:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880706.238354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880706.238362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880706.238368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880706.238369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880706.238437:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880706.238442:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880706.238445:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880707.262345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880707.262352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880707.262358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880707.262362:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880707.262364:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880707.262367:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880707.262370:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880707.262372:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880707.262374:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880707.262376:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880707.262440:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880707.262445:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880707.262448:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880708.286353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880708.286360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880708.286367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880708.286368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880708.286433:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880708.286437:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880708.286440:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880709.310345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880709.310352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880709.310358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880709.310359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880709.310428:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880709.310435:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880709.310440:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880710.270300:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306341327, at 4306341376 00000100:00000040:2.0:1788880710.270304:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306341327, at 4306341376 00000100:00000040:3.0:1788880710.270307:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306341327, at 4306341376 00000100:00000010:0.0:1788880710.270319:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a3bf26bf. 00000100:00000010:3.0:1788880710.270323:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005c6a0f3f. 00000100:00000010:2.0:1788880710.270326:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000668fd8a. 00000100:00000040:1.0:1788880710.270330:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306341327, at 4306341376 00000100:00000010:1.0:1788880710.270339:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000007e595eb4. 00000100:00000040:2.0:1788880710.270343:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880710.270349:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880710.270357:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000020:00000040:2.0:1788880710.270371:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880710.270376:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880710.270392:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880710.270397:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880710.270404:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003139f6ba. 00000100:00000001:2.0:1788880710.270411:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880710.270421:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880710.270431:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880710.270433:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880710.270438:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_03+3] req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880710.270466:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880710.270468:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880710.270471:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880710.270474:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000629edf55. 00000020:00000040:2.0:1788880710.270477:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880710.270480:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880710.270481:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880710.270483:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880710.270486:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880710.270486:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f47bc5fc. 00000100:00000001:1.0:1788880710.270490:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880710.270491:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880710.270496:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880710.270500:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880710.270502:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880710.270508:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:1.0:1788880710.270509:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880710.270509:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.270512:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880710.270523:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000629edf55] to pc [ptlrpcd_00_00+0] req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880710.270540:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880710.270550:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880710.270555:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880710.270558:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880710.270561:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880710.270562:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880710.270566:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.270568:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880710.270572:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000005a8bb1de. 00000100:00100000:1.0:1788880710.270573:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780679168:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880710.270577:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880710.270577:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880710.270580:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880710.270582:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880710.270585:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880710.270586:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788880710.270587:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880710.270587:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880710.270590:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880710.270591:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880710.270592:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880710.270596:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880710.270597:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880710.270599:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880710.270603:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 02000000:00000010:1.0:1788880710.270604:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000087acc4de. 02000000:00000001:1.0:1788880710.270609:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.270612:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880710.270615:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880710.270617:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880710.270619:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:1.0:1788880710.270619:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008ffd4b4e. 00000100:00000040:2.0:1788880710.270620:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880710.270622:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880710.270626:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880710.270628:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000100:00100000:0.0:1788880710.270630:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780679296:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880710.270637:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780679168, portal 25 00000100:00000001:0.0:1788880710.270638:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880710.270641:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880710.270643:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880710.270644:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880710.270647:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880710.270648:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 02000000:00000001:0.0:1788880710.270653:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880710.270656:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880710.270658:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a8bb1de] to pc [ptlrpcd_00_01+1] req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880710.270659:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880726 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880710.270660:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000fc6edce. 02000000:00000001:0.0:1788880710.270664:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880710.270669:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 00000400:00000010:0.0:1788880710.270675:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000200:0.0:1788880710.270679:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780679296, portal 10 00000100:00000001:0.0:1788880710.270683:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880710.270685:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938432768 : -126823771118848 : ffff8ca78a2c6300) 00000100:00000001:1.0:1788880710.270690:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880710.270694:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880710.270698:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880710.270703:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 00000100:00000001:0.0:1788880710.270705:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880710.270707:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880710.270708:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780679168, offset 0 00000100:00000040:0.0:1788880710.270712:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880710.270713:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:0.0:1788880710.270718:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005c796512. 00000400:00000200:1.0:1788880710.270721:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880710.270721:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000200:0.0:1788880710.270728:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780679296, offset 0 00000100:00000040:2.0:1788880710.270729:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11679) 00000100:00000001:3.0:1788880710.270731:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880710.270735:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880710.270738:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:0.0:1788880710.270739:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880710.270739:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:3.0:1788880710.270743:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880710.270748:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880710.270756:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880710.270756:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880710.270759:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880710.270762:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880710.270763:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880710.270766:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880710.270769:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880710.270770:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880710.270775:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780679424:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880710.270778:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [380:380:256:4294967295] 192.168.204.111@tcp LPNI seq info [380:380:8:4294967295] 00000100:00000001:3.0:1788880710.270792:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880710.270796:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880710.270798:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880710.270801:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880710.270805:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000010:3.0:1788880710.270807:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d99c594f. 02000000:00000001:3.0:1788880710.270810:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880710.270812:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880710.270815:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880710.270819:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:1.0:1788880710.270820:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000400:00000200:0.0:1788880710.270821:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:3.0:1788880710.270821:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000400:00000200:0.0:1788880710.270824:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:3.0:1788880710.270824:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000400:00000200:0.0:1788880710.270826:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:3.0:1788880710.270827:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780679424, portal 10 00000400:00000200:0.0:1788880710.270831:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [381:381:255:4294967295] 192.168.204.111@tcp LPNI seq info [381:381:7:4294967295] 00000100:00000001:3.0:1788880710.270831:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880710.270832:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938427392 : -126823771124224 : ffff8ca78a2c4e00) 00000800:00000200:1.0:1788880710.270838:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:3.0:1788880710.270839:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880710.270843:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880710.270848:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880710.270849:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880710.270852:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f0a0057b. 00000800:00000200:1.0:1788880710.270855:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:3.0:1788880710.270856:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880710.270859:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:3.0:1788880710.270859:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f73021f1. 00000800:00000200:1.0:1788880710.270863:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000100:00000200:3.0:1788880710.270863:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780679424, offset 0 00000800:00000200:0.0:1788880710.270866:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880710.270868:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880710.270871:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f0a0057b type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880710.270874:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880710.270876:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880710.270878:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880710.270881:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880710.270882:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880710.270882:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880710.270882:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880710.270887:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [382:382:254:4294967295] 192.168.204.111@tcp LPNI seq info [382:382:6:4294967295] 00000100:00000040:0.0:1788880710.270889:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 230176 usecs (231 jiffies) 00000100:00000001:1.0:1788880710.270889:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306346447 00000100:00000001:0.0:1788880710.270893:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306341607 00000100:00000010:1.0:1788880710.270895:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000400:00000200:3.0:1788880710.270899:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880710.270901:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:3.0:1788880710.270905:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880710.270906:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306346447 00000100:00000001:0.0:1788880710.270907:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306346447 00000800:00000010:3.0:1788880710.270909:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:0.0:1788880710.270912:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306341607 00000100:00000010:1.0:1788880710.270912:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000800:00000200:3.0:1788880710.270916:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880710.270918:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:3.0:1788880710.270922:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880710.270923:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306346447 00000100:00000001:1.0:1788880710.270923:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306346447 00000800:00000200:3.0:1788880710.270926:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880710.270928:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306341607 00000100:00000010:1.0:1788880710.270928:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000100:00000001:3.0:1788880710.270929:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880710.270933:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000040:3.0:1788880710.270934:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 15738 usecs (16 jiffies) 00000100:00000001:0.0:1788880710.270938:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306346447 00000100:00000001:1.0:1788880710.270938:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306346447 00000100:00000001:3.0:1788880710.270938:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306341392 00000100:00000010:1.0:1788880710.270943:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000100:00000001:0.0:1788880710.270944:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306341607 00000100:00000001:3.0:1788880710.270944:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306346447 00000800:00000200:2.0:1788880710.270946:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880710.270947:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880710.270948:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306341392 00000100:00000001:1.0:1788880710.270953:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880710.270954:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306346447 00000800:00000010:2.0:1788880710.270954:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000100:00000001:3.0:1788880710.270954:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306346447 00000100:00000001:1.0:1788880710.270958:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880710.270960:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880710.270963:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880710.270963:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880710.270963:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306341392 00000400:00000200:2.0:1788880710.270964:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880710.270967:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880710.270967:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306346447 00000100:00000001:1.0:1788880710.270968:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880710.270971:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880710.270971:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306341392 00000400:00000200:2.0:1788880710.270975:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880710.270975:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306346447 00000100:00000001:1.0:1788880710.270976:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880710.270977:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880710.270979:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880710.270981:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880710.270981:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880710.270982:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880710.270985:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000100:00000001:1.0:1788880710.270988:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880710.270988:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880710.270989:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:2.0:1788880710.270989:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:0.0:1788880710.270992:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880710.270993:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880710.270993:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880710.270995:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.270998:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880710.270999:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880710.270999:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880710.270999:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880710.271002:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880710.271005:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880710.271005:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880710.271007:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880726 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880710.271009:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880710.271013:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880710.271017:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.271025:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880710.271029:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880726 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880710.271036:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.271038:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880710.271051:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880710.271053:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f0a0057b. 00000400:00000200:2.0:1788880710.271062:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880710.271066:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880710.271068:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005c796512 00000400:00000010:2.0:1788880710.271070:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005c796512. 00000100:00000001:2.0:1788880710.271073:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880710.271076:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880710.271083:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880710.271085:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880710.271091:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.271092:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880710.271158:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880710.271160:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:2.0:1788880710.271163:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880710.271166:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880710.271168:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f73021f1 00000400:00000010:2.0:1788880710.271170:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f73021f1. 00000100:00000001:2.0:1788880710.271172:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880710.271175:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880710.271181:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880710.271183:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880710.271189:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.271190:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880710.273350:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880710.273361:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880710.273364:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880710.273367:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880710.273375:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880710.273386:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b880 00000400:00000200:1.0:1788880710.273402:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xcb9 [1] + 224 00000800:00000001:1.0:1788880710.273407:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880710.273410:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880710.273412:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880710.273415:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880710.273419:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880710.273423:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880710.273432:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880710.273447:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880710.273453:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880710.273456:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880710.273458:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880710.273460:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880710.273463:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880710.273470:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0b900 00000400:00000200:1.0:1788880710.273475:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xcc5 [1] + 224 00000800:00000001:1.0:1788880710.273479:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880710.273481:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880710.273483:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880710.273485:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880710.273487:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880710.273489:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880710.273492:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880710.273496:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880710.273497:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880710.273500:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880710.273504:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000400:00000200:0.0:1788880710.273509:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000100:00000040:1.0:1788880710.273512:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880710.273513:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:0.0:1788880710.273518:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880710.273524:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880710.273533:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880710.273534:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880710.273540:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880710.273543:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880710.273544:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880710.273546:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880710.273549:0:201285:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:1.0:1788880710.273553:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880710.273556:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880710.273560:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880710.273564:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:1.0:1788880710.273564:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880710.273569:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880710.273571:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880710.273573:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880710.273576:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0b800 00000100:00000001:3.0:1788880710.273576:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880710.273582:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00001000:0.0:1788880710.273584:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=78 v=5 (5 5 5 5) 00000400:00000001:3.0:1788880710.273585:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880710.273588:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000100:00001000:0.0:1788880710.273591:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=78 v=5 (1 1 1 1) 00000400:00000200:3.0:1788880710.273593:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000100:00000001:0.0:1788880710.273598:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000010:3.0:1788880710.273598:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:0.0:1788880710.273601:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880710.273603:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:0.0:1788880710.273605:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:1.0:1788880710.273606:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xcb5 [1] + 224 00010000:00000001:0.0:1788880710.273609:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880710.273610:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880710.273616:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000800:00000001:1.0:1788880710.273616:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1788880710.273621:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880710.273623:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880710.273626:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880710.273626:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880710.273631:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:0.0:1788880710.273635:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880710.273641:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:3.0:1788880710.273645:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880710.273649:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880710.273656:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880710.273659:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880710.273665:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880710.273668:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880710.273671:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880710.273675:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880710.273678:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880710.273680:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880710.273684:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880710.273687:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:1.0:1788880710.273690:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880710.273690:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880710.273694:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780679296:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880710.273695:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880710.273703:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=88 v=5 (5 5 5 5) 00000100:00000001:0.0:1788880710.273705:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880710.273709:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880710.273714:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000629edf55 x1875776780679296/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880710.273715:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=45 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880710.273716:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880710.273722:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880710.273725:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:1.0:1788880710.273727:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880710.273728:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880710.273730:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00010000:00000001:3.0:1788880710.273733:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:0.0:1788880710.273734:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880710.273738:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000fc6edce. 00010000:00080000:3.0:1788880710.273740:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880710.273743:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880710.273746:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880710.273749:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00010000:00000001:3.0:1788880710.273753:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880710.273754:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880710.273757:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880710.273759:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880710.273763:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000010:0.0:1788880710.273764:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f47bc5fc. 00000100:00000001:3.0:1788880710.273765:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880710.273770:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:3.0:1788880710.273771:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880710.273773:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880710.273776:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000629edf55. 00000100:00000001:0.0:1788880710.273794:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880710.273796:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880710.273801:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880710.273802:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880710.273807:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880710.273808:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880710.273809:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880710.273811:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880710.273812:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880710.273816:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880710.273816:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880710.273819:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.273824:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880710.273830:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00100000:3.0:1788880710.273830:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780679424:192.168.204.111@tcp:400: 00000400:00000001:2.0:1788880710.273834:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880710.273837:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008ffd4b4e. 00000100:00000001:3.0:1788880710.273838:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880710.273843:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788880710.273844:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000100:00000040:3.0:1788880710.273847:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000005a8bb1de x1875776780679424/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880710.273848:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:2.0:1788880710.273853:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880710.273857:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880710.273860:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880710.273863:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d99c594f. 00000100:00000200:2.0:1788880710.273865:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880710.273868:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880710.273871:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880710.273874:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880710.273877:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880710.273879:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 00000100:00000200:2.0:1788880710.273882:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880710.273889:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880710.273892:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880710.273894:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000005a8bb1de. 00000100:00000001:2.0:1788880710.273896:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880710.273897:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880710.273900:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880710.273901:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880710.273902:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880710.273907:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880710.273908:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788880710.273910:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880710.273911:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.273913:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880710.273914:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880710.273917:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880710.273918:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880710.273924:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880710.273927:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=88 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880710.273931:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=88 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880710.273933:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880710.273934:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880710.273936:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880710.273938:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.273940:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880710.273943:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880710.273949:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880710.273951:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880710.273952:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880710.273955:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880710.273962:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780679168:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880710.273966:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880710.273968:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880710.273970:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780679168/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880726 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880710.273980:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880710.273982:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880710.273990:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000087acc4de. 02000000:00000001:2.0:1788880710.273993:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880710.273994:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880710.273995:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880710.273997:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880710.273999:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003139f6ba. 02000000:00000001:2.0:1788880710.274001:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880710.274002:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880710.274003:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000100:00000001:2.0:1788880710.274006:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880710.274007:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880710.274008:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880710.274012:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880710.274014:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880710.274016:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880710.274017:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880710.334410:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880710.334417:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880710.334423:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880710.334424:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880710.334487:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880710.334491:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880710.334493:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880711.358351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880711.358360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880711.358366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880711.358367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880711.358435:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880711.358441:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880711.358446:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880712.382359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880712.382370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880712.382377:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880712.382379:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880712.382454:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880712.382459:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880712.382463:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880713.406434:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880713.406441:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880713.406446:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880713.406447:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880713.406474:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880713.406478:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880713.406480:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880714.430291:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880714.430301:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880714.430305:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880714.430306:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880714.430349:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880714.430352:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880714.430354:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880715.390348:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306346447, at 4306346496 00000100:00000040:1.0:1788880715.390348:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306346447, at 4306346496 00000100:00000040:2.0:1788880715.390348:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306346447, at 4306346496 00000100:00000040:3.0:1788880715.390348:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306346447, at 4306346496 00000100:00000010:2.0:1788880715.390365:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000010:3.0:1788880715.390367:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000010:1.0:1788880715.390370:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:0.0:1788880715.390374:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000040:2.0:1788880715.390388:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880715.390393:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880715.390401:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000020:00000040:2.0:1788880715.390405:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880715.390409:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880715.390411:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880715.390416:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880715.390421:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:2.0:1788880715.390428:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880715.390437:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880715.390446:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880715.390448:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880715.390453:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_02+2] req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880715.390476:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880715.390490:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880715.390493:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880715.390496:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cc13c26b. 00000020:00000040:2.0:1788880715.390498:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880715.390499:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880715.390501:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880715.390503:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880715.390505:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880715.390505:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880715.390505:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880715.390510:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880715.390511:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880715.390518:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880715.390520:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 00000100:00000001:2.0:1788880715.390528:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880715.390533:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880715.390536:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880715.390537:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880715.390544:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880715.390547:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880715.390548:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880715.390549:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880715.390552:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cc13c26b] to pc [ptlrpcd_00_03+3] req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880715.390561:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880715.390567:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780679552:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880715.390568:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880715.390572:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880715.390575:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880715.390576:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880715.390580:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000910cb22c. 02000000:00000001:3.0:1788880715.390581:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000020:00000040:2.0:1788880715.390583:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:3.0:1788880715.390585:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880715.390586:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880715.390589:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880715.390590:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880715.390590:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880715.390594:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880715.390595:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:3.0:1788880715.390595:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000060f4a118. 00000100:00000001:1.0:1788880715.390598:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880715.390599:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880715.390600:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 00000100:00000001:2.0:1788880715.390604:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880715.390607:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880715.390607:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000100:00000040:2.0:1788880715.390611:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:3.0:1788880715.390615:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000200:3.0:1788880715.390623:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780679552, portal 25 00000100:00000001:2.0:1788880715.390627:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880715.390628:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880715.390630:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880715.390631:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 00000100:00000040:2.0:1788880715.390638:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000910cb22c] to pc [ptlrpcd_00_00+0] req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880715.390639:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788880715.390642:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880731 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880715.390643:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880715.390651:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880715.390654:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880715.390659:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880715.390664:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00100000:1.0:1788880715.390668:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780679680:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880715.390675:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880715.390679:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880715.390679:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880715.390694:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880715.390694:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 02000000:00000001:1.0:1788880715.390699:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:3.0:1788880715.390700:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780679552, offset 0 00000100:00000001:2.0:1788880715.390702:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880715.390705:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004d9329ab. 00000100:00000040:2.0:1788880715.390705:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:3.0:1788880715.390709:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880715.390711:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880715.390713:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:1.0:1788880715.390719:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a3419a3. 00000100:00000040:2.0:1788880715.390719:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11684) 00000100:00000001:0.0:1788880715.390720:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880715.390726:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880715.390728:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:1.0:1788880715.390728:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000641230df. 00000400:00000200:3.0:1788880715.390733:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:1.0:1788880715.390734:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780679680, portal 10 00000100:00000040:0.0:1788880715.390735:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880715.390740:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788880715.390740:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880715.390744:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880715.390745:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938433664 : -126823771117952 : ffff8ca78a2c6680) 00000100:00000001:0.0:1788880715.390749:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880715.390749:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880715.390750:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880715.390753:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880715.390756:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:1.0:1788880715.390757:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880715.390758:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1788880715.390760:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880715.390764:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880715.390766:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00100000:0.0:1788880715.390770:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780679808:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880715.390777:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [383:383:256:4294967295] 192.168.204.111@tcp LPNI seq info [383:383:8:4294967295] 00000100:00000001:0.0:1788880715.390778:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880715.390779:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880715.390781:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880715.390784:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880715.390787:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880715.390788:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880715.390792:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009e1ad299. 00000400:00000010:1.0:1788880715.390793:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000974f52f4. 02000000:00000001:0.0:1788880715.390796:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880715.390800:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 00000400:00000200:3.0:1788880715.390801:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:1.0:1788880715.390803:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780679680, offset 0 00000400:00000010:0.0:1788880715.390805:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000800:00000200:3.0:1788880715.390813:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880715.390816:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880715.390820:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000200:0.0:1788880715.390821:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780679808, portal 10 00000400:00000200:1.0:1788880715.390824:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880715.390825:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880715.390828:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938437248 : -126823771114368 : ffff8ca78a2c7480) 00000400:00000200:1.0:1788880715.390832:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:3.0:1788880715.390832:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880715.390838:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880715.390843:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:0.0:1788880715.390844:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880715.390847:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880715.390850:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880715.390854:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880715.390860:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880715.390860:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [384:384:255:4294967295] 192.168.204.111@tcp LPNI seq info [384:384:7:4294967295] 00000800:00000200:3.0:1788880715.390861:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000400:00000010:0.0:1788880715.390863:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a08ac001. 00000100:00000200:0.0:1788880715.390867:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780679808, offset 0 00000400:00000200:0.0:1788880715.390875:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880715.390881:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880715.390882:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880715.390888:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306351567 00000800:00000200:1.0:1788880715.390889:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880715.390893:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880715.390896:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000055d7d92. 00000100:00000010:3.0:1788880715.390896:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014c076ca. 00000400:00000200:0.0:1788880715.390899:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880715.390904:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880715.390906:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880715.390909:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880715.390912:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306351567 00000400:00000200:0.0:1788880715.390913:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:3.0:1788880715.390918:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000015631aee. 00000800:00000200:1.0:1788880715.390919:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880715.390921:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880715.390923:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:1.0:1788880715.390924:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000055d7d92 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880715.390928:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306351567 00000400:00000200:0.0:1788880715.390931:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [385:385:254:4294967295] 192.168.204.111@tcp LPNI seq info [385:385:6:4294967295] 00000100:00000001:1.0:1788880715.390932:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880715.390933:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a9d0cfcc. 00000800:00000200:2.0:1788880715.390936:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:1.0:1788880715.390939:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 228172 usecs (229 jiffies) 00000800:00000010:2.0:1788880715.390941:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:3.0:1788880715.390943:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880715.390944:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306346725 00000400:00000200:2.0:1788880715.390948:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880715.390949:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306351567 00000400:00000200:0.0:1788880715.390954:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880715.390955:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306351567 00000100:00000001:1.0:1788880715.390956:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306346725 00000400:00000200:2.0:1788880715.390959:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880715.390962:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306351567 00000100:00000010:3.0:1788880715.390965:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e62b7132. 00000800:00000200:0.0:1788880715.390968:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880715.390969:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306346725 00000400:00000200:2.0:1788880715.390969:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000100:00000001:3.0:1788880715.390971:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000010:2.0:1788880715.390974:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000800:00000010:0.0:1788880715.390976:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f0a0057b. 00000100:00000001:1.0:1788880715.390976:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306351567 00000100:00000001:3.0:1788880715.390977:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880715.390980:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880715.390983:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306346725 00000100:00000001:3.0:1788880715.390984:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880715.390986:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880715.390988:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880731 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880715.390990:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306351567 00000100:00000001:3.0:1788880715.390991:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880715.390997:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880715.390997:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880715.391000:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880715.391005:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880715.391007:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f0a0057b type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880715.391007:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880715.391011:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880715.391011:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880715.391015:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880715.391016:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880715.391017:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880715.391017:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880715.391021:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880731 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880715.391021:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880715.391022:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 15630 usecs (16 jiffies) 00000100:00000001:3.0:1788880715.391025:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880715.391030:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306346512 00000100:00000001:3.0:1788880715.391031:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880715.391032:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880715.391036:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880715.391037:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306351567 00000100:00000001:2.0:1788880715.391038:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880715.391042:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880715.391044:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306346512 00000100:00000001:2.0:1788880715.391044:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880715.391045:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880715.391050:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306351567 00000100:00000001:1.0:1788880715.391050:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880715.391055:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306346512 00000100:00000001:1.0:1788880715.391057:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880715.391058:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880715.391060:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306351567 00000800:00000010:2.0:1788880715.391062:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000055d7d92. 00000100:00000001:0.0:1788880715.391066:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306346512 00000400:00000200:2.0:1788880715.391068:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880715.391071:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880715.391077:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306351567 00000400:00000200:2.0:1788880715.391077:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000974f52f4 00000400:00000010:2.0:1788880715.391081:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000974f52f4. 00000100:00000001:0.0:1788880715.391082:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880715.391086:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880715.391087:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880715.391089:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880715.391092:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880715.391093:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880715.391100:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880715.391102:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880715.391104:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880715.391106:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880715.391107:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880715.391109:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880715.391111:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880715.391114:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880715.391121:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880715.391123:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880715.391176:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880715.391178:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f0a0057b. 00000400:00000200:2.0:1788880715.391186:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880715.391189:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880715.391191:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a08ac001 00000400:00000010:2.0:1788880715.391192:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a08ac001. 00000100:00000001:2.0:1788880715.391194:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880715.391196:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880715.391202:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880715.391204:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880715.391209:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880715.391210:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880715.392779:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880715.392788:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880715.392791:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880715.392792:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880715.392798:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880715.392809:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0b980 00000400:00000200:2.0:1788880715.392815:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xccd [1] + 224 00000800:00000001:2.0:1788880715.392820:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880715.392823:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880715.392824:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880715.392827:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880715.392832:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880715.392835:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880715.392842:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880715.392861:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880715.392868:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880715.392871:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880715.392873:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880715.392875:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880715.392880:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880715.392882:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880715.392886:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ba00 00000100:00000001:3.0:1788880715.392887:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880715.392893:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xcd5 [1] + 224 00000800:00000001:2.0:1788880715.392896:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880715.392899:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880715.392900:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880715.392902:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:3.0:1788880715.392928:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880715.392931:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880715.392933:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:3.0:1788880715.392936:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880715.392937:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880715.392939:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880715.392940:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880715.392944:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880715.392948:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880715.392966:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880715.392981:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:2.0:1788880715.392984:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880715.392984:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880715.392989:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880715.392992:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880715.392994:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880715.392997:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880715.392998:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880715.393003:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:3.0:1788880715.393003:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=93 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880715.393007:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=93 v=5 (1 1 1 1) 00000800:00000001:2.0:1788880715.393009:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880715.393012:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880715.393014:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880715.393015:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880715.393019:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880715.393022:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880715.393025:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00010000:00000001:3.0:1788880715.393028:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880715.393031:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880715.393033:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880715.393033:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880715.393035:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ba80 00000100:00000001:1.0:1788880715.393037:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880715.393041:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880715.393046:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xcdd [1] + 224 00000400:00000001:1.0:1788880715.393055:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880715.393056:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880715.393056:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000001:1.0:1788880715.393059:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880715.393060:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000010:1.0:1788880715.393061:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a3419a3. 00000800:00000001:2.0:1788880715.393063:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880715.393063:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880715.393065:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000641230df 00000400:00000010:1.0:1788880715.393068:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000641230df. 00000400:00000200:2.0:1788880715.393068:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:3.0:1788880715.393069:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880715.393074:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880715.393078:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:1.0:1788880715.393081:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880715.393085:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780679552:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880715.393086:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880715.393095:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880715.393097:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:2.0:1788880715.393099:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880715.393100:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880715.393103:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780679552/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880715.393127:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880715.393131:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880715.393133:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880715.393133:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:2.0:1788880715.393136:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880715.393138:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:3.0:1788880715.393138:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880715.393141:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000010:3.0:1788880715.393142:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000060f4a118. 00000100:00000001:1.0:1788880715.393144:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880715.393148:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880715.393152:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:1.0:1788880715.393153:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=83 v=5 (5 5 5 5) 00000020:00000001:3.0:1788880715.393155:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00001000:1.0:1788880715.393157:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=83 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880715.393157:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000040:3.0:1788880715.393159:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880715.393160:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880715.393162:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880715.393164:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788880715.393164:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:1.0:1788880715.393166:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880715.393169:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000020:00000001:3.0:1788880715.393169:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000800:00000001:2.0:1788880715.393171:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880715.393172:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000010:3.0:1788880715.393173:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 00000100:00000001:1.0:1788880715.393176:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880715.393178:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880715.393180:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:1.0:1788880715.393182:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880715.393188:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880715.393189:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880715.393191:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:3.0:1788880715.393193:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000400:00000001:0.0:1788880715.393196:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880715.393199:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880715.393201:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000400:00000010:0.0:1788880715.393202:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000100:00000001:1.0:1788880715.393203:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880715.393205:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880715.393208:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000100:00000001:1.0:1788880715.393209:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880715.393209:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880715.393211:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:1.0:1788880715.393212:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880715.393215:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880715.393216:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:1.0:1788880715.393219:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880715.393219:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880715.393223:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:0.0:1788880715.393224:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880715.393226:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880715.393260:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780679680:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880715.393267:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880715.393269:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880715.393270:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880715.393274:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cc13c26b x1875776780679680/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880715.393292:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880715.393293:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880715.393294:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880715.393297:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004d9329ab. 00000100:00000001:0.0:1788880715.393298:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880715.393301:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880715.393302:0:201285:0:(client.c:1580:after_reply()) Process entered 00000020:00000001:1.0:1788880715.393304:0:201288:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000001:0.0:1788880715.393305:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000040:1.0:1788880715.393306:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880715.393308:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000001:1.0:1788880715.393309:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880715.393310:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880715.393311:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 02000000:00000001:0.0:1788880715.393313:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880715.393315:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880715.393317:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00001000:0.0:1788880715.393318:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=93 v=5 (5 5 5 5) 00000100:00000010:1.0:1788880715.393318:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cc13c26b. 00000100:00000001:1.0:1788880715.393320:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880715.393322:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880715.393323:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=50 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880715.393324:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880715.393329:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880715.393330:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880715.393330:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880715.393333:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880715.393334:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880715.393336:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880715.393337:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:0.0:1788880715.393344:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880715.393349:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880715.393351:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880715.393352:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880715.393353:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880715.393354:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880715.393356:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880715.393359:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880715.393360:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880715.393362:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880715.393363:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880715.393367:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780679808:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880715.393370:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880715.393371:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880715.393372:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000910cb22c x1875776780679808/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880731 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880715.393375:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880715.393376:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880715.393378:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009e1ad299. 02000000:00000001:0.0:1788880715.393380:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880715.393381:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880715.393381:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880715.393383:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880715.393383:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 02000000:00000001:0.0:1788880715.393385:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880715.393386:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880715.393387:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000910cb22c. 00000100:00000001:0.0:1788880715.393389:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880715.393393:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880715.393394:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880715.393396:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880715.393397:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880715.393398:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880715.393400:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880715.454341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880715.454350:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880715.454353:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880715.454357:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880715.454378:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880715.454380:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880715.454382:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880715.454385:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880715.454389:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880715.454391:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880715.454455:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880715.454460:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880715.454463:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880716.478321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880716.478327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880716.478333:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880716.478333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880716.478386:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880716.478390:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880716.478392:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880717.502330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880717.502337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880717.502343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880717.502346:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880717.502348:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880717.502351:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880717.502354:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880717.502357:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880717.502358:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880717.502360:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880717.502393:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880717.502396:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880717.502398:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880718.526408:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880718.526415:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880718.526419:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880718.526420:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880718.526462:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880718.526465:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880718.526468:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880719.550333:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880719.550339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880719.550345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880719.550346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880719.550379:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880719.550382:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880719.550385:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880720.510314:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306351567, at 4306351616 00000100:00000040:1.0:1788880720.510314:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306351567, at 4306351616 00000100:00000040:3.0:1788880720.510315:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306351567, at 4306351616 00000100:00000040:2.0:1788880720.510318:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306351567, at 4306351616 00000100:00000010:1.0:1788880720.510331:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000015631aee. 00000100:00000010:0.0:1788880720.510336:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014c076ca. 00000100:00000010:2.0:1788880720.510338:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a9d0cfcc. 00000100:00000010:3.0:1788880720.510341:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e62b7132. 00000100:00000040:2.0:1788880720.510350:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880720.510355:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880720.510361:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ea101a56. 00000020:00000040:2.0:1788880720.510365:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880720.510368:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880720.510371:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880720.510375:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880720.510381:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 00000100:00000001:2.0:1788880720.510387:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880720.510395:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880720.510402:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880720.510404:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880720.510409:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ea101a56] to pc [ptlrpcd_00_01+1] req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880720.510430:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880720.510442:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880720.510446:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880720.510448:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000071512652. 00000100:00000001:0.0:1788880720.510450:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880720.510450:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880720.510451:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880720.510454:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880720.510455:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880720.510455:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880720.510456:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880720.510459:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880720.510460:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880720.510464:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e53a8f34. 00000100:00000040:0.0:1788880720.510467:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880720.510468:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880720.510474:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880720.510490:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880720.510490:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880720.510493:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880720.510495:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880720.510498:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000071512652] to pc [ptlrpcd_00_02+2] req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880720.510501:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880720.510505:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880720.510509:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880720.510513:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780679936:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880720.510513:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880720.510517:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880720.510520:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880720.510522:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880720.510523:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00100000:1.0:1788880720.510524:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:1.0:1788880720.510527:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:2.0:1788880720.510527:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000910cb22c. 02000000:00000001:0.0:1788880720.510528:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880720.510531:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880720.510533:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880720.510535:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880720.510536:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880720.510539:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880720.510539:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880720.510541:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004ffd965d. 00000100:00000001:2.0:1788880720.510541:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880720.510542:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880720.510544:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880720.510545:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880720.510546:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880720.510550:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880720.510550:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880720.510553:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880720.510554:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 02000000:00000010:2.0:1788880720.510557:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c54a5c6f. 00000100:00000040:1.0:1788880720.510560:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880720.510561:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000001:2.0:1788880720.510563:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880720.510567:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780679936, portal 25 00000100:00000001:0.0:1788880720.510570:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880720.510571:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880720.510573:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880720.510574:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938436352 : -126823771115264 : ffff8ca78a2c7100) 00000100:00000001:1.0:1788880720.510574:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880720.510578:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880720.510580:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880720.510582:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880736 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880720.510584:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880720.510599:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:1.0:1788880720.510602:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000071512652 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780680064:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880720.510603:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880720.510609:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880720.510609:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880720.510612:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000910cb22c] to pc [ptlrpcd_00_03+3] req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880720.510613:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880720.510615:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880720.510616:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880720.510620:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880720.510624:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004d9329ab. 00000400:00000010:0.0:1788880720.510625:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009fe01c6d. 02000000:00000001:1.0:1788880720.510626:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880720.510627:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780679936, offset 0 00000400:00000010:1.0:1788880720.510630:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a3419a3. 00000400:00000200:0.0:1788880720.510638:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880720.510646:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000641230df. 00000100:00000200:1.0:1788880720.510650:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780680064, portal 10 00000100:00000001:1.0:1788880720.510651:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880720.510652:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880720.510652:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430080 : -126823771121536 : ffff8ca78a2c5880) 00000100:00000001:2.0:1788880720.510653:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880720.510658:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:0.0:1788880720.510660:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788880720.510661:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880720.510662:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:0.0:1788880720.510666:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:2.0:1788880720.510669:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11689) 00000100:00000001:1.0:1788880720.510671:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788880720.510672:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880720.510676:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788880720.510676:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880720.510678:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000038c5ac58. 00000400:00000200:0.0:1788880720.510681:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [386:386:256:4294967295] 192.168.204.111@tcp LPNI seq info [386:386:8:4294967295] 00000100:00000200:1.0:1788880720.510681:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780680064, offset 0 00000400:00000200:1.0:1788880720.510686:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880720.510693:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880720.510695:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00100000:3.0:1788880720.510698:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:3.0:1788880720.510702:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880720.510703:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880720.510705:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788880720.510709:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b016e4f4. 00000400:00000200:1.0:1788880720.510709:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880720.510709:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880720.510713:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880720.510715:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880720.510717:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:0.0:1788880720.510718:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880720.510719:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880720.510721:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880720.510723:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880720.510724:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:0.0:1788880720.510729:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:3.0:1788880720.510731:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880720.510734:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b016e4f4 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880720.510735:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [387:387:255:4294967295] 192.168.204.111@tcp LPNI seq info [387:387:7:4294967295] 00000400:00000200:1.0:1788880720.510748:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880720.510752:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880720.510753:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880720.510756:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306356687 00000800:00000010:1.0:1788880720.510757:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000100:00000010:0.0:1788880720.510759:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001d590ced. 00000800:00000200:1.0:1788880720.510760:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880720.510763:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:1.0:1788880720.510766:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880720.510767:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306356687 00000100:00000010:0.0:1788880720.510770:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000bc13d1e4. 00000800:00000200:1.0:1788880720.510770:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880720.510776:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880720.510776:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880720.510779:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306356687 00000100:00000040:1.0:1788880720.510781:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 226198 usecs (227 jiffies) 00000100:00000010:0.0:1788880720.510784:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009f22a9a0. 00000100:00000001:1.0:1788880720.510784:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306351843 00000100:00000001:3.0:1788880720.510785:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880720.510786:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306356687 00000100:00000001:0.0:1788880720.510788:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880720.510789:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880720.510790:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306351843 00000100:00000001:0.0:1788880720.510791:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306356687 00000100:00000001:1.0:1788880720.510794:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306356687 02000000:00000001:3.0:1788880720.510795:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000010:0.0:1788880720.510796:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e64ede4. 00000100:00000001:1.0:1788880720.510796:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306351843 00000100:00000001:0.0:1788880720.510800:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880720.510800:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306356687 02000000:00000001:3.0:1788880720.510800:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880720.510804:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306351843 00000100:00000001:0.0:1788880720.510806:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880720.510807:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880720.510808:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306356687 00000100:00000001:0.0:1788880720.510809:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880720.510809:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880720.510812:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00100000:3.0:1788880720.510815:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780680192:192.168.204.111@tcp:400: 00000800:00000010:2.0:1788880720.510816:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b016e4f4. 00000100:00000001:0.0:1788880720.510817:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880720.510817:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880720.510822:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880720.510822:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880720.510824:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880720.510825:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880720.510826:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880720.510827:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:3.0:1788880720.510828:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880720.510830:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880720.510832:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880720.510832:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880720.510834:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880720.510834:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880720.510838:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880720.510838:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880720.510838:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:3.0:1788880720.510838:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880720.510842:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880720.510843:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880720.510845:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880720.510846:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009fe01c6d 02000000:00000010:3.0:1788880720.510847:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000056df9207. 00000400:00000010:2.0:1788880720.510849:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009fe01c6d. 02000000:00000001:3.0:1788880720.510851:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880720.510854:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000400:00000010:3.0:1788880720.510858:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000100:00000200:2.0:1788880720.510859:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880736 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880720.510865:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000001:2.0:1788880720.510868:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:3.0:1788880720.510871:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780680192, portal 10 00000100:00000040:2.0:1788880720.510872:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880736 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880720.510877:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880720.510881:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938437248 : -126823771114368 : ffff8ca78a2c7480) 00000100:00000001:2.0:1788880720.510882:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880720.510886:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000040:3.0:1788880720.510892:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880720.510900:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880720.510903:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880720.510905:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005f2b805f. 00000100:00000200:3.0:1788880720.510917:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780680192, offset 0 00000400:00000200:3.0:1788880720.510924:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880720.510932:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880720.510936:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880720.510937:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:3.0:1788880720.510940:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000010:2.0:1788880720.510941:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:2.0:1788880720.510947:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880720.510947:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880720.510955:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880720.510955:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880720.510963:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [388:388:255:4294967295] 192.168.204.111@tcp LPNI seq info [388:388:7:4294967295] 00000400:00000200:3.0:1788880720.510972:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880720.510983:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880720.510984:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000038c5ac58 00000400:00000010:2.0:1788880720.510987:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000038c5ac58. 00000800:00000010:3.0:1788880720.510988:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000001:2.0:1788880720.510991:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880720.510996:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880720.511002:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880720.511013:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880720.511017:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880720.511030:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880720.511031:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880720.511079:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880720.511083:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880720.511103:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880720.511107:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 15518 usecs (16 jiffies) 00000100:00000001:3.0:1788880720.511110:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306351632 00000100:00000001:3.0:1788880720.511113:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306356687 00000100:00000001:3.0:1788880720.511115:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306351632 00000100:00000001:3.0:1788880720.511117:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306356687 00000100:00000001:3.0:1788880720.511119:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306351632 00000100:00000001:3.0:1788880720.511121:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306356687 00000100:00000001:3.0:1788880720.511123:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306351632 00000100:00000001:3.0:1788880720.511125:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306356687 00000100:00000001:3.0:1788880720.511127:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880720.511130:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880720.511132:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880720.511135:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880720.511146:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880720.511148:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:2.0:1788880720.511151:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880720.511154:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880720.511156:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005f2b805f 00000400:00000010:2.0:1788880720.511157:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005f2b805f. 00000100:00000001:2.0:1788880720.511163:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880720.511166:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880720.511170:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880720.511171:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880720.511175:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880720.511176:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880720.512650:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880720.512659:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880720.512661:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880720.512662:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880720.512667:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880720.512678:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0bb00 00000400:00000200:2.0:1788880720.512683:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xce5 [1] + 224 00000800:00000001:2.0:1788880720.512688:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880720.512690:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880720.512692:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880720.512695:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880720.512698:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880720.512701:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880720.512708:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880720.512724:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880720.512729:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880720.512731:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880720.512732:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880720.512734:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880720.512736:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880720.512740:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0bb80 00000400:00000200:2.0:1788880720.512744:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xced [1] + 224 00000800:00000001:2.0:1788880720.512746:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880720.512748:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880720.512749:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880720.512751:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880720.512754:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880720.512756:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880720.512756:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880720.512760:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880720.512765:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880720.512766:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880720.512770:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880720.512774:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000400:00000200:0.0:1788880720.512778:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000100:00000001:2.0:1788880720.512780:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000010:0.0:1788880720.512781:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:0.0:1788880720.512785:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880720.512790:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880720.512793:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880720.512799:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880720.512802:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880720.512803:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880720.512808:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880720.512809:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880720.512813:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880720.512816:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000001:1.0:1788880720.512817:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880720.512820:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880720.512821:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788880720.512821:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880720.512824:0:201286:0:(client.c:1580:after_reply()) Process entered 00000400:00000010:1.0:1788880720.512825:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a3419a3. 02000000:00000001:0.0:1788880720.512828:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880720.512830:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:2.0:1788880720.512833:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0bc00 00000100:00000001:0.0:1788880720.512834:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880720.512835:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000641230df 00000400:00000010:1.0:1788880720.512839:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000641230df. 02000000:00000001:0.0:1788880720.512840:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880720.512843:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:0.0:1788880720.512847:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=98 v=5 (5 5 5 5) 00000400:00000200:2.0:1788880720.512847:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xcf5 [1] + 224 00000100:00000200:1.0:1788880720.512850:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880720.512854:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=98 v=5 (1 1 1 1) 00000800:00000001:2.0:1788880720.512858:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880720.512863:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:2.0:1788880720.512865:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880720.512866:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880720.512868:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880720.512870:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00010000:00000001:0.0:1788880720.512872:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880720.512877:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880720.512880:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880720.512884:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880720.512888:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880720.512888:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:0.0:1788880720.512891:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880720.512892:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880720.512896:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880720.512898:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000200:2.0:1788880720.512900:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880720.512901:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880720.512904:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880720.512908:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880720.512914:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=88 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880720.512922:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=88 v=5 (1 1 1 1) 00000100:00000040:2.0:1788880720.512923:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880720.512929:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880720.512931:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880720.512935:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880720.512938:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880720.512942:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788880720.512945:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880720.512945:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880720.512947:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880720.512948:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880720.512951:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880720.512952:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880720.512955:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880720.512957:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788880720.512958:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880720.512961:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880720.512966:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880720.512974:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880720.512980:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880720.512984:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:0.0:1788880720.512986:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780679936:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880720.512990:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880720.512992:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880720.512994:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880720.512994:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880720.512999:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000001:3.0:1788880720.513000:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880720.513004:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:0.0:1788880720.513006:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ea101a56 x1875776780679936/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880720.513007:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000100:00100000:1.0:1788880720.513011:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000071512652 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780680064:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880720.513013:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:3.0:1788880720.513016:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:3.0:1788880720.513020:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880720.513026:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:3.0:1788880720.513027:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880720.513028:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880720.513030:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880720.513033:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000010:0.0:1788880720.513035:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004ffd965d. 00000100:00000040:1.0:1788880720.513038:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000071512652 x1875776780680064/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880720.513040:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880720.513044:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880720.513047:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000200:3.0:1788880720.513049:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880720.513053:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880720.513055:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000010:0.0:1788880720.513056:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 02000000:00000001:1.0:1788880720.513058:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880720.513060:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004d9329ab. 02000000:00000001:0.0:1788880720.513064:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880720.513065:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:0.0:1788880720.513068:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000020:00000001:1.0:1788880720.513068:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880720.513071:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000010:0.0:1788880720.513072:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ea101a56. 00000100:00000001:3.0:1788880720.513073:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:1.0:1788880720.513074:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880720.513078:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e53a8f34. 00000100:00000001:3.0:1788880720.513078:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880720.513079:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:1.0:1788880720.513081:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880720.513082:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880720.513083:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880720.513083:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788880720.513084:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:1.0:1788880720.513087:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000071512652. 02000000:00000001:3.0:1788880720.513088:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880720.513089:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880720.513090:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880720.513091:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880720.513092:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880720.513093:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880720.513095:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880720.513096:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880720.513097:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880720.513100:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880720.513102:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880720.513103:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880720.513106:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880720.513107:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=98 v=5 (5 5 5 5) 00000100:00000001:1.0:1788880720.513108:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880720.513111:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880720.513112:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=55 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880720.513116:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880720.513117:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880720.513118:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880720.513121:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880720.513125:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880720.513127:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880720.513127:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880720.513128:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880720.513129:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880720.513131:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880720.513134:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880720.513135:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880720.513136:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880720.513137:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880720.513142:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780680192:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880720.513145:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880720.513146:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880720.513147:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000910cb22c x1875776780680192/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880736 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880720.513158:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880720.513160:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880720.513161:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000056df9207. 02000000:00000001:3.0:1788880720.513163:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880720.513164:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880720.513165:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880720.513166:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880720.513167:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c54a5c6f. 02000000:00000001:3.0:1788880720.513169:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880720.513170:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880720.513170:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000910cb22c. 00000100:00000001:3.0:1788880720.513172:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880720.513173:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880720.513174:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880720.513177:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880720.513178:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880720.513179:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880720.513180:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880720.574305:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880720.574311:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880720.574316:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880720.574317:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880720.574383:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880720.574388:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880720.574391:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880721.598362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880721.598368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880721.598372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880721.598373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880721.600522:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880721.600527:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880721.600532:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880722.622336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880722.622343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880722.622348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880722.622350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880722.622397:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880722.622400:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880722.622402:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880723.646363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880723.646372:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880723.646399:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880723.646401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880723.646463:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880723.646467:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880723.646470:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880724.670345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880724.670350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880724.670355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880724.670355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880724.670405:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880724.670409:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880724.670411:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880725.630348:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306356687, at 4306356736 00000100:00000040:1.0:1788880725.630348:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306356687, at 4306356736 00000100:00000040:2.0:1788880725.630363:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306356687, at 4306356736 00000100:00000010:1.0:1788880725.630364:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000bc13d1e4. 00000100:00000010:0.0:1788880725.630368:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001d590ced. 00000100:00000010:2.0:1788880725.630374:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009f22a9a0. 00000100:00000040:1.0:1788880725.630398:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880725.630405:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880725.630414:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000663051e5. 00000020:00000040:1.0:1788880725.630420:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880725.630424:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880725.630427:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880725.630431:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880725.630439:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:1.0:1788880725.630445:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880725.630454:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306356687, at 4306356736 00000100:00000040:1.0:1788880725.630455:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880725.630475:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880725.630478:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880725.630479:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e64ede4. 00000100:00000040:1.0:1788880725.630504:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000663051e5] to pc [ptlrpcd_00_00+0] req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880725.630529:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880725.630531:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880725.630534:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880725.630537:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 00000020:00000040:1.0:1788880725.630539:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880725.630542:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880725.630543:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880725.630545:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880725.630548:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000252f4482. 00000100:00000001:0.0:1788880725.630555:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880725.630558:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880725.630559:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880725.630562:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880725.630567:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880725.630568:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880725.630572:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880725.630572:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880725.630581:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880725.630581:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880725.630582:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880725.630583:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880725.630585:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_00_01+1] req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880725.630586:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880725.630588:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880725.630591:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880725.630594:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780680320:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880725.630598:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880725.630599:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880725.630602:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880725.630604:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880725.630605:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:0.0:1788880725.630607:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880725.630609:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c27d4622. 02000000:00000001:0.0:1788880725.630610:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000020:00000040:1.0:1788880725.630612:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000010:0.0:1788880725.630615:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000094422c44. 00000100:00000001:1.0:1788880725.630615:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880725.630616:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880725.630617:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880725.630617:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880725.630619:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880725.630620:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880725.630621:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880725.630624:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 02000000:00000010:1.0:1788880725.630626:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000040:2.0:1788880725.630628:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880725.630630:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880725.630631:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000200:0.0:1788880725.630638:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780680320, portal 25 00000100:00000040:1.0:1788880725.630638:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880725.630642:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880725.630643:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880725.630645:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938053248 : -126823771498368 : ffff8ca78a269880) 00000100:00000001:1.0:1788880725.630647:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880725.630648:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880725.630650:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880741 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880725.630652:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880725.630654:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c27d4622] to pc [ptlrpcd_00_02+2] req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880725.630658:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880725.630662:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880725.630663:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:2.0:1788880725.630666:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880725.630670:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00100000:2.0:1788880725.630672:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780680448:192.168.204.111@tcp:400: 00000400:00000010:0.0:1788880725.630675:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000033f4d237. 00000100:00000001:2.0:1788880725.630676:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000200:0.0:1788880725.630680:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780680320, offset 0 00000100:00000001:1.0:1788880725.630680:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880725.630680:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880725.630682:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880725.630684:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880725.630685:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880725.630689:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880725.630689:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000010:2.0:1788880725.630691:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007f19b4dd. 00000100:00000040:1.0:1788880725.630694:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11694) 02000000:00000001:2.0:1788880725.630699:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880725.630703:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000048045eba. 00000400:00000200:0.0:1788880725.630705:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880725.630708:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:2.0:1788880725.630708:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005071f3f0. 00000100:00100000:1.0:1788880725.630713:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000400:00000200:0.0:1788880725.630714:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:2.0:1788880725.630714:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780680448, portal 10 00000100:00000001:2.0:1788880725.630720:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880725.630720:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880725.630725:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880725.630729:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 00000100:00000001:1.0:1788880725.630730:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880725.630730:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880725.630736:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880725.630744:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:2.0:1788880725.630744:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880725.630746:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880725.630750:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880725.630756:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [389:389:256:4294967295] 192.168.204.111@tcp LPNI seq info [389:389:8:4294967295] 00000100:00000001:1.0:1788880725.630756:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880725.630757:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880725.630761:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880725.630765:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880725.630765:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880725.630770:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000078d4a4ff. 00000100:00000040:1.0:1788880725.630772:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880725.630773:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780680448, offset 0 00000400:00000200:0.0:1788880725.630782:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880725.630784:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:2.0:1788880725.630785:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880725.630786:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880725.630790:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:0.0:1788880725.630792:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880725.630792:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880725.630794:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880725.630797:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000400:00000200:2.0:1788880725.630797:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00100000:1.0:1788880725.630800:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780680576:192.168.204.111@tcp:400: 00000800:00000200:0.0:1788880725.630805:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880725.630805:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880725.630806:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880725.630811:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788880725.630817:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880725.630818:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:1.0:1788880725.630819:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880725.630822:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880725.630824:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:1.0:1788880725.630824:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880725.630826:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000010:1.0:1788880725.630828:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000033d66d82. 02000000:00000001:1.0:1788880725.630832:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880725.630833:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [390:390:255:4294967295] 192.168.204.111@tcp LPNI seq info [390:390:7:4294967295] 00000400:00000010:1.0:1788880725.630838:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a3419a3. 00000400:00000010:1.0:1788880725.630844:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000641230df. 00000400:00000200:2.0:1788880725.630846:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880725.630847:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880725.630848:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780680576, portal 10 00000100:00000001:1.0:1788880725.630851:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880725.630853:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306361807 00000100:00000001:1.0:1788880725.630854:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938049664 : -126823771501952 : ffff8ca78a268a80) 00000100:00000010:0.0:1788880725.630858:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001d590ced. 00000800:00000200:2.0:1788880725.630858:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880725.630861:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880725.630864:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000010:2.0:1788880725.630866:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000100:00000001:0.0:1788880725.630869:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306361807 00000100:00000001:1.0:1788880725.630872:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:0.0:1788880725.630873:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000085563e6f. 00000800:00000200:2.0:1788880725.630876:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880725.630879:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:2.0:1788880725.630884:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880725.630887:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306361807 00000800:00000200:2.0:1788880725.630889:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 320 niov 1 nkiov 1 00000100:00000040:1.0:1788880725.630890:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880725.630894:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000061cfcb81. 00000100:00000001:2.0:1788880725.630897:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880725.630898:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014321469. 00000100:00000200:1.0:1788880725.630898:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780680576, offset 0 00000100:00000040:2.0:1788880725.630903:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 224257 usecs (225 jiffies) 00000100:00000001:0.0:1788880725.630904:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:1.0:1788880725.630907:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880725.630909:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306361807 00000100:00000001:2.0:1788880725.630911:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306356961 00000100:00000010:0.0:1788880725.630912:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001a2ff840. 00000400:00000200:1.0:1788880725.630918:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880725.630918:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306361807 00000100:00000001:0.0:1788880725.630920:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788880725.630924:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880725.630926:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306356961 00000800:00000200:3.0:1788880725.630926:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880725.630928:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:3.0:1788880725.630936:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000100:00000001:0.0:1788880725.630937:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880725.630937:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306361807 00000400:00000200:1.0:1788880725.630940:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880725.630946:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880725.630948:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880725.630949:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880725.630950:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880725.630951:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306356961 00000100:00000001:0.0:1788880725.630954:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788880725.630955:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880725.630958:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880725.630960:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306361807 00000100:00000001:0.0:1788880725.630961:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880725.630966:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880725.630966:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [391:391:254:4294967295] 192.168.204.111@tcp LPNI seq info [391:391:6:4294967295] 00000400:00000200:3.0:1788880725.630971:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880725.630972:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880725.630972:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306356961 00000100:00000001:0.0:1788880725.630976:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880725.630978:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306361807 00000400:00000200:1.0:1788880725.630983:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880725.630984:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880725.630995:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880725.631001:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880725.631001:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880725.631001:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000033f4d237 00000400:00000010:3.0:1788880725.631009:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000033f4d237. 00000100:00000001:2.0:1788880725.631011:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:1.0:1788880725.631013:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000055d7d92. 00000100:00000001:3.0:1788880725.631019:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880725.631021:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:1.0:1788880725.631023:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880725.631025:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000200:3.0:1788880725.631030:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880741 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880725.631033:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880725.631033:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880725.631038:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:1.0:1788880725.631039:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000055d7d92 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880725.631046:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880725.631049:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880725.631058:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880725.631060:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 15407 usecs (16 jiffies) 00000100:00000001:1.0:1788880725.631070:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306356752 00000100:00000001:1.0:1788880725.631074:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306361807 00000100:00000001:1.0:1788880725.631076:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306356752 00000100:00000001:3.0:1788880725.631076:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880725.631083:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306361807 00000100:00000001:1.0:1788880725.631084:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306356752 00000100:00000040:3.0:1788880725.631088:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880741 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880725.631095:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306361807 00000100:00000001:1.0:1788880725.631100:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306356752 00000100:00000001:3.0:1788880725.631101:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880725.631108:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306361807 00000100:00000001:3.0:1788880725.631108:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880725.631113:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880725.631115:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880725.631117:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880725.631120:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880725.631130:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880725.631134:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000400:00000200:3.0:1788880725.631138:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880725.631142:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880725.631144:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000078d4a4ff 00000400:00000010:3.0:1788880725.631146:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000078d4a4ff. 00000100:00000001:3.0:1788880725.631149:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880725.631152:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880725.631161:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880725.631163:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880725.631168:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880725.631170:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880725.631205:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880725.631208:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000055d7d92. 00000400:00000200:3.0:1788880725.631212:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880725.631215:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880725.631217:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000061cfcb81 00000400:00000010:3.0:1788880725.631219:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000061cfcb81. 00000100:00000001:3.0:1788880725.631222:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880725.631224:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880725.631305:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880725.631308:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880725.631315:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880725.631316:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880725.632771:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880725.632783:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880725.632785:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880725.632786:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880725.632791:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880725.632801:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0bc80 00000400:00000200:2.0:1788880725.632806:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xcfd [1] + 224 00000800:00000001:2.0:1788880725.632811:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880725.632812:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880725.632815:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880725.632818:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880725.632822:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880725.632826:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880725.632832:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880725.632853:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880725.632860:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880725.632862:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880725.632863:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880725.632865:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880725.632867:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880725.632872:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0bd00 00000400:00000200:2.0:1788880725.632876:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd05 [1] + 224 00000800:00000001:2.0:1788880725.632879:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880725.632887:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880725.632891:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880725.632892:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880725.632894:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880725.632897:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880725.632898:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880725.632902:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880725.632904:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880725.632908:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880725.632911:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000100:00000040:2.0:1788880725.632911:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880725.632918:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:0.0:1788880725.632921:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:2.0:1788880725.632925:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880725.632926:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:2.0:1788880725.632929:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880725.632933:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880725.632933:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880725.632937:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880725.632941:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880725.632950:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000200:0.0:1788880725.632953:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880725.632959:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0bd80 00000400:00000200:2.0:1788880725.632968:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd0d [1] + 224 00000100:00000001:0.0:1788880725.632969:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880725.632973:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880725.632974:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880725.632978:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880725.632982:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788880725.632982:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880725.632985:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:2.0:1788880725.632986:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880725.632989:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880725.632991:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880725.632993:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:0.0:1788880725.632995:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880725.632996:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880725.632997:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:3.0:1788880725.633001:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00001000:0.0:1788880725.633004:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=103 v=5 (5 5 5 5) 00000400:00000001:3.0:1788880725.633004:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000200:2.0:1788880725.633005:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880725.633009:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000048045eba. 00000100:00001000:0.0:1788880725.633014:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=103 v=5 (1 1 1 1) 00000400:00000200:3.0:1788880725.633016:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005071f3f0 00000100:00000040:2.0:1788880725.633019:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880725.633020:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005071f3f0. 00000100:00000001:0.0:1788880725.633022:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880725.633026:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880725.633027:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:0.0:1788880725.633032:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000200:3.0:1788880725.633034:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880725.633036:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880725.633037:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880725.633043:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880725.633044:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880725.633050:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880725.633052:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880725.633058:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880725.633066:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880725.633067:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880725.633069:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880725.633070:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880725.633074:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880725.633074:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880725.633079:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:0.0:1788880725.633081:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880725.633083:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880725.633083:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880725.633086:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880725.633088:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880725.633093:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880725.633094:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780680320:192.168.204.111@tcp:400: 00000400:00000001:1.0:1788880725.633094:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880725.633098:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880725.633101:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a3419a3. 00000100:00001000:3.0:1788880725.633101:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=93 v=5 (5 5 5 5) 00000100:00000001:0.0:1788880725.633102:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788880725.633106:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=93 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880725.633110:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880725.633114:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880725.633114:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880725.633118:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:0.0:1788880725.633119:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000663051e5 x1875776780680320/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880725.633120:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000641230df 00010000:00000001:3.0:1788880725.633123:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880725.633124:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000641230df. 00000100:00000001:1.0:1788880725.633129:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880725.633131:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788880725.633134:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:3.0:1788880725.633134:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000200:1.0:1788880725.633135:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880725.633137:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880725.633139:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000010:0.0:1788880725.633140:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000094422c44. 02000000:00000001:0.0:1788880725.633146:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:1.0:1788880725.633148:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880725.633150:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880725.633151:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880725.633154:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880725.633159:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000040:3.0:1788880725.633160:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880725.633161:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:0.0:1788880725.633163:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 00000100:00000001:1.0:1788880725.633164:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880725.633168:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880725.633170:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880725.633171:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:0.0:1788880725.633174:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880725.633174:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880725.633175:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000010:0.0:1788880725.633177:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000663051e5. 00000100:00000001:1.0:1788880725.633178:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880725.633179:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:1.0:1788880725.633182:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880725.633183:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880725.633184:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00001000:1.0:1788880725.633186:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=103 v=5 (5 5 5 5) 00000100:00000001:0.0:1788880725.633187:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880725.633190:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880725.633192:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880725.633193:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=60 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880725.633199:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880725.633200:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880725.633203:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880725.633204:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880725.633208:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:1.0:1788880725.633209:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880725.633212:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880725.633213:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780680448:192.168.204.111@tcp:400: 00010000:00080000:1.0:1788880725.633215:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880725.633221:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880725.633224:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880725.633226:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880725.633227:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880725.633229:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880725.633231:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000040:3.0:1788880725.633251:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780680448/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880725.633259:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880725.633262:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880725.633265:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:1.0:1788880725.633266:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880725.633268:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007f19b4dd. 02000000:00000001:3.0:1788880725.633273:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880725.633277:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880725.633279:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880725.633280:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880725.633283:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:1.0:1788880725.633284:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:3.0:1788880725.633286:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880725.633292:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880725.633294:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000252f4482. 02000000:00000001:3.0:1788880725.633298:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880725.633300:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880725.633302:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000001:3.0:1788880725.633307:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00100000:1.0:1788880725.633309:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780680576:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880725.633310:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880725.633314:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880725.633315:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880725.633317:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880725.633320:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c27d4622 x1875776780680576/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880741 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880725.633320:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880725.633323:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880725.633328:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880725.633329:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880725.633331:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880725.633332:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880725.633335:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000033d66d82. 02000000:00000001:1.0:1788880725.633337:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880725.633338:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880725.633339:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880725.633340:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880725.633341:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 02000000:00000001:1.0:1788880725.633343:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880725.633344:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880725.633344:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c27d4622. 00000100:00000001:1.0:1788880725.633347:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880725.633348:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880725.633349:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880725.633352:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880725.633352:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880725.633354:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880725.633355:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880725.694333:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880725.694339:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880725.694341:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880725.694343:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880725.694347:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880725.694348:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880725.694350:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880725.694351:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880725.694354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880725.694355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880725.694402:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880725.694405:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880725.694406:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880726.718322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880726.718343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880726.718347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880726.718348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880726.718419:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880726.718424:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880726.718444:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880727.742376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880727.742384:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880727.742390:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880727.742394:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880727.742396:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880727.742400:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880727.742403:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880727.742405:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880727.742406:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880727.742410:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880727.742485:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880727.742489:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880727.742491:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880728.766306:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880728.766313:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880728.766319:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880728.766321:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880728.766374:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880728.766378:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880728.766382:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880729.790371:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880729.790378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880729.790384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880729.790385:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880729.790431:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880729.790434:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880729.790436:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880730.750339:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306361807, at 4306361856 00000100:00000040:3.0:1788880730.750339:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306361807, at 4306361856 00000100:00000040:1.0:1788880730.750347:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306361807, at 4306361856 00000100:00000010:2.0:1788880730.750353:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014321469. 00000100:00000010:3.0:1788880730.750356:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001a2ff840. 00000100:00000010:1.0:1788880730.750358:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000085563e6f. 00000100:00000040:0.0:1788880730.750440:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306361807, at 4306361856 00000100:00000010:0.0:1788880730.750446:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001d590ced. 00000100:00000040:0.0:1788880730.750456:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880730.750461:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880730.750468:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000451fe31a. 00000020:00000040:0.0:1788880730.750472:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880730.750476:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880730.750478:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880730.750483:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880730.750490:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000030024c2c. 00000100:00000001:0.0:1788880730.750497:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880730.750506:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880730.750517:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880730.750532:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880730.750539:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000451fe31a] to pc [ptlrpcd_00_03+3] req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880730.750558:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880730.750559:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880730.750562:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880730.750564:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006b32fd51. 00000020:00000040:0.0:1788880730.750565:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880730.750567:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880730.750568:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880730.750570:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880730.750572:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000ac392bfe. 00000100:00000001:0.0:1788880730.750575:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880730.750578:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880730.750581:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880730.750585:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880730.750587:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880730.750587:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880730.750587:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:3.0:1788880730.750592:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880730.750593:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880730.750595:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006b32fd51] to pc [ptlrpcd_00_00+0] req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880730.750599:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880730.750602:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880730.750604:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880730.750605:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880730.750607:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880730.750612:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880730.750619:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880730.750620:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880730.750622:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880730.750624:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880730.750626:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880730.750628:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780680704:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880730.750632:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880730.750636:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880730.750638:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880730.750640:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880730.750640:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880730.750643:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008d264e32. 00000100:00000001:2.0:1788880730.750644:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880730.750647:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880730.750648:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:3.0:1788880730.750653:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006860a6e1. 00000100:00000040:2.0:1788880730.750654:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880730.750659:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000200:3.0:1788880730.750666:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780680704, portal 25 00000100:00000001:2.0:1788880730.750667:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880730.750668:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880730.750670:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880730.750671:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849366784 : -126822860184832 : ffff8ca7c0782300) 02000000:00000001:2.0:1788880730.750676:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1788880730.750678:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880746 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880730.750685:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880730.750688:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880730.750692:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780680832:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880730.750701:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880730.750701:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880730.750702:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880730.750706:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880730.750707:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880730.750708:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880730.750709:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 02000000:00000001:2.0:1788880730.750711:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880730.750713:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780680704, offset 0 02000000:00000001:2.0:1788880730.750717:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880730.750718:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000400:00000200:3.0:1788880730.750722:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880730.750723:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ff0be673. 00000020:00000040:0.0:1788880730.750728:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000010:2.0:1788880730.750728:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000070c1ee08. 00000100:00000001:1.0:1788880730.750729:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880730.750732:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880730.750732:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880730.750733:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:2.0:1788880730.750734:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880730.750735:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880730.750739:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880730.750741:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000998db106. 00000400:00000200:3.0:1788880730.750741:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:0.0:1788880730.750746:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 00000400:00000200:3.0:1788880730.750747:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:2.0:1788880730.750749:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f779aa21. 00000100:00000001:0.0:1788880730.750753:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880730.750754:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:2.0:1788880730.750755:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780680832, portal 10 00000400:00000200:3.0:1788880730.750759:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880730.750760:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880730.750764:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880730.750764:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849362304 : -126822860189312 : ffff8ca7c0781180) 00000400:00000200:3.0:1788880730.750765:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [392:392:256:4294967295] 192.168.204.111@tcp LPNI seq info [392:392:8:4294967295] 00000100:00000040:2.0:1788880730.750776:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880730.750788:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880730.750801:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:3.0:1788880730.750805:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880730.750809:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880730.750810:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880730.750812:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880730.750813:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c6ecbca9. 00000800:00000010:3.0:1788880730.750813:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000040:0.0:1788880730.750817:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ff0be673] to pc [ptlrpcd_00_01+1] req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880730.750819:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780680832, offset 0 00000800:00000200:3.0:1788880730.750826:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880730.750832:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880730.750838:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880730.750840:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880730.750841:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788880730.750842:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000040:0.0:1788880730.750844:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:2.0:1788880730.750850:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:0.0:1788880730.750851:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:2.0:1788880730.750856:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880730.750857:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880730.750858:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11699) 00000100:00000001:1.0:1788880730.750859:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880730.750862:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880730.750862:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880730.750862:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306366927 00000100:00000001:1.0:1788880730.750867:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880730.750869:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:3.0:1788880730.750869:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c451d2b9. 00000100:00000040:1.0:1788880730.750874:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880730.750874:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:2.0:1788880730.750878:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [393:393:255:4294967295] 192.168.204.111@tcp LPNI seq info [393:393:7:4294967295] 00000100:00000001:3.0:1788880730.750881:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306366927 00000100:00000010:3.0:1788880730.750885:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001f846689. 00000100:00000001:3.0:1788880730.750890:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880730.750894:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880730.750895:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306366927 00000400:00000200:2.0:1788880730.750900:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:3.0:1788880730.750900:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e62b7132. 00000100:00000001:1.0:1788880730.750902:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880730.750905:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 02000000:00000001:1.0:1788880730.750908:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880730.750911:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306366927 02000000:00000001:1.0:1788880730.750912:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880730.750913:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788880730.750916:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fbefa178. 02000000:00000001:1.0:1788880730.750918:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880730.750920:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e53a8f34. 00000100:00000001:3.0:1788880730.750924:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:2.0:1788880730.750929:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880730.750930:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00100000:1.0:1788880730.750934:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780680960:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880730.750935:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880730.750940:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880730.750941:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880730.750943:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880730.750944:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880730.750947:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e53a8f34 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:1.0:1788880730.750948:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880730.750952:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880730.750952:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880730.750956:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880730.750958:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 222351 usecs (223 jiffies) 02000000:00000010:1.0:1788880730.750961:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000033d66d82. 00000800:00000200:0.0:1788880730.750962:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880730.750962:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306362079 02000000:00000001:1.0:1788880730.750965:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880730.750966:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000100:00000001:2.0:1788880730.750968:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306366927 00000400:00000200:0.0:1788880730.750970:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:1.0:1788880730.750971:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a3419a3. 00000400:00000010:1.0:1788880730.750975:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000641230df. 00000400:00000200:0.0:1788880730.750977:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:1.0:1788880730.750979:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780680960, portal 10 00000100:00000001:1.0:1788880730.750981:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880730.750981:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306362079 00000400:00000200:0.0:1788880730.750983:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000100:00000001:1.0:1788880730.750984:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000400:00000010:0.0:1788880730.750987:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:2.0:1788880730.750987:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306366927 00000100:00000040:1.0:1788880730.750990:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880730.750993:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880730.750993:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306362079 00000100:00000200:0.0:1788880730.750998:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880746 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880730.750999:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306366927 00000100:00000001:1.0:1788880730.751002:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880730.751004:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306362079 00000100:00000040:1.0:1788880730.751007:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880730.751010:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306366927 00000100:00000001:0.0:1788880730.751012:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000010:1.0:1788880730.751012:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000100:00000001:2.0:1788880730.751014:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788880730.751015:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780680960, offset 0 00000100:00000040:0.0:1788880730.751016:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880746 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880730.751020:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880730.751025:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880730.751027:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880730.751028:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880730.751032:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:1.0:1788880730.751032:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880730.751036:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880730.751039:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880730.751042:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880730.751043:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:0.0:1788880730.751045:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:1.0:1788880730.751045:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880730.751046:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880730.751049:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e53a8f34. 00000400:00000200:1.0:1788880730.751050:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880730.751052:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880730.751053:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880730.751053:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880730.751058:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880730.751059:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880730.751061:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880730.751064:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [394:394:255:4294967295] 192.168.204.111@tcp LPNI seq info [394:394:7:4294967295] 00000400:00000200:1.0:1788880730.751073:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:0.0:1788880730.751077:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c6ecbca9 00000800:00000200:1.0:1788880730.751077:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880730.751079:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c6ecbca9. 00000800:00000010:1.0:1788880730.751080:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000100:00000001:0.0:1788880730.751081:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880730.751084:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880730.751084:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880730.751088:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880730.751091:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880730.751096:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880730.751098:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880730.751098:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880730.751101:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 15298 usecs (16 jiffies) 00000100:00000001:0.0:1788880730.751104:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880730.751104:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306361872 00000100:00000001:0.0:1788880730.751106:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880730.751106:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306366927 00000100:00000001:1.0:1788880730.751108:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306361872 00000100:00000001:1.0:1788880730.751110:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306366927 00000100:00000001:1.0:1788880730.751112:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306361872 00000100:00000001:1.0:1788880730.751113:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306366927 00000100:00000001:1.0:1788880730.751115:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306361872 00000100:00000001:1.0:1788880730.751117:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306366927 00000100:00000001:1.0:1788880730.751118:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880730.751120:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880730.751122:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880730.751123:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880730.751127:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880730.751128:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880730.751131:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880730.751132:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880730.751133:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880730.751134:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880730.751135:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880730.751136:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:0.0:1788880730.751139:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880730.751142:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880730.751145:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000400:00000010:0.0:1788880730.751146:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:0.0:1788880730.751149:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880730.751151:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880730.751157:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880730.751159:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880730.751166:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880730.751167:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880730.752379:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880730.752495:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880730.752497:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880730.752500:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880730.752504:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880730.752511:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0be00 00000400:00000200:3.0:1788880730.752516:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xd15 [1] + 224 00000800:00000001:3.0:1788880730.752521:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880730.752624:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880730.752626:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880730.752629:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880730.752632:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880730.752635:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880730.752641:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880730.752653:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880730.752657:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880730.752659:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880730.752660:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880730.752661:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880730.752664:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880730.752667:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0be80 00000400:00000200:3.0:1788880730.752671:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd1d [1] + 224 00000800:00000001:3.0:1788880730.752674:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880730.752676:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880730.752677:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880730.752684:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880730.752687:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880730.752691:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:1.0:1788880730.752694:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880730.752697:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880730.752699:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006860a6e1. 00000400:00000200:1.0:1788880730.752703:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000400:00000010:1.0:1788880730.752704:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:1.0:1788880730.752707:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880730.752708:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880730.752713:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880730.752714:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880730.752730:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880730.752740:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880730.752745:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880730.752749:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880730.752751:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880730.752754:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880730.752756:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880730.752764:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880730.752768:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:3.0:1788880730.752771:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0bf00 00000100:00000001:2.0:1788880730.752772:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880730.752773:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880730.752775:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880730.752779:0:201287:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:3.0:1788880730.752779:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd25 [1] + 224 02000000:00000001:1.0:1788880730.752783:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000001:2.0:1788880730.752785:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788880730.752786:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880730.752787:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000001:2.0:1788880730.752788:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880730.752791:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880730.752791:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000998db106. 00000400:00000200:2.0:1788880730.752795:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f779aa21 02000000:00000001:1.0:1788880730.752796:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880730.752798:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f779aa21. 00000100:00000001:2.0:1788880730.752802:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:3.0:1788880730.752803:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880730.752805:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=108 v=5 (5 5 5 5) 00000100:00000200:2.0:1788880730.752807:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880730.752808:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00001000:1.0:1788880730.752813:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=108 v=5 (1 1 1 1) 00000400:00000200:3.0:1788880730.752819:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880730.752821:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880730.752825:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880730.752826:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880730.752827:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:1.0:1788880730.752832:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000200:3.0:1788880730.752833:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880730.752836:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880730.752842:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880730.752850:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880730.752850:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880730.752853:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880730.752854:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880730.752857:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880730.752860:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880730.752866:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880730.752868:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880730.752871:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880730.752876:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=98 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880730.752877:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:2.0:1788880730.752879:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=98 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880730.752881:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:3.0:1788880730.752882:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880730.752884:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880730.752884:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880730.752886:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788880730.752888:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880730.752891:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880730.752892:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880730.752895:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880730.752895:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880730.752898:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000040:1.0:1788880730.752901:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880730.752901:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880730.752904:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880730.752910:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880730.752911:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880730.752914:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00100000:1.0:1788880730.752914:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780680704:192.168.204.111@tcp:400: 00000400:00000001:0.0:1788880730.752919:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880730.752921:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880730.752924:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880730.752925:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a3419a3. 00000100:00000001:2.0:1788880730.752926:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880730.752930:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1788880730.752931:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000641230df 00000100:00000001:2.0:1788880730.752931:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880730.752933:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880730.752935:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000641230df. 00000100:00000040:1.0:1788880730.752935:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000451fe31a x1875776780680704/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880730.752939:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880730.752940:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880730.752947:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880730.752955:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880730.752959:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880730.752961:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008d264e32. 02000000:00000001:1.0:1788880730.752967:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00100000:2.0:1788880730.752969:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780680832:192.168.204.111@tcp:400: 00000020:00000001:1.0:1788880730.752970:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000200:0.0:1788880730.752971:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:1.0:1788880730.752973:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880730.752975:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880730.752979:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000020:00000001:1.0:1788880730.752983:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000040:2.0:1788880730.752984:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006b32fd51 x1875776780680832/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880730.752986:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:1.0:1788880730.752987:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000030024c2c. 02000000:00000001:1.0:1788880730.752992:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880730.752993:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880730.752994:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880730.752995:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880730.752997:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000010:1.0:1788880730.752998:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000451fe31a. 02000000:00000001:2.0:1788880730.752999:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:0.0:1788880730.753000:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880730.753002:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000010:2.0:1788880730.753002:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000070c1ee08. 00000100:00000001:0.0:1788880730.753003:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880730.753005:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880730.753006:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880730.753008:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880730.753008:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:0.0:1788880730.753009:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880730.753011:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880730.753014:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000020:00000040:2.0:1788880730.753014:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00001000:0.0:1788880730.753015:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=108 v=5 (5 5 5 5) 00000020:00000001:2.0:1788880730.753016:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880730.753017:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880730.753019:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000ac392bfe. 00000100:00000001:1.0:1788880730.753022:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:0.0:1788880730.753023:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=65 v=5 (1 1 0 0) 00000100:00000001:1.0:1788880730.753025:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880730.753029:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880730.753030:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000001:2.0:1788880730.753031:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880730.753033:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880730.753034:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006b32fd51. 00000100:00000001:2.0:1788880730.753038:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880730.753040:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880730.753042:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880730.753044:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880730.753047:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880730.753049:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880730.753052:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00080000:0.0:1788880730.753055:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880730.753055:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880730.753064:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880730.753068:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880730.753069:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880730.753071:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880730.753073:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880730.753076:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880730.753082:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880730.753084:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880730.753085:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880730.753087:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880730.753094:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780680960:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880730.753098:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880730.753100:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880730.753102:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ff0be673 x1875776780680960/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880746 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880730.753108:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880730.753109:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880730.753116:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000033d66d82. 02000000:00000001:0.0:1788880730.753119:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880730.753121:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880730.753122:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880730.753124:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880730.753125:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 02000000:00000001:0.0:1788880730.753128:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880730.753129:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880730.753130:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ff0be673. 00000100:00000001:0.0:1788880730.753132:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880730.753134:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880730.753135:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880730.753139:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880730.753140:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880730.753141:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880730.753142:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880730.814317:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880730.814324:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880730.814331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880730.814332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880730.814381:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880730.814384:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880730.814386:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880731.838323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880731.838328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880731.838336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880731.838337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880731.838379:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880731.838383:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880731.838385:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880732.862297:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880732.862307:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880732.862321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880732.862322:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880732.862424:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880732.862431:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880732.862451:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880733.886367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880733.886373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880733.886380:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880733.886381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880733.886425:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880733.886430:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880733.886433:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880734.910368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880734.910376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880734.910383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880734.910384:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880734.910440:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880734.910444:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880734.910447:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880735.870304:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306366927, at 4306366976 00000100:00000040:1.0:1788880735.870306:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306366927, at 4306366976 00000100:00000040:3.0:1788880735.870309:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306366927, at 4306366976 00000100:00000010:1.0:1788880735.870319:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001f846689. 00000100:00000010:2.0:1788880735.870321:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e62b7132. 00000100:00000010:3.0:1788880735.870323:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fbefa178. 00000100:00000040:0.0:1788880735.870328:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306366927, at 4306366976 00000100:00000010:0.0:1788880735.870337:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c451d2b9. 00000100:00000040:1.0:1788880735.870348:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880735.870353:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880735.870359:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c27d4622. 00000020:00000040:1.0:1788880735.870373:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880735.870376:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880735.870378:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880735.870382:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880735.870387:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:1.0:1788880735.870393:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880735.870400:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880735.870407:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880735.870409:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880735.870413:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c27d4622] to pc [ptlrpcd_00_02+2] req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880735.870432:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880735.870434:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880735.870436:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880735.870438:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000094565713. 00000020:00000040:1.0:1788880735.870439:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880735.870441:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880735.870442:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880735.870443:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880735.870446:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5b428a0. 00000100:00000001:1.0:1788880735.870448:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880735.870448:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880735.870451:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880735.870453:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880735.870454:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880735.870457:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880735.870461:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880735.870461:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880735.870462:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880735.870464:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880735.870469:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000094565713] to pc [ptlrpcd_00_03+3] req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880735.870483:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880735.870485:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880735.870488:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880735.870490:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880735.870491:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880735.870492:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880735.870496:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788880735.870499:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780681088:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880735.870502:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880735.870504:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880735.870507:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880735.870508:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:3.0:1788880735.870508:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880735.870510:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000010:1.0:1788880735.870511:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 02000000:00000001:3.0:1788880735.870512:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880735.870516:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:2.0:1788880735.870517:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880735.870517:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880735.870522:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000010:3.0:1788880735.870523:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000010b36294. 02000000:00000001:1.0:1788880735.870525:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880735.870527:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880735.870529:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880735.870534:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000010:1.0:1788880735.870536:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:2.0:1788880735.870536:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880735.870537:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007f8335d2. 00000100:00000001:1.0:1788880735.870542:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880735.870542:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:3.0:1788880735.870544:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000082116f14. 02000000:00000001:2.0:1788880735.870545:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880735.870549:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880735.870549:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880735.870550:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780681088, portal 25 00000100:00000001:3.0:1788880735.870555:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00100000:2.0:1788880735.870556:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780681216:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880735.870557:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938049664 : -126823771501952 : ffff8ca78a268a80) 00000100:00000001:2.0:1788880735.870564:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880735.870567:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880751 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880735.870569:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788880735.870569:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880735.870572:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880735.870573:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880735.870578:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880735.870580:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca5e4c87] to pc [ptlrpcd_00_00+0] req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880735.870580:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880735.870584:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880735.870588:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ec90f18d. 02000000:00000010:2.0:1788880735.870591:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009330bb87. 02000000:00000001:2.0:1788880735.870594:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880735.870598:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780681088, offset 0 00000400:00000010:2.0:1788880735.870599:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000998db106. 00000100:00000001:1.0:1788880735.870600:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880735.870603:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:3.0:1788880735.870607:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880735.870608:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:2.0:1788880735.870614:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f779aa21. 00000100:00000040:1.0:1788880735.870618:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11704) 00000100:00000200:2.0:1788880735.870619:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780681216, portal 10 00000400:00000200:3.0:1788880735.870621:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880735.870625:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880735.870628:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880735.870628:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055040 : -126823771496576 : ffff8ca78a269f80) 00000400:00000200:3.0:1788880735.870628:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880735.870633:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880735.870634:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880735.870635:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880735.870638:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880735.870638:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880735.870640:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1788880735.870643:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880735.870646:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880735.870653:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880735.870662:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [395:395:256:4294967295] 192.168.204.111@tcp LPNI seq info [395:395:8:4294967295] 00000100:00000001:2.0:1788880735.870665:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880735.870672:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880735.870673:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880735.870676:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880735.870676:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c6ecbca9. 00000100:00000200:2.0:1788880735.870680:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780681216, offset 0 00000400:00000200:3.0:1788880735.870686:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:2.0:1788880735.870689:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880735.870692:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880735.870695:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880735.870698:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880735.870699:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880735.870704:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:3.0:1788880735.870704:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00100000:0.0:1788880735.870707:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780681344:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880735.870711:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:3.0:1788880735.870715:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880735.870716:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880735.870716:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880735.870723:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1788880735.870723:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1788880735.870727:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880735.870727:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880735.870730:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788880735.870732:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:3.0:1788880735.870733:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 02000000:00000010:0.0:1788880735.870739:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e8a109c2. 00000400:00000200:2.0:1788880735.870743:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [396:396:255:4294967295] 192.168.204.111@tcp LPNI seq info [396:396:7:4294967295] 02000000:00000001:0.0:1788880735.870744:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880735.870746:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880735.870750:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306372047 00000100:00000010:3.0:1788880735.870756:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fbefa178. 00000400:00000010:0.0:1788880735.870757:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 00000400:00000200:2.0:1788880735.870758:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880735.870761:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000010:0.0:1788880735.870765:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000001:3.0:1788880735.870767:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306372047 00000100:00000200:0.0:1788880735.870771:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780681344, portal 10 00000800:00000200:2.0:1788880735.870772:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788880735.870773:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000dba5f48d. 00000100:00000001:0.0:1788880735.870778:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880735.870779:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000010:2.0:1788880735.870780:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000eaa15d72. 00000100:00000001:0.0:1788880735.870781:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000001:3.0:1788880735.870782:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306372047 00000100:00000010:3.0:1788880735.870787:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000837757ae. 00000800:00000200:2.0:1788880735.870789:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880735.870790:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000040:0.0:1788880735.870794:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880735.870797:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306372047 00000800:00000200:2.0:1788880735.870802:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:3.0:1788880735.870803:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000945234cf. 00000800:00000200:2.0:1788880735.870809:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000eaa15d72 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880735.870811:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880735.870812:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880735.870815:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880735.870816:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880735.870820:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880735.870821:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 220475 usecs (221 jiffies) 00000100:00000001:3.0:1788880735.870821:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880735.870825:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 00000100:00000001:2.0:1788880735.870825:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306367197 00000100:00000001:2.0:1788880735.870827:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306372047 00000100:00000001:3.0:1788880735.870831:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880735.870832:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780681344, offset 0 00000100:00000001:3.0:1788880735.870834:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880735.870839:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306367197 00000800:00000200:1.0:1788880735.870843:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880735.870843:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306372047 00000400:00000200:0.0:1788880735.870844:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880735.870844:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:1.0:1788880735.870848:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000100:00000001:2.0:1788880735.870849:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306367197 00000100:00000001:3.0:1788880735.870849:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880735.870854:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880735.870854:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880735.870855:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306372047 00000100:00000001:3.0:1788880735.870856:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880735.870860:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306367197 00000100:00000001:3.0:1788880735.870860:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880735.870864:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880735.870864:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880735.870865:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306372047 00000100:00000001:3.0:1788880735.870865:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880735.870871:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880735.870872:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880735.870873:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880735.870873:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880735.870878:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880735.870878:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880735.870882:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880735.870885:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880735.870887:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880735.870889:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880735.870891:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [397:397:254:4294967295] 192.168.204.111@tcp LPNI seq info [397:397:6:4294967295] 00000100:00000001:2.0:1788880735.870892:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880735.870894:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880735.870896:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880735.870899:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880735.870908:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880735.870915:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880735.870915:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ec90f18d 00000400:00000010:1.0:1788880735.870919:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ec90f18d. 00000800:00000010:0.0:1788880735.870920:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000da26fd2f. 00000100:00000001:1.0:1788880735.870924:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:0.0:1788880735.870928:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:1.0:1788880735.870932:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880751 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880735.870935:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880735.870939:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000da26fd2f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880735.870945:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880735.870948:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 15188 usecs (16 jiffies) 00000100:00000001:1.0:1788880735.870950:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880735.870951:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306366992 00000100:00000001:0.0:1788880735.870955:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306372047 00000100:00000040:1.0:1788880735.870955:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880751 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880735.870957:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306366992 00000100:00000001:0.0:1788880735.870958:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306372047 00000100:00000001:0.0:1788880735.870967:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306366992 00000100:00000001:1.0:1788880735.870967:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880735.870969:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306372047 00000100:00000001:1.0:1788880735.870970:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880735.870973:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306366992 00000100:00000001:0.0:1788880735.870975:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306372047 00000100:00000001:0.0:1788880735.870976:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880735.870977:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880735.870979:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880735.870981:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880735.870983:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880735.870984:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880735.870986:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:1.0:1788880735.870986:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880735.870989:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000010:1.0:1788880735.870990:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000eaa15d72. 00000100:00000001:0.0:1788880735.870991:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880735.870994:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880735.870994:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880735.871001:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880735.871004:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c6ecbca9 00000400:00000010:1.0:1788880735.871005:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c6ecbca9. 00000100:00000001:1.0:1788880735.871008:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880735.871010:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880735.871015:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880735.871017:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880735.871021:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880735.871022:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880735.871029:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880735.871030:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000da26fd2f. 00000400:00000200:1.0:1788880735.871031:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880735.871033:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880735.871034:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000400:00000010:1.0:1788880735.871035:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000001:1.0:1788880735.871037:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880735.871038:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880735.871042:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880735.871043:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880735.871046:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880735.871047:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880735.872274:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880735.872284:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880735.872287:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880735.872289:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880735.872307:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880735.872317:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0bf80 00000400:00000200:0.0:1788880735.872324:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xd2d [1] + 224 00000800:00000001:0.0:1788880735.872329:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880735.872335:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880735.872337:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880735.872340:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880735.872344:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880735.872347:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880735.872353:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880735.872364:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880735.872375:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880735.872377:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880735.872449:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880735.872451:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880735.872454:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880735.872456:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880735.872457:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007f8335d2. 00000400:00000200:3.0:1788880735.872460:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000082116f14 00000400:00000010:3.0:1788880735.872461:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000082116f14. 00000100:00000001:3.0:1788880735.872464:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880735.872467:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880735.872469:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1788880735.872476:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880735.872477:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880735.872482:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880735.872485:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880735.872491:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880735.872500:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c000 00000400:00000200:2.0:1788880735.872505:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd35 [1] + 224 00000800:00000001:2.0:1788880735.872509:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880735.872512:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880735.872513:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880735.872517:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880735.872521:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880735.872523:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880735.872526:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880735.872527:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880735.872528:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880735.872531:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880735.872533:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880735.872536:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880735.872539:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880735.872545:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880735.872546:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=113 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880735.872549:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=113 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880735.872551:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880735.872551:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880735.872555:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880735.872558:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880735.872561:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880735.872567:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880735.872581:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880735.872581:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880735.872585:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880735.872586:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880735.872589:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880735.872590:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:3.0:1788880735.872595:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780681088:192.168.204.111@tcp:400: 00000800:00000001:2.0:1788880735.872596:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880735.872602:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880735.872602:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880735.872606:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880735.872608:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880735.872608:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880735.872609:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c27d4622 x1875776780681088/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880735.872611:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880735.872618:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000001:1.0:1788880735.872621:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880735.872621:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 02000000:00000001:3.0:1788880735.872621:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000001:1.0:1788880735.872625:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000010:3.0:1788880735.872625:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000010b36294. 00000400:00000010:1.0:1788880735.872628:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000998db106. 02000000:00000001:3.0:1788880735.872629:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880735.872632:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000200:1.0:1788880735.872633:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f779aa21 00000400:00000200:2.0:1788880735.872633:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c080 00000020:00000040:3.0:1788880735.872634:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000010:1.0:1788880735.872636:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f779aa21. 00000020:00000001:3.0:1788880735.872640:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880735.872642:0:201288:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000010:3.0:1788880735.872643:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 00000400:00000200:2.0:1788880735.872645:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd3d [1] + 224 00000100:00000200:1.0:1788880735.872649:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880735.872650:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000800:00000001:2.0:1788880735.872651:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788880735.872654:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880735.872657:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c27d4622. 00000800:00000001:2.0:1788880735.872658:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880735.872660:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880735.872663:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880735.872665:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880735.872669:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880735.872671:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:2.0:1788880735.872672:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880735.872673:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880735.872675:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880735.872679:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880735.872679:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880735.872681:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:1.0:1788880735.872682:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880735.872684:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880735.872685:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000200:2.0:1788880735.872685:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880735.872687:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880735.872688:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880735.872691:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880735.872692:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880735.872699:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=103 v=5 (5 5 5 5) 00000100:00000040:2.0:1788880735.872701:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880735.872703:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=103 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880735.872709:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880735.872711:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880735.872716:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880735.872718:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880735.872720:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880735.872721:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880735.872722:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880735.872723:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880735.872725:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880735.872727:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880735.872729:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880735.872733:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880735.872735:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880735.872737:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880735.872739:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880735.872740:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880735.872752:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780681216:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880735.872756:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880735.872757:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880735.872758:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000094565713 x1875776780681216/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880735.872763:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880735.872763:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880735.872765:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880735.872767:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:1.0:1788880735.872768:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009330bb87. 02000000:00000001:1.0:1788880735.872772:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880735.872773:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000001:0.0:1788880735.872774:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000040:1.0:1788880735.872775:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000001:0.0:1788880735.872778:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000020:00000001:1.0:1788880735.872779:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000010:0.0:1788880735.872781:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 02000000:00000010:1.0:1788880735.872782:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5b428a0. 00000400:00000200:0.0:1788880735.872787:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 02000000:00000001:1.0:1788880735.872788:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000010:0.0:1788880735.872792:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 02000000:00000001:1.0:1788880735.872792:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880735.872795:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000094565713. 00000100:00000001:0.0:1788880735.872797:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880735.872800:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880735.872802:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880735.872804:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880735.872806:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880735.872813:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880735.872815:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880735.872816:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880735.872821:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880735.872825:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880735.872825:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880735.872827:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880735.872829:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880735.872831:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880735.872831:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880735.872832:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880735.872834:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880735.872837:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=113 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880735.872839:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=70 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880735.872841:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880735.872841:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880735.872843:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880735.872844:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880735.872849:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880735.872850:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880735.872851:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880735.872852:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880735.872852:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880735.872854:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880735.872866:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880735.872867:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880735.872868:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880735.872870:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880735.872874:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780681344:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880735.872876:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880735.872877:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880735.872878:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780681344/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880751 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880735.872882:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880735.872883:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880735.872884:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e8a109c2. 02000000:00000001:0.0:1788880735.872886:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880735.872887:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880735.872887:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880735.872889:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880735.872889:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 02000000:00000001:0.0:1788880735.872891:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880735.872892:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880735.872893:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:0.0:1788880735.872894:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880735.872895:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880735.872896:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880735.872898:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880735.872899:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880735.872900:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880735.872901:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880735.934357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880735.934365:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880735.934368:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880735.934370:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880735.934375:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880735.934378:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880735.934380:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880735.934382:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880735.934386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880735.934388:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880735.934424:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880735.934428:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880735.934430:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880736.958380:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880736.958413:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880736.958421:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880736.958422:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880736.958486:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880736.958489:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880736.958492:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880737.982373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880737.982381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880737.982387:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880737.982390:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880737.982392:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880737.982395:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880737.982397:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880737.982399:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880737.982400:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880737.982402:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880737.982448:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880737.982453:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880737.982456:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880739.006349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880739.006355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880739.006360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880739.006361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880739.006413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880739.006418:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880739.006440:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880740.030316:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880740.030321:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880740.030326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880740.030327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880740.030353:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880740.030357:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880740.030360:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880740.990352:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306372047, at 4306372096 00000100:00000040:1.0:1788880740.990352:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306372047, at 4306372096 00000100:00000040:2.0:1788880740.990352:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306372047, at 4306372096 00000100:00000010:2.0:1788880740.990371:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000837757ae. 00000100:00000010:1.0:1788880740.990373:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000dba5f48d. 00000100:00000010:0.0:1788880740.990375:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fbefa178. 00000100:00000040:0.0:1788880740.990485:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880740.990497:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880740.990502:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ff0be673. 00000020:00000040:0.0:1788880740.990507:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880740.990511:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880740.990514:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880740.990519:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880740.990526:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000004e0f6846. 00000100:00000001:0.0:1788880740.990533:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880740.990543:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880740.990552:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880740.990555:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880740.990561:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ff0be673] to pc [ptlrpcd_00_01+1] req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880740.990581:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880740.990584:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880740.990587:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880740.990591:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000118779b0. 00000020:00000040:0.0:1788880740.990594:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880740.990596:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880740.990598:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880740.990600:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880740.990603:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c93f60a2. 00000100:00000001:0.0:1788880740.990607:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880740.990607:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788880740.990612:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000040:0.0:1788880740.990613:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880740.990617:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880740.990625:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880740.990626:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880740.990628:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880740.990630:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880740.990634:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000118779b0] to pc [ptlrpcd_00_02+2] req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880740.990635:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880740.990638:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880740.990642:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880740.990659:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880740.990663:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880740.990668:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880740.990668:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880740.990672:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880740.990673:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880740.990674:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880740.990677:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880740.990677:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880740.990680:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1788880740.990682:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:0.0:1788880740.990685:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001d70df98. 00000100:00000040:2.0:1788880740.990686:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1788880740.990691:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880740.990694:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880740.990698:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880740.990702:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880740.990704:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:0.0:1788880740.990706:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880740.990706:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780681472:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880740.990707:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880740.990713:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880740.990714:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000010:0.0:1788880740.990715:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 02000000:00000001:2.0:1788880740.990717:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880740.990720:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880740.990721:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880740.990722:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880740.990725:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880740.990728:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000118779b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780681600:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880740.990730:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880740.990730:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880740.990736:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000cf77ae66. 00000100:00000001:2.0:1788880740.990736:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880740.990740:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880740.990741:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880740.990744:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880740.990748:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:1.0:1788880740.990749:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b34466e7. 00000100:00000001:0.0:1788880740.990752:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000010:2.0:1788880740.990754:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000094e066a1. 00000100:00000001:0.0:1788880740.990755:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880740.990757:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000062e8e6f4. 02000000:00000001:2.0:1788880740.990758:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880740.990763:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001d70df98] to pc [ptlrpcd_00_03+3] req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880740.990765:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780681472, portal 25 00000400:00000010:2.0:1788880740.990765:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bcbb0958. 00000100:00000001:1.0:1788880740.990771:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000010:2.0:1788880740.990772:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005071f3f0. 00000100:00000001:1.0:1788880740.990774:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000100:00000200:2.0:1788880740.990779:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780681600, portal 10 00000100:00000040:1.0:1788880740.990784:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880756 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880740.990784:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880740.990785:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880740.990788:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849360512 : -126822860191104 : ffff8ca7c0780a80) 00000100:00000040:0.0:1788880740.990792:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880740.990799:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880740.990802:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880740.990804:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880740.990812:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11709) 00000100:00000040:1.0:1788880740.990812:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880740.990817:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d71e0867. 00000100:00000001:2.0:1788880740.990818:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:1.0:1788880740.990822:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780681472, offset 0 00000100:00000040:2.0:1788880740.990823:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880740.990825:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:2.0:1788880740.990828:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000078d4a4ff. 00000100:00000001:0.0:1788880740.990829:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880740.990833:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788880740.990834:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880740.990840:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880740.990841:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780681600, offset 0 00000400:00000200:1.0:1788880740.990849:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880740.990854:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880740.990858:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1788880740.990859:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880740.990861:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880740.990866:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880740.990868:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880740.990870:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880740.990871:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00100000:0.0:1788880740.990875:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000001d70df98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780681728:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880740.990878:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880740.990882:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880740.990888:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [398:398:256:4294967295] 192.168.204.111@tcp LPNI seq info [398:398:8:4294967295] 00000100:00000001:0.0:1788880740.990894:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880740.990898:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880740.990900:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880740.990903:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880740.990908:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000070c1ee08. 00000400:00000200:1.0:1788880740.990909:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880740.990914:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880740.990919:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000fcf1d693. 00000800:00000200:1.0:1788880740.990920:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880740.990922:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880740.990927:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:0.0:1788880740.990928:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000800:00000010:1.0:1788880740.990928:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000400:00000200:2.0:1788880740.990932:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:0.0:1788880740.990935:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780681728, portal 10 00000400:00000200:2.0:1788880740.990936:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:1.0:1788880740.990940:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880740.990942:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880740.990945:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849368576 : -126822860183040 : ffff8ca7c0782a00) 00000400:00000200:2.0:1788880740.990949:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880740.990954:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:0.0:1788880740.990957:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880740.990958:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [399:399:255:4294967295] 192.168.204.111@tcp LPNI seq info [399:399:7:4294967295] 00000800:00000200:1.0:1788880740.990968:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880740.990974:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:2.0:1788880740.990978:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880740.990983:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880740.990983:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880740.990988:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880740.990989:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000100:00000001:1.0:1788880740.990990:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306377167 00000800:00000010:2.0:1788880740.990994:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000045330cf1. 00000100:00000200:0.0:1788880740.990996:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780681728, offset 0 00000100:00000010:1.0:1788880740.990997:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000800:00000200:2.0:1788880740.991002:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880740.991006:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880740.991007:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880740.991012:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880740.991013:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306377167 00000800:00000200:2.0:1788880740.991017:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000045330cf1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880740.991018:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:1.0:1788880740.991019:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000100:00000001:2.0:1788880740.991023:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880740.991025:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880740.991027:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:2.0:1788880740.991029:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 218630 usecs (219 jiffies) 00000100:00000001:1.0:1788880740.991031:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306377167 00000400:00000200:0.0:1788880740.991034:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880740.991034:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306372315 00000100:00000010:1.0:1788880740.991038:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000100:00000001:2.0:1788880740.991039:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306377167 00000400:00000200:0.0:1788880740.991041:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880740.991044:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880740.991045:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306372315 00000400:00000200:0.0:1788880740.991048:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880740.991050:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306377167 00000100:00000001:2.0:1788880740.991050:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306377167 00000100:00000001:2.0:1788880740.991055:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306372315 00000400:00000200:0.0:1788880740.991056:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [400:400:254:4294967295] 192.168.204.111@tcp LPNI seq info [400:400:6:4294967295] 00000100:00000001:1.0:1788880740.991058:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306372047 00000100:00000001:2.0:1788880740.991061:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306377167 00000100:00000001:1.0:1788880740.991065:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880740.991066:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306372315 00000100:00000001:1.0:1788880740.991068:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880740.991071:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306377167 00000100:00000001:1.0:1788880740.991074:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880740.991076:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880740.991079:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880740.991079:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880740.991080:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880740.991087:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880740.991091:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880740.991091:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880740.991092:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880740.991096:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880740.991098:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ac392bfe. 00000100:00000001:2.0:1788880740.991099:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880740.991101:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880740.991105:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:0.0:1788880740.991106:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880740.991108:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880740.991111:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880740.991115:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880740.991116:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880740.991121:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ac392bfe type 1, nob 320 niov 1 nkiov 1 00000800:00000200:1.0:1788880740.991123:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880740.991125:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880740.991128:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000400:00000200:1.0:1788880740.991133:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:0.0:1788880740.991134:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 15080 usecs (16 jiffies) 00000100:00000001:0.0:1788880740.991139:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306372112 00000400:00000200:1.0:1788880740.991143:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880740.991144:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306377167 00000100:00000001:0.0:1788880740.991150:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306372112 00000100:00000001:0.0:1788880740.991152:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306377167 00000400:00000200:1.0:1788880740.991153:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d71e0867 00000100:00000001:0.0:1788880740.991154:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306372112 00000400:00000010:1.0:1788880740.991158:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d71e0867. 00000100:00000001:0.0:1788880740.991159:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306377167 00000100:00000001:0.0:1788880740.991163:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306372112 00000100:00000001:1.0:1788880740.991164:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880740.991166:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306377167 00000100:00000001:0.0:1788880740.991170:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788880740.991170:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880756 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880740.991175:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880740.991181:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880740.991185:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880740.991188:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880740.991190:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788880740.991205:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880756 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880740.991209:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880740.991215:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880740.991217:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880740.991219:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880740.991220:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880740.991222:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880740.991227:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880740.991263:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880740.991266:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000045330cf1. 00000400:00000200:1.0:1788880740.991270:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880740.991275:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880740.991278:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000078d4a4ff 00000400:00000010:1.0:1788880740.991280:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000078d4a4ff. 00000100:00000001:1.0:1788880740.991283:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880740.991287:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880740.991294:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880740.991297:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880740.991303:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880740.991304:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880740.991324:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880740.991326:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ac392bfe. 00000400:00000200:1.0:1788880740.991331:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880740.991335:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880740.991338:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000400:00000010:1.0:1788880740.991339:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:1.0:1788880740.991343:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880740.991346:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880740.991353:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880740.991355:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880740.991361:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880740.991362:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880740.993015:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880740.993024:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880740.993028:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880740.993030:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880740.993037:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880740.993048:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0c100 00000400:00000200:0.0:1788880740.993055:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xd45 [1] + 224 00000800:00000001:0.0:1788880740.993061:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880740.993064:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880740.993067:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880740.993070:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880740.993076:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880740.993087:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880740.993097:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880740.993112:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880740.993120:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880740.993124:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880740.993126:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880740.993128:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880740.993132:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880740.993138:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c180 00000400:00000200:0.0:1788880740.993143:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd49 [1] + 224 00000800:00000001:0.0:1788880740.993147:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880740.993152:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880740.993154:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880740.993155:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880740.993158:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880740.993162:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:1.0:1788880740.993165:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880740.993168:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880740.993170:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b34466e7. 00000400:00000200:1.0:1788880740.993188:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000062e8e6f4 00000400:00000010:1.0:1788880740.993190:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000062e8e6f4. 00000100:00000001:1.0:1788880740.993193:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880740.993206:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880740.993208:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880740.993214:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880740.993226:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880740.993252:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880740.993262:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880740.993268:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880740.993275:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880740.993276:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880740.993277:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880740.993280:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880740.993283:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880740.993284:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000001:2.0:1788880740.993287:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880740.993288:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000001:2.0:1788880740.993289:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880740.993292:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880740.993292:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000bcbb0958. 00000400:00000200:2.0:1788880740.993297:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005071f3f0 00000800:00000001:0.0:1788880740.993298:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880740.993298:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880740.993300:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005071f3f0. 00000800:00000001:0.0:1788880740.993305:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880740.993305:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:1.0:1788880740.993307:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=118 v=5 (5 5 5 5) 00000800:00000001:0.0:1788880740.993311:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880740.993311:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880740.993314:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=118 v=5 (1 1 1 1) 00000400:00000200:0.0:1788880740.993322:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880740.993322:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880740.993326:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880740.993329:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880740.993331:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:0.0:1788880740.993335:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c200 00010000:00000001:1.0:1788880740.993335:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880740.993339:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880740.993341:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880740.993342:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880740.993346:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd55 [1] + 224 00000100:00000001:2.0:1788880740.993347:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880740.993349:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:1.0:1788880740.993351:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880740.993351:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:0.0:1788880740.993354:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880740.993355:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880740.993359:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880740.993367:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00001000:2.0:1788880740.993367:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=108 v=5 (5 5 5 5) 00000800:00000001:0.0:1788880740.993368:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880740.993372:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:0.0:1788880740.993373:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00001000:2.0:1788880740.993374:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=108 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880740.993376:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880740.993382:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:0.0:1788880740.993383:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:1.0:1788880740.993385:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880740.993386:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880740.993390:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880740.993396:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880740.993397:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880740.993401:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880740.993402:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780681472:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880740.993407:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:1.0:1788880740.993411:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880740.993411:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880740.993416:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880740.993416:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880740.993420:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880740.993421:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880740.993423:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ff0be673 x1875776780681472/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880740.993429:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880740.993454:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880740.993459:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880740.993461:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880740.993462:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880740.993462:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880740.993464:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880740.993467:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000cf77ae66. 00000800:00000001:0.0:1788880740.993470:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:1.0:1788880740.993471:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:2.0:1788880740.993471:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788880740.993474:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000800:00000001:0.0:1788880740.993476:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:1.0:1788880740.993477:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880740.993482:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880740.993485:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000004e0f6846. 00000100:00000001:3.0:1788880740.993486:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880740.993488:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880740.993490:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880740.993493:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880740.993495:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ff0be673. 00000400:00000001:3.0:1788880740.993495:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880740.993499:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880740.993500:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000400:00000010:3.0:1788880740.993502:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000fcf1d693. 00000100:00000001:1.0:1788880740.993504:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:2.0:1788880740.993507:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000118779b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780681600:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880740.993507:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000100:00000001:1.0:1788880740.993508:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880740.993511:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:1.0:1788880740.993514:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880740.993514:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880740.993515:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880740.993518:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880740.993520:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880740.993522:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:3.0:1788880740.993522:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880740.993525:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880740.993527:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000118779b0 x1875776780681600/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880740.993544:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880740.993546:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880740.993548:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000094e066a1. 00000100:00000200:3.0:1788880740.993550:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880740.993553:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880740.993556:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880740.993558:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880740.993558:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880740.993561:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788880740.993564:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880740.993564:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000010:2.0:1788880740.993567:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c93f60a2. 02000000:00000001:3.0:1788880740.993567:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880740.993570:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:2.0:1788880740.993572:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880740.993572:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880740.993576:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:3.0:1788880740.993577:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880740.993579:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000118779b0. 00000100:00000001:2.0:1788880740.993582:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00001000:3.0:1788880740.993582:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=118 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880740.993584:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880740.993588:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788880740.993589:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=75 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880740.993592:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880740.993592:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880740.993595:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880740.993595:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880740.993599:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:3.0:1788880740.993599:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880740.993601:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:3.0:1788880740.993604:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880740.993609:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880740.993610:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880740.993611:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880740.993612:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880740.993613:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880740.993614:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880740.993626:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880740.993628:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880740.993628:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880740.993630:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880740.993635:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000001d70df98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780681728:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880740.993637:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880740.993638:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880740.993639:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000001d70df98 x1875776780681728/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880756 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880740.993645:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880740.993646:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880740.993647:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000070c1ee08. 02000000:00000001:3.0:1788880740.993650:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880740.993651:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880740.993651:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880740.993653:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880740.993654:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 02000000:00000001:3.0:1788880740.993655:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880740.993656:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880740.993657:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000001d70df98. 00000100:00000001:3.0:1788880740.993659:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880740.993660:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880740.993661:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880740.993664:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880740.993664:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880740.993666:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880740.993667:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880741.054335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880741.054343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880741.054349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880741.054350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880741.054401:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880741.054404:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880741.054406:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880742.078320:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880742.078325:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880742.078330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880742.078331:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880742.078372:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880742.078376:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880742.078379:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880743.102315:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880743.102321:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880743.102326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880743.102327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880743.102354:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880743.102359:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880743.102362:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880744.126339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880744.126347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880744.126355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880744.126356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880744.126405:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880744.126408:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880744.126411:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880745.150333:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880745.150340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880745.150344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880745.150362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880745.150406:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880745.150415:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880745.150417:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880746.110330:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306377167, at 4306377216 00000100:00000040:2.0:1788880746.110330:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306377167, at 4306377216 00000100:00000040:3.0:1788880746.110330:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306377167, at 4306377216 00000100:00000040:0.0:1788880746.110344:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306377167, at 4306377216 00000100:00000010:3.0:1788880746.110346:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000945234cf. 00000100:00000010:1.0:1788880746.110347:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000010:0.0:1788880746.110356:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:2.0:1788880746.110358:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000040:1.0:1788880746.110382:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880746.110386:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880746.110390:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000094565713. 00000020:00000040:1.0:1788880746.110394:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880746.110398:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880746.110400:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880746.110404:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880746.110410:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5b428a0. 00000100:00000001:1.0:1788880746.110416:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880746.110424:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880746.110431:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880746.110433:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880746.110437:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000094565713] to pc [ptlrpcd_00_00+0] req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880746.110457:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880746.110458:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880746.110461:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880746.110462:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c09231ec. 00000020:00000040:1.0:1788880746.110464:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880746.110466:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880746.110467:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880746.110468:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880746.110470:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:1.0:1788880746.110472:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880746.110475:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880746.110475:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880746.110481:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880746.110483:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880746.110485:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880746.110485:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880746.110490:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880746.110490:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880746.110500:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880746.110503:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c09231ec] to pc [ptlrpcd_00_01+1] req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880746.110522:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880746.110522:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880746.110524:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880746.110525:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880746.110529:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:0.0:1788880746.110531:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000010:1.0:1788880746.110532:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000003b122924. 00000020:00000040:1.0:1788880746.110535:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880746.110536:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880746.110537:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880746.110540:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880746.110540:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880746.110541:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880746.110544:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880746.110545:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00100000:0.0:1788880746.110548:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780681856:192.168.204.111@tcp:400: 02000000:00000010:1.0:1788880746.110550:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:2.0:1788880746.110550:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880746.110556:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880746.110556:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880746.110559:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880746.110563:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880746.110564:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880746.110568:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880746.110574:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880746.110580:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ed94d533. 02000000:00000001:0.0:1788880746.110585:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880746.110586:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880746.110587:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880746.110590:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880746.110590:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880746.110593:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000048e2c403. 02000000:00000001:2.0:1788880746.110596:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:1.0:1788880746.110597:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003b122924] to pc [ptlrpcd_00_02+2] req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880746.110601:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 02000000:00000001:2.0:1788880746.110601:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880746.110607:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880746.110610:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780681856, portal 25 00000100:00100000:2.0:1788880746.110613:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780681984:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880746.110616:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880746.110619:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880746.110621:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:2.0:1788880746.110622:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880746.110624:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055040 : -126823771496576 : ffff8ca78a269f80) 00000100:00000001:1.0:1788880746.110625:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880746.110626:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880746.110631:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880746.110633:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880762 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880746.110633:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000094e066a1. 00000100:00000040:1.0:1788880746.110638:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:2.0:1788880746.110639:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880746.110646:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bcbb0958. 00000100:00000001:3.0:1788880746.110647:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880746.110648:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880746.110648:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11714) 00000100:00000001:3.0:1788880746.110650:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:2.0:1788880746.110651:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005071f3f0. 00000100:00000001:3.0:1788880746.110653:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880746.110656:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788880746.110656:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780681984, portal 10 00000100:00000001:2.0:1788880746.110660:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:3.0:1788880746.110660:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880746.110662:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938056832 : -126823771494784 : ffff8ca78a26a680) 00000400:00000010:0.0:1788880746.110663:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d6cef011. 00000100:00000200:0.0:1788880746.110669:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780681856, offset 0 00000100:00000001:3.0:1788880746.110671:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880746.110673:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880746.110674:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880746.110677:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880746.110677:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880746.110678:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788880746.110681:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880746.110682:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880746.110687:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880746.110689:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00100000:3.0:1788880746.110690:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780682112:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880746.110691:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880746.110694:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:2.0:1788880746.110695:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000006bff322d. 02000000:00000001:3.0:1788880746.110696:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000200:2.0:1788880746.110698:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780681984, offset 0 02000000:00000001:3.0:1788880746.110698:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880746.110700:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880746.110701:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880746.110706:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000010:3.0:1788880746.110706:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a581a743. 02000000:00000001:3.0:1788880746.110708:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880746.110709:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:3.0:1788880746.110711:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007f8335d2. 00000400:00000200:0.0:1788880746.110715:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:3.0:1788880746.110717:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000082116f14. 00000400:00000200:0.0:1788880746.110721:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:3.0:1788880746.110721:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780682112, portal 10 00000100:00000001:3.0:1788880746.110726:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880746.110728:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880746.110729:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938062208 : -126823771489408 : ffff8ca78a26bb80) 00000400:00000200:0.0:1788880746.110736:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [401:401:256:4294967295] 192.168.204.111@tcp LPNI seq info [401:401:8:4294967295] 00000100:00000040:3.0:1788880746.110737:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880746.110749:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880746.110754:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880746.110757:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005b101ac6. 00000400:00000200:0.0:1788880746.110761:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:3.0:1788880746.110761:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780682112, offset 0 00000400:00000200:3.0:1788880746.110769:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880746.110773:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880746.110774:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788880746.110781:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000400:00000200:2.0:1788880746.110782:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880746.110787:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788880746.110791:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880746.110792:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880746.110796:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880746.110799:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [402:402:255:4294967295] 192.168.204.111@tcp LPNI seq info [402:402:7:4294967295] 00000800:00000200:0.0:1788880746.110804:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880746.110811:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880746.110811:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880746.110819:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880746.110820:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880746.110822:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 00000400:00000200:3.0:1788880746.110823:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880746.110825:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880746.110826:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880746.110827:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880746.110829:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880746.110830:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788880746.110831:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880746.110835:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880746.110835:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [403:403:254:4294967295] 192.168.204.111@tcp LPNI seq info [403:403:6:4294967295] 00000100:00000001:0.0:1788880746.110837:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306382287 00000100:00000001:2.0:1788880746.110840:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880746.110844:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001d590ced. 00000100:00000040:2.0:1788880746.110844:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 216821 usecs (217 jiffies) 00000400:00000200:3.0:1788880746.110845:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880746.110850:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306377433 00000100:00000001:0.0:1788880746.110853:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880746.110860:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306382287 00000100:00000001:2.0:1788880746.110860:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306382287 00000800:00000200:3.0:1788880746.110860:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880746.110864:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306377433 00000800:00000010:3.0:1788880746.110865:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000010:0.0:1788880746.110867:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006bd48548. 00000800:00000200:3.0:1788880746.110870:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880746.110873:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:3.0:1788880746.110877:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880746.110879:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306382287 00000100:00000001:2.0:1788880746.110879:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306382287 00000800:00000200:3.0:1788880746.110881:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880746.110882:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306377433 00000100:00000001:3.0:1788880746.110886:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880746.110888:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000029f028ab. 00000100:00000040:3.0:1788880746.110889:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 14980 usecs (15 jiffies) 00000100:00000001:3.0:1788880746.110891:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306377231 00000800:00000200:1.0:1788880746.110892:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880746.110894:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306382287 00000800:00000010:1.0:1788880746.110899:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000100:00000001:3.0:1788880746.110899:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306377231 00000100:00000001:0.0:1788880746.110900:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880746.110904:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306382287 00000100:00000001:2.0:1788880746.110907:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306382287 00000100:00000001:0.0:1788880746.110908:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306382287 00000400:00000200:1.0:1788880746.110908:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880746.110908:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306377231 00000100:00000001:2.0:1788880746.110910:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306377433 00000100:00000001:3.0:1788880746.110915:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306382287 00000100:00000010:0.0:1788880746.110917:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c0bfe00. 00000100:00000001:3.0:1788880746.110918:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306377231 00000400:00000200:1.0:1788880746.110921:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880746.110923:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880746.110928:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880746.110928:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306382287 00000400:00000200:1.0:1788880746.110931:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d6cef011 00000100:00000001:2.0:1788880746.110933:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880746.110933:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306382287 00000100:00000001:0.0:1788880746.110934:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880746.110935:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d6cef011. 00000100:00000001:3.0:1788880746.110937:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880746.110938:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880746.110939:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880746.110940:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880746.110940:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880746.110946:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880746.110946:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880746.110946:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880746.110948:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880746.110952:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880746.110955:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880762 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880746.110956:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880746.110957:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880746.110957:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880746.110960:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880746.110962:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880746.110962:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880746.110965:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880746.110969:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880746.110972:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880746.110973:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880746.110974:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880746.110975:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880746.110975:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880746.110977:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880746.110977:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880746.110980:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880762 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880746.110980:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880746.110980:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880746.110983:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880746.110984:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880746.110993:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880746.110994:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880746.111012:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880746.111013:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000400:00000200:1.0:1788880746.111015:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880746.111017:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880746.111019:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000006bff322d 00000400:00000010:1.0:1788880746.111020:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000006bff322d. 00000100:00000001:1.0:1788880746.111021:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880746.111023:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880746.111027:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880746.111028:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880746.111031:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880746.111032:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880746.111086:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880746.111088:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:1.0:1788880746.111091:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880746.111094:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880746.111106:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005b101ac6 00000400:00000010:1.0:1788880746.111108:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005b101ac6. 00000100:00000001:1.0:1788880746.111110:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880746.111113:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880746.111119:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880746.111121:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880746.111131:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880746.111133:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880746.112985:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880746.112994:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880746.112996:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880746.112997:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880746.113002:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880746.113012:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0c280 00000400:00000200:1.0:1788880746.113019:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xd5d [1] + 224 00000800:00000001:1.0:1788880746.113025:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880746.113029:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880746.113031:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880746.113034:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880746.113047:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880746.113054:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880746.113063:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880746.113078:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880746.113085:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880746.113089:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880746.113090:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880746.113092:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880746.113096:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880746.113111:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c300 00000400:00000200:1.0:1788880746.113116:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd61 [1] + 224 00000800:00000001:1.0:1788880746.113119:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880746.113121:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880746.113122:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880746.113125:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880746.113125:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880746.113129:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880746.113130:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880746.113136:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:1.0:1788880746.113139:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880746.113140:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880746.113143:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000048e2c403. 00000400:00000200:0.0:1788880746.113149:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000100:00000040:1.0:1788880746.113150:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880746.113153:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000001:0.0:1788880746.113158:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880746.113164:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880746.113167:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880746.113172:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880746.113175:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880746.113176:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880746.113179:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880746.113181:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880746.113190:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880746.113193:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880746.113196:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880746.113198:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c380 00000100:00000001:0.0:1788880746.113200:0:201285:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:1.0:1788880746.113204:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd6d [1] + 224 02000000:00000001:0.0:1788880746.113205:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:1.0:1788880746.113207:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880746.113207:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880746.113208:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880746.113211:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880746.113212:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880746.113214:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880746.113217:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880746.113217:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:2.0:1788880746.113217:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880746.113222:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:1.0:1788880746.113223:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00001000:0.0:1788880746.113226:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=124 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880746.113231:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=124 v=5 (1 1 1 1) 00000400:00000010:2.0:1788880746.113253:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000bcbb0958. 00000400:00000200:2.0:1788880746.113258:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005071f3f0 00000400:00000010:2.0:1788880746.113261:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005071f3f0. 00000100:00000001:0.0:1788880746.113264:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880746.113265:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880746.113266:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880746.113272:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880746.113273:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880746.113273:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880746.113278:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880746.113283:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880746.113289:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880746.113298:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880746.113307:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880746.113307:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880746.113313:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880746.113319:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880746.113322:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880746.113331:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880746.113339:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880746.113343:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880746.113345:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:0.0:1788880746.113349:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780681856:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880746.113349:0:201286:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:1.0:1788880746.113352:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:2.0:1788880746.113353:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:1.0:1788880746.113356:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880746.113357:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880746.113358:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880746.113361:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880746.113362:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:2.0:1788880746.113364:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880746.113368:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000094565713 x1875776780681856/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:2.0:1788880746.113371:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=114 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880746.113377:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:2.0:1788880746.113378:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=114 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880746.113381:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880746.113384:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880746.113386:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000001:0.0:1788880746.113388:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000001:3.0:1788880746.113388:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880746.113389:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880746.113391:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000010:0.0:1788880746.113393:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ed94d533. 00010000:00000001:2.0:1788880746.113393:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000010:3.0:1788880746.113395:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007f8335d2. 00010000:00000001:2.0:1788880746.113397:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880746.113398:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000200:3.0:1788880746.113401:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000082116f14 00000020:00000001:0.0:1788880746.113402:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880746.113402:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880746.113406:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000010:3.0:1788880746.113406:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000082116f14. 00000020:00000040:0.0:1788880746.113407:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880746.113412:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880746.113412:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000001:0.0:1788880746.113413:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880746.113415:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880746.113417:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5b428a0. 00000100:00000200:3.0:1788880746.113421:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880746.113422:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880746.113425:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880746.113428:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880746.113430:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000094565713. 00000100:00000001:0.0:1788880746.113436:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880746.113439:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880746.113444:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880746.113451:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:3.0:1788880746.113452:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880746.113454:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880746.113458:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880746.113460:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880746.113463:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880746.113468:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880746.113469:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880746.113471:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880746.113472:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880746.113475:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880746.113479:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:2.0:1788880746.113481:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880746.113482:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880746.113485:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880746.113489:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880746.113495:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=124 v=5 (5 5 5 5) 00000100:00100000:2.0:1788880746.113496:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780681984:192.168.204.111@tcp:400: 00000100:00001000:3.0:1788880746.113503:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=81 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880746.113505:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880746.113508:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880746.113509:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880746.113511:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880746.113514:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c09231ec x1875776780681984/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880746.113515:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880746.113522:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880746.113529:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880746.113532:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880746.113535:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000094e066a1. 00010000:00000001:3.0:1788880746.113539:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880746.113541:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880746.113544:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880746.113545:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880746.113547:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880746.113548:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880746.113550:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880746.113552:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880746.113554:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880746.113557:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 00000100:00000040:3.0:1788880746.113561:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880746.113562:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880746.113565:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880746.113567:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c09231ec. 00000100:00000001:2.0:1788880746.113572:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880746.113574:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880746.113576:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880746.113577:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880746.113582:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880746.113584:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880746.113585:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880746.113588:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880746.113591:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880746.113592:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880746.113594:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880746.113604:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780682112:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880746.113609:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880746.113610:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880746.113613:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000003b122924 x1875776780682112/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880762 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880746.113619:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880746.113620:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880746.113623:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a581a743. 02000000:00000001:3.0:1788880746.113626:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880746.113627:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880746.113628:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880746.113630:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880746.113631:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 02000000:00000001:3.0:1788880746.113634:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880746.113635:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880746.113636:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000003b122924. 00000100:00000001:3.0:1788880746.113639:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880746.113640:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880746.113641:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880746.113645:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880746.113646:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880746.113647:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880746.113649:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880746.174330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880746.174337:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880746.174339:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880746.174341:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880746.174346:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880746.174347:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880746.174349:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880746.174350:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880746.174369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880746.174370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880746.174419:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880746.174421:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880746.174423:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880747.199361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880747.199368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880747.199374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880747.199377:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880747.199379:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880747.199382:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880747.199385:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880747.199387:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880747.199388:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880747.199390:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880747.199443:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880747.199447:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880747.199451:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880748.222374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880748.222384:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880748.222391:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880748.222393:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880748.222431:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880748.222436:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880748.222440:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880749.246402:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880749.246409:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880749.246415:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880749.246416:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880749.246469:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880749.246472:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880749.246474:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880750.270313:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880750.270318:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880750.270322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880750.270323:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880750.270364:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880750.270380:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880750.270382:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880751.230323:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306382287, at 4306382336 00000100:00000040:1.0:1788880751.230323:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306382287, at 4306382336 00000100:00000040:2.0:1788880751.230323:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306382287, at 4306382336 00000100:00000040:3.0:1788880751.230324:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306382287, at 4306382336 00000100:00000010:1.0:1788880751.230338:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006bd48548. 00000100:00000010:0.0:1788880751.230339:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001d590ced. 00000100:00000010:2.0:1788880751.230341:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000029f028ab. 00000100:00000010:3.0:1788880751.230343:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c0bfe00. 00000100:00000040:2.0:1788880751.230355:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880751.230359:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880751.230365:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cc13c26b. 00000020:00000040:2.0:1788880751.230369:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880751.230372:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880751.230374:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880751.230377:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880751.230383:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:2.0:1788880751.230389:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880751.230396:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880751.230404:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880751.230405:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880751.230409:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cc13c26b] to pc [ptlrpcd_00_03+3] req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880751.230429:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880751.230430:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880751.230433:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880751.230434:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000020:00000040:2.0:1788880751.230436:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880751.230437:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880751.230438:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880751.230439:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880751.230440:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 00000100:00000001:2.0:1788880751.230442:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880751.230444:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880751.230450:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880751.230454:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880751.230456:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880751.230461:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880751.230462:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880751.230466:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880751.230480:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880751.230483:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880751.230484:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880751.230488:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880751.230488:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_00+0] req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880751.230495:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880751.230498:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880751.230502:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880751.230513:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780682240:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880751.230514:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880751.230520:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880751.230521:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880751.230526:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880751.230528:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880751.230528:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:2.0:1788880751.230530:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000005a8bb1de. 00000100:00000001:0.0:1788880751.230532:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880751.230533:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880751.230535:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:2.0:1788880751.230536:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880751.230539:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880751.230540:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880751.230540:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880751.230542:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880751.230545:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000dc9842d3. 02000000:00000001:2.0:1788880751.230546:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880751.230546:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880751.230550:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880751.230551:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880751.230558:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f47bc5fc. 00000100:00000001:0.0:1788880751.230559:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880751.230561:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880751.230562:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b34466e7. 02000000:00000001:0.0:1788880751.230564:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880751.230564:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880751.230567:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788880751.230570:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000062e8e6f4. 02000000:00000001:0.0:1788880751.230572:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880751.230572:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000200:1.0:1788880751.230578:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780682240, portal 25 00000100:00100000:0.0:1788880751.230580:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780682368:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880751.230584:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880751.230587:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938433664 : -126823771117952 : ffff8ca78a2c6680) 00000100:00000001:0.0:1788880751.230590:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880751.230593:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788880751.230596:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:1.0:1788880751.230596:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880767 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880751.230597:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880751.230600:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880751.230606:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880751.230608:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a8bb1de] to pc [ptlrpcd_00_01+1] req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880751.230612:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ed94d533. 00000100:00000001:1.0:1788880751.230614:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880751.230616:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880751.230620:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880751.230623:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000048e2c403. 00000400:00000010:1.0:1788880751.230625:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d71e0867. 00000400:00000010:0.0:1788880751.230630:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 00000100:00000200:1.0:1788880751.230630:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780682240, offset 0 00000100:00000200:0.0:1788880751.230637:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780682368, portal 10 00000100:00000001:2.0:1788880751.230639:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880751.230641:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880751.230642:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880751.230645:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 00000100:00000040:2.0:1788880751.230646:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880751.230652:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788880751.230654:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880751.230656:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880751.230657:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880751.230660:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11720) 00000100:00000001:3.0:1788880751.230661:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880751.230662:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880751.230665:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880751.230670:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880751.230670:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788880751.230672:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880751.230678:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:0.0:1788880751.230679:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880751.230685:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880751.230687:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880751.230690:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880751.230691:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [404:404:256:4294967295] 192.168.204.111@tcp LPNI seq info [404:404:8:4294967295] 00000400:00000010:0.0:1788880751.230692:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a09430ba. 02000000:00000001:3.0:1788880751.230693:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880751.230695:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880751.230699:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780682368, offset 0 02000000:00000001:3.0:1788880751.230699:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880751.230706:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780682496:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880751.230709:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:0.0:1788880751.230710:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880751.230715:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880751.230718:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000800:00000200:1.0:1788880751.230720:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880751.230720:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880751.230725:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880751.230725:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000010:1.0:1788880751.230727:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000400:00000200:0.0:1788880751.230730:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:3.0:1788880751.230730:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a4a3549f. 02000000:00000001:3.0:1788880751.230733:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880751.230736:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007f8335d2. 00000800:00000200:1.0:1788880751.230738:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:3.0:1788880751.230742:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000082116f14. 00000400:00000200:0.0:1788880751.230743:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880751.230747:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:3.0:1788880751.230747:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780682496, portal 10 00000100:00000001:3.0:1788880751.230752:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:1.0:1788880751.230753:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880751.230755:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880751.230755:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938427392 : -126823771124224 : ffff8ca78a2c4e00) 00000800:00000200:1.0:1788880751.230761:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880751.230762:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [405:405:255:4294967295] 192.168.204.111@tcp LPNI seq info [405:405:7:4294967295] 00000100:00000040:3.0:1788880751.230765:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880751.230778:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788880751.230780:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880751.230784:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880751.230789:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880751.230789:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880751.230791:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306387407 00000400:00000010:3.0:1788880751.230793:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000035d99bc2. 00000100:00000200:3.0:1788880751.230797:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780682496, offset 0 00000800:00000010:0.0:1788880751.230798:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000100:00000010:1.0:1788880751.230798:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000400:00000200:3.0:1788880751.230803:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880751.230806:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:0.0:1788880751.230811:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880751.230811:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306387407 00000400:00000200:3.0:1788880751.230811:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:1.0:1788880751.230817:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000400:00000200:3.0:1788880751.230819:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880751.230820:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:0.0:1788880751.230824:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880751.230825:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880751.230826:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306387407 00000800:00000200:0.0:1788880751.230829:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 00000100:00000010:1.0:1788880751.230831:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000400:00000200:3.0:1788880751.230831:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880751.230836:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880751.230837:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:3.0:1788880751.230838:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:0.0:1788880751.230841:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 215043 usecs (216 jiffies) 00000100:00000001:1.0:1788880751.230843:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306387407 00000100:00000001:0.0:1788880751.230846:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306382552 00000800:00000200:2.0:1788880751.230846:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:3.0:1788880751.230846:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [406:406:254:4294967295] 192.168.204.111@tcp LPNI seq info [406:406:6:4294967295] 00000100:00000010:1.0:1788880751.230847:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000800:00000010:2.0:1788880751.230853:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000100:00000001:0.0:1788880751.230854:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306387407 00000100:00000001:1.0:1788880751.230854:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880751.230859:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880751.230860:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306382552 00000400:00000200:2.0:1788880751.230862:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880751.230863:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880751.230864:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880751.230865:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306387407 00000100:00000001:0.0:1788880751.230870:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306382552 00000100:00000001:1.0:1788880751.230870:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880751.230874:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880751.230875:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880751.230876:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306387407 00000100:00000001:0.0:1788880751.230879:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306382552 00000100:00000001:0.0:1788880751.230881:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306387407 00000100:00000001:1.0:1788880751.230882:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:3.0:1788880751.230883:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880751.230886:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880751.230888:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880751.230890:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880751.230891:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d71e0867 00000100:00000001:0.0:1788880751.230892:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880751.230892:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:1.0:1788880751.230894:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:2.0:1788880751.230896:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d71e0867. 00000100:00000001:0.0:1788880751.230899:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880751.230899:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880751.230900:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880751.230904:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880751.230907:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880751.230907:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880751.230911:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:2.0:1788880751.230912:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880767 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880751.230916:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:3.0:1788880751.230916:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880751.230920:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880751.230922:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880751.230927:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880751.230928:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 14882 usecs (15 jiffies) 00000100:00000001:0.0:1788880751.230930:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880751.230932:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306382351 00000100:00000001:0.0:1788880751.230934:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880751.230936:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306387407 00000100:00000001:2.0:1788880751.230939:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880751.230940:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306382351 00000100:00000001:3.0:1788880751.230944:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306387407 00000100:00000001:0.0:1788880751.230946:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880751.230947:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880767 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880751.230947:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306382351 00000100:00000001:3.0:1788880751.230950:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306387407 00000100:00000001:3.0:1788880751.230953:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306382351 00000100:00000001:3.0:1788880751.230957:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306387407 00000100:00000001:3.0:1788880751.230960:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880751.230964:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880751.230966:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880751.230968:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880751.230971:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880751.230972:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880751.230976:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880751.230978:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880751.230979:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880751.230981:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880751.230982:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880751.230983:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880751.230989:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880751.230993:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000400:00000200:2.0:1788880751.230997:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880751.231010:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880751.231013:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a09430ba 00000400:00000010:2.0:1788880751.231015:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a09430ba. 00000100:00000001:2.0:1788880751.231018:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880751.231020:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880751.231027:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880751.231029:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880751.231034:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880751.231036:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880751.231093:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880751.231096:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:2.0:1788880751.231099:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880751.231103:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880751.231106:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000035d99bc2 00000400:00000010:2.0:1788880751.231108:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000035d99bc2. 00000100:00000001:2.0:1788880751.231110:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880751.231113:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880751.231131:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880751.231133:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880751.231139:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880751.231140:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880751.232730:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880751.232758:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880751.232761:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880751.232763:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880751.232769:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880751.232779:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0c400 00000400:00000200:1.0:1788880751.232786:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xd75 [1] + 224 00000800:00000001:1.0:1788880751.232791:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880751.232793:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880751.232801:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880751.232804:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880751.232809:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880751.232813:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880751.232822:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880751.232835:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880751.232841:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880751.232844:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880751.232845:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880751.232847:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880751.232852:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880751.232852:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880751.232855:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880751.232858:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c480 00000400:00000001:3.0:1788880751.232861:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880751.232863:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880751.232864:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b34466e7. 00000400:00000200:3.0:1788880751.232867:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000062e8e6f4 00000400:00000010:3.0:1788880751.232868:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000062e8e6f4. 00000100:00000001:3.0:1788880751.232870:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788880751.232872:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd7d [1] + 224 00000100:00000200:3.0:1788880751.232874:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880751.232877:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880751.232880:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880751.232883:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:3.0:1788880751.232886:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880751.232890:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880751.232894:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880751.232899:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880751.232899:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880751.232902:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880751.232906:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880751.232908:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:1.0:1788880751.232911:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880751.232911:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880751.232914:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880751.232917:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880751.232924:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=129 v=5 (5 5 5 5) 00000100:00000001:1.0:1788880751.232928:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:3.0:1788880751.232928:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=129 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880751.232932:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:1.0:1788880751.232933:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880751.232934:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880751.232937:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:1.0:1788880751.232938:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880751.232940:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880751.232941:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880751.232945:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880751.232945:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880751.232951:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880751.232954:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880751.232959:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880751.232962:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c500 00000100:00000001:0.0:1788880751.232964:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880751.232969:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880751.232973:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880751.232977:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000048e2c403. 00000100:00000001:3.0:1788880751.232977:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880751.232982:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:0.0:1788880751.232983:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000100:00000001:3.0:1788880751.232983:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880751.232988:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000040:3.0:1788880751.232990:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880751.232994:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788880751.232998:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd85 [1] + 224 00000100:00100000:3.0:1788880751.233013:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780682240:192.168.204.111@tcp:400: 00000100:00000200:0.0:1788880751.233014:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880751.233017:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880751.233019:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880751.233022:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:1.0:1788880751.233023:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880751.233027:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:3.0:1788880751.233028:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cc13c26b x1875776780682240/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880751.233034:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880751.233041:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880751.233042:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:0.0:1788880751.233045:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880751.233046:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000200:1.0:1788880751.233047:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880751.233049:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000dc9842d3. 02000000:00000001:3.0:1788880751.233054:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880751.233057:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880751.233059:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880751.233064:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880751.233066:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 02000000:00000001:3.0:1788880751.233070:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880751.233072:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880751.233075:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cc13c26b. 00000100:00000001:0.0:1788880751.233077:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880751.233078:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880751.233079:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880751.233081:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880751.233082:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880751.233086:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880751.233088:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880751.233091:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1788880751.233092:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880751.233093:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880751.233095:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880751.233096:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880751.233098:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880751.233099:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880751.233100:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788880751.233105:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880751.233106:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880751.233110:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=119 v=5 (5 5 5 5) 00000800:00000001:1.0:1788880751.233110:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880751.233117:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=119 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880751.233120:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880751.233122:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880751.233124:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880751.233127:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880751.233130:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880751.233132:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880751.233132:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880751.233135:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880751.233136:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880751.233139:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788880751.233139:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880751.233142:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:0.0:1788880751.233144:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880751.233144:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007f8335d2. 00000400:00000200:2.0:1788880751.233148:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000082116f14 00000400:00000010:2.0:1788880751.233151:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000082116f14. 00000100:00000001:2.0:1788880751.233154:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880751.233157:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788880751.233160:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880751.233162:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880751.233165:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880751.233171:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880751.233178:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880751.233181:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880751.233184:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:0.0:1788880751.233186:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880751.233187:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880751.233190:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880751.233192:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880751.233197:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880751.233204:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780682368:192.168.204.111@tcp:400: 00000100:00001000:2.0:1788880751.233204:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=129 v=5 (5 5 5 5) 00000100:00000001:0.0:1788880751.233212:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880751.233212:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=86 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880751.233217:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880751.233218:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:0.0:1788880751.233221:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780682368/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880751.233222:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880751.233227:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880751.233254:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880751.233258:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880751.233260:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880751.233263:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ed94d533. 00010000:00000001:2.0:1788880751.233265:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880751.233268:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880751.233270:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880751.233271:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880751.233275:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880751.233275:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000010:0.0:1788880751.233279:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 00000100:00000040:2.0:1788880751.233279:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:0.0:1788880751.233283:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880751.233284:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000001:0.0:1788880751.233286:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788880751.233287:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880751.233289:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000100:00000001:0.0:1788880751.233293:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:2.0:1788880751.233294:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880751.233295:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880751.233298:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880751.233304:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880751.233307:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880751.233307:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880751.233312:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880751.233313:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880751.233315:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880751.233316:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880751.233321:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880751.233329:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780682496:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880751.233334:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880751.233336:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880751.233337:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000005a8bb1de x1875776780682496/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880767 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880751.233342:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880751.233344:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880751.233346:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a4a3549f. 02000000:00000001:2.0:1788880751.233349:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880751.233351:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880751.233352:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880751.233353:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880751.233354:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f47bc5fc. 02000000:00000001:2.0:1788880751.233356:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880751.233357:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880751.233358:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000005a8bb1de. 00000100:00000001:2.0:1788880751.233361:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880751.233362:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880751.233372:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880751.233379:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880751.233380:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880751.233382:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880751.233389:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880751.294336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880751.294341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880751.294345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880751.294346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880751.294394:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880751.294398:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880751.294400:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880752.318284:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880752.318290:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880752.318295:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880752.318296:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880752.318333:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880752.318336:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880752.318338:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880753.342320:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880753.342326:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880753.342331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880753.342332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880753.342375:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880753.342378:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880753.342388:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880754.366358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880754.366366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880754.366373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880754.366375:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880754.366432:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880754.366436:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880754.366438:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880755.390334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880755.390344:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880755.390346:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880755.390350:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880755.390355:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880755.390375:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880755.390377:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880755.390379:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880755.390384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880755.390386:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880755.390448:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880755.390453:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880755.390456:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880756.350367:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306387407, at 4306387456 00000100:00000040:2.0:1788880756.350367:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306387407, at 4306387456 00000100:00000040:3.0:1788880756.350367:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306387407, at 4306387456 00000100:00000040:0.0:1788880756.350378:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880756.350391:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880756.350391:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000010:3.0:1788880756.350403:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000010:0.0:1788880756.350404:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001d70df98. 00000100:00000010:2.0:1788880756.350404:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000020:00000040:0.0:1788880756.350424:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880756.350430:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880756.350434:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880756.350439:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880756.350448:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b016e4f4. 00000100:00000001:0.0:1788880756.350455:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880756.350475:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880756.350485:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880756.350487:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880756.350491:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001d70df98] to pc [ptlrpcd_00_02+2] req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880756.350519:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880756.350520:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880756.350523:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880756.350526:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000118779b0. 00000020:00000040:0.0:1788880756.350528:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880756.350531:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880756.350532:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880756.350534:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880756.350536:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f0a0057b. 00000100:00000001:0.0:1788880756.350539:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880756.350541:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880756.350544:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880756.350549:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880756.350550:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880756.350551:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880756.350556:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000118779b0] to pc [ptlrpcd_00_03+3] req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880756.350557:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880756.350558:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880756.350568:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880756.350571:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880756.350573:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880756.350575:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880756.350583:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880756.350584:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880756.350592:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880756.350602:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880756.350606:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ff0be673. 00000100:00000040:1.0:1788880756.350609:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880756.350611:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000020:00000040:0.0:1788880756.350612:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880756.350617:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880756.350621:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880756.350627:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880756.350630:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880756.350630:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880756.350634:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:3.0:1788880756.350635:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880756.350638:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880756.350645:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880756.350648:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006df406c6. 02000000:00000001:1.0:1788880756.350648:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880756.350653:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00100000:3.0:1788880756.350653:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000001d70df98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780682624:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880756.350656:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880756.350659:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880756.350673:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00100000:1.0:1788880756.350676:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000118779b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780682752:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880756.350679:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:3.0:1788880756.350683:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880756.350686:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880756.350688:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880756.350691:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880756.350695:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:1.0:1788880756.350696:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880756.350703:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880756.350706:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008f22742c. 02000000:00000010:1.0:1788880756.350709:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cd0104b. 00000100:00000001:0.0:1788880756.350710:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880756.350711:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880756.350716:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880756.350717:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880756.350723:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009f5de6c9. 00000100:00000040:0.0:1788880756.350725:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ff0be673] to pc [ptlrpcd_00_00+0] req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880756.350725:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000066432675. 00000400:00000010:3.0:1788880756.350735:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000999bcd04. 00000400:00000010:1.0:1788880756.350737:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000200:3.0:1788880756.350748:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780682624, portal 25 00000100:00000200:1.0:1788880756.350749:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780682752, portal 10 00000100:00000001:3.0:1788880756.350755:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880756.350756:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880756.350760:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849360512 : -126822860191104 : ffff8ca7c0780a80) 00000100:00000001:3.0:1788880756.350760:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849368576 : -126822860183040 : ffff8ca7c0782a00) 00000100:00000001:0.0:1788880756.350766:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880756.350769:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880756.350773:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880772 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880756.350776:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880756.350785:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880756.350786:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880756.350792:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880756.350799:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880756.350803:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11725) 00000100:00000040:2.0:1788880756.350810:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880756.350816:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880756.350823:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880756.350825:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880756.350828:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306387407, at 4306387456 00000400:00000010:3.0:1788880756.350831:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cf82a3fa. 00000100:00000001:2.0:1788880756.350835:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880756.350844:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880756.350846:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780682624, offset 0 00000100:00000010:0.0:1788880756.350847:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 02000000:00000001:2.0:1788880756.350852:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:1.0:1788880756.350855:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880756.350859:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788880756.350862:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 02000000:00000001:2.0:1788880756.350864:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880756.350867:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:2.0:1788880756.350872:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780682880:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880756.350875:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780682752, offset 0 00000100:00000001:2.0:1788880756.350877:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880756.350878:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880756.350882:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880756.350882:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880756.350885:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880756.350889:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880756.350891:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880756.350891:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880756.350897:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000826ae8f4. 02000000:00000001:2.0:1788880756.350910:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880756.350915:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000bcbb0958. 00000400:00000010:2.0:1788880756.350920:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005071f3f0. 00000400:00000200:3.0:1788880756.350920:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:2.0:1788880756.350925:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780682880, portal 10 00000400:00000200:3.0:1788880756.350928:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880756.350929:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880756.350932:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000400:00000200:3.0:1788880756.350936:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880756.350941:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:2.0:1788880756.350948:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880756.350950:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [407:407:256:4294967295] 192.168.204.111@tcp LPNI seq info [407:407:8:4294967295] 00000100:00000001:2.0:1788880756.350968:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788880756.350969:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:2.0:1788880756.350975:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880756.350979:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a3b2cf4a. 00000100:00000200:2.0:1788880756.350982:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780682880, offset 0 00000800:00000200:3.0:1788880756.350983:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880756.350984:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880756.350991:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880756.350993:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000400:00000200:1.0:1788880756.350994:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880756.351000:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880756.351004:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788880756.351008:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880756.351010:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880756.351015:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [408:408:255:4294967295] 192.168.204.111@tcp LPNI seq info [408:408:7:4294967295] 00000800:00000200:3.0:1788880756.351021:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880756.351028:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880756.351029:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880756.351037:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880756.351041:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880756.351043:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000400:00000200:2.0:1788880756.351045:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880756.351049:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880756.351051:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880756.351055:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880756.351057:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306392527 00000400:00000200:2.0:1788880756.351058:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:3.0:1788880756.351066:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002c2d3615. 00000800:00000200:1.0:1788880756.351068:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880756.351068:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880756.351074:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880756.351074:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:2.0:1788880756.351075:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [409:409:254:4294967295] 192.168.204.111@tcp LPNI seq info [409:409:6:4294967295] 00000100:00000001:3.0:1788880756.351083:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306392527 00000100:00000001:1.0:1788880756.351085:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880756.351091:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000007dfb4281. 00000400:00000200:2.0:1788880756.351092:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:1.0:1788880756.351093:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 213293 usecs (214 jiffies) 00000100:00000001:3.0:1788880756.351097:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880756.351101:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306387670 00000100:00000001:3.0:1788880756.351103:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306392527 00000800:00000200:2.0:1788880756.351105:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880756.351107:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306392527 00000100:00000010:3.0:1788880756.351110:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d5dcfb55. 00000800:00000010:2.0:1788880756.351111:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000100:00000001:1.0:1788880756.351112:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306387670 00000100:00000001:3.0:1788880756.351115:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880756.351117:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306392527 00000800:00000200:2.0:1788880756.351120:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880756.351122:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880756.351122:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306392527 00000100:00000001:1.0:1788880756.351123:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306387670 00000800:00000010:0.0:1788880756.351133:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000100:00000001:1.0:1788880756.351133:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306392527 00000800:00000200:2.0:1788880756.351134:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:3.0:1788880756.351134:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000016ab1a15. 00000100:00000001:1.0:1788880756.351142:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306387670 00000100:00000001:3.0:1788880756.351146:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:2.0:1788880756.351148:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880756.351150:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880756.351154:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306392527 00000100:00000001:2.0:1788880756.351154:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880756.351154:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880756.351162:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 14783 usecs (15 jiffies) 00000100:00000001:1.0:1788880756.351163:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880756.351164:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880756.351167:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880756.351169:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306387471 00000100:00000001:3.0:1788880756.351172:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880756.351174:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880756.351180:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306392527 00000100:00000001:3.0:1788880756.351181:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880756.351187:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306387471 00000400:00000200:0.0:1788880756.351188:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cf82a3fa 00000100:00000001:1.0:1788880756.351188:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880756.351194:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306392527 00000100:00000001:3.0:1788880756.351194:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:0.0:1788880756.351198:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cf82a3fa. 00000100:00000001:1.0:1788880756.351198:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880756.351199:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306387471 00000100:00000001:3.0:1788880756.351199:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880756.351210:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306392527 00000100:00000001:1.0:1788880756.351214:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880756.351215:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306387471 00000100:00000001:3.0:1788880756.351216:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880756.351218:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880756.351218:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880756.351220:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880756.351228:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880756.351229:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306392527 00000100:00000001:3.0:1788880756.351230:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880756.351254:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880756.351256:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880756.351257:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880756.351259:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880756.351263:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880756.351265:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880756.351268:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880756.351270:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880756.351274:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880756.351274:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880756.351274:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880756.351276:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880772 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880756.351277:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880756.351284:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880756.351292:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880756.351306:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880756.351311:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880772 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880756.351316:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880756.351318:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880756.351336:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880756.351338:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:0.0:1788880756.351340:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880756.351342:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880756.351344:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:0.0:1788880756.351345:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:0.0:1788880756.351347:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880756.351349:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880756.351353:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880756.351354:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880756.351358:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880756.351359:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880756.351407:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880756.351410:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000400:00000200:0.0:1788880756.351412:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880756.351416:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880756.351418:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a3b2cf4a 00000400:00000010:0.0:1788880756.351419:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a3b2cf4a. 00000100:00000001:0.0:1788880756.351422:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880756.351424:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880756.351431:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880756.351433:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880756.351437:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880756.351438:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880756.353434:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880756.353448:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880756.353452:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880756.353455:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880756.353464:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880756.353485:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0c580 00000400:00000200:2.0:1788880756.353497:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xd8d [1] + 224 00000800:00000001:2.0:1788880756.353504:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880756.353507:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880756.353511:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880756.353514:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880756.353520:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880756.353524:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880756.353533:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880756.353546:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880756.353554:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880756.353557:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880756.353558:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880756.353560:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880756.353563:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880756.353576:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c600 00000400:00000200:2.0:1788880756.353580:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd91 [1] + 224 00000800:00000001:2.0:1788880756.353584:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880756.353590:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880756.353592:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880756.353596:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880756.353599:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880756.353603:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880756.353611:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880756.353626:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880756.353632:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880756.353635:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880756.353637:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880756.353640:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880756.353646:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880756.353647:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880756.353651:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880756.353653:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c680 00000400:00000001:3.0:1788880756.353658:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880756.353659:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000001:3.0:1788880756.353661:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880756.353662:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:3.0:1788880756.353665:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009f5de6c9. 00000400:00000200:3.0:1788880756.353670:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000999bcd04 00000400:00000010:3.0:1788880756.353672:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000999bcd04. 00000100:00000001:3.0:1788880756.353676:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880756.353680:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xd9d [1] + 224 00000400:00000001:1.0:1788880756.353685:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880756.353685:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880756.353689:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880756.353690:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000066432675. 00000800:00000001:2.0:1788880756.353691:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880756.353695:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880756.353696:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:1.0:1788880756.353700:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000400:00000200:2.0:1788880756.353701:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880756.353705:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880756.353707:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880756.353712:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880756.353714:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880756.353721:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880756.353744:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880756.353747:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880756.353749:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880756.353773:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880756.353776:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880756.353779:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880756.353782:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880756.353782:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880756.353787:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880756.353787:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880756.353791:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:3.0:1788880756.353792:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880756.353795:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880756.353795:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880756.353798:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880756.353799:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880756.353801:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880756.353804:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788880756.353804:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880756.353810:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=124 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880756.353811:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=134 v=5 (5 5 5 5) 00000800:00000001:2.0:1788880756.353814:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880756.353822:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=124 v=5 (1 1 1 1) 00000100:00001000:3.0:1788880756.353822:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=134 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880756.353828:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880756.353831:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880756.353831:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880756.353832:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880756.353835:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880756.353835:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880756.353840:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880756.353841:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000001:0.0:1788880756.353843:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880756.353845:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00010000:00000001:3.0:1788880756.353845:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880756.353847:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880756.353850:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000bcbb0958. 00000100:00000001:3.0:1788880756.353852:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880756.353854:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880756.353860:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880756.353864:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000200:0.0:1788880756.353865:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005071f3f0 00000100:00000040:3.0:1788880756.353866:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880756.353875:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880756.353879:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005071f3f0. 00000100:00000040:1.0:1788880756.353890:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880756.353891:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880756.353895:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880756.353898:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:0.0:1788880756.353914:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880756.353916:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880756.353927:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880756.353932:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880756.353941:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:0.0:1788880756.353946:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880756.353946:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880756.353963:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880756.353999:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:3.0:1788880756.354000:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000001d70df98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780682624:192.168.204.111@tcp:400: 00000100:00100000:1.0:1788880756.354007:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000118779b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780682752:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880756.354008:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880756.354010:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880756.354018:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880756.354020:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:0.0:1788880756.354026:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880756.354029:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880756.354030:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880756.354030:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000001d70df98 x1875776780682624/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880756.354032:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880756.354043:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880756.354044:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880756.354054:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000118779b0 x1875776780682752/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880756.354055:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00001000:0.0:1788880756.354056:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=134 v=5 (5 5 5 5) 02000000:00000001:3.0:1788880756.354059:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880756.354066:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008f22742c. 00000100:00001000:0.0:1788880756.354067:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=91 v=5 (1 1 0 0) 02000000:00000001:3.0:1788880756.354075:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880756.354079:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000020:00000001:3.0:1788880756.354080:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880756.354082:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880756.354084:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00010000:00000001:0.0:1788880756.354093:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:3.0:1788880756.354093:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880756.354094:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000010:3.0:1788880756.354096:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b016e4f4. 02000000:00000001:1.0:1788880756.354104:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:3.0:1788880756.354107:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00010000:00080000:0.0:1788880756.354110:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880756.354112:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cd0104b. 02000000:00000001:3.0:1788880756.354112:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880756.354116:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000001d70df98. 00000100:00000001:3.0:1788880756.354127:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00010000:00000001:0.0:1788880756.354128:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880756.354128:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880756.354131:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880756.354137:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880756.354137:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880756.354138:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:0.0:1788880756.354140:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000020:00000040:1.0:1788880756.354142:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880756.354143:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880756.354144:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880756.354153:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880756.354154:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880756.354160:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880756.354161:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f0a0057b. 00000100:00000001:3.0:1788880756.354162:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880756.354174:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880756.354176:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880756.354181:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880756.354184:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880756.354191:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000118779b0. 00000100:00000001:0.0:1788880756.354192:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880756.354198:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880756.354199:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880756.354207:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880756.354208:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880756.354219:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880756.354221:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880756.354231:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880756.354266:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880756.354270:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880756.354271:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880756.354283:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780682880:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880756.354288:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880756.354290:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880756.354293:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ff0be673 x1875776780682880/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880772 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880756.354298:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880756.354299:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880756.354301:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000826ae8f4. 02000000:00000001:0.0:1788880756.354303:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880756.354304:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880756.354306:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880756.354307:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880756.354309:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006df406c6. 02000000:00000001:0.0:1788880756.354311:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880756.354312:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880756.354313:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ff0be673. 00000100:00000001:0.0:1788880756.354315:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880756.354316:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880756.354317:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880756.354322:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880756.354323:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880756.354325:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880756.354326:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880756.414370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880756.414379:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880756.414419:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880756.414421:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880756.414493:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880756.414499:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880756.414502:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880757.438349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880757.438357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880757.438362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880757.438365:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880757.438367:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880757.438370:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880757.438373:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880757.438375:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880757.438376:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880757.438378:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880757.438449:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880757.438453:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880757.438456:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880758.462309:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880758.462314:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880758.462319:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880758.462320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880758.462371:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880758.462374:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880758.462376:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880759.486303:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880759.486315:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880759.486321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880759.486322:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880759.486373:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880759.486376:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880759.486379:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880760.510355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880760.510361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880760.510367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880760.510368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880760.510413:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880760.510417:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880760.510418:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880761.470316:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306392527, at 4306392576 00000100:00000040:0.0:1788880761.470326:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306392527, at 4306392576 00000100:00000010:1.0:1788880761.470333:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000007dfb4281. 00000100:00000040:3.0:1788880761.470334:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306392527, at 4306392576 00000100:00000010:0.0:1788880761.470335:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002c2d3615. 00000100:00000010:3.0:1788880761.470344:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000016ab1a15. 00000100:00000040:2.0:1788880761.470422:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880761.470428:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880761.470435:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000005a8bb1de. 00000020:00000040:2.0:1788880761.470439:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880761.470443:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880761.470446:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880761.470450:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880761.470456:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000001:2.0:1788880761.470463:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880761.470472:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880761.470483:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880761.470486:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880761.470492:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a8bb1de] to pc [ptlrpcd_00_01+1] req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880761.470541:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880761.470545:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880761.470547:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880761.470552:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880761.470560:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880761.470561:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880761.470576:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880761.470577:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880761.470579:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880761.470581:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780683008:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880761.470584:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880761.470588:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880761.470590:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880761.470591:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880761.470595:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000115c4ce2. 02000000:00000001:0.0:1788880761.470596:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880761.470601:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000048e2c403. 00000400:00000010:0.0:1788880761.470605:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 00000100:00000200:0.0:1788880761.470610:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780683008, portal 25 00000100:00000001:0.0:1788880761.470611:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880761.470612:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938427392 : -126823771124224 : ffff8ca78a2c4e00) 00000100:00000040:0.0:1788880761.470615:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880777 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880761.470620:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880761.470622:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880761.470623:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000115221d8. 00000100:00000200:0.0:1788880761.470625:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780683008, offset 0 00000400:00000200:0.0:1788880761.470629:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880761.470639:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880761.470643:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880761.470646:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880761.470648:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880761.470650:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880761.470653:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [410:410:256:4294967295] 192.168.204.111@tcp LPNI seq info [410:410:8:4294967295] 00000400:00000200:0.0:1788880761.470660:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880761.470664:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880761.470667:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000800:00000200:0.0:1788880761.470672:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880761.470683:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880761.470686:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880761.470697:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880761.470700:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306397647 00000100:00000010:0.0:1788880761.470702:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001d590ced. 00000100:00000001:0.0:1788880761.470706:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880761.470708:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306397647 00000100:00000010:0.0:1788880761.470710:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005b8bebe2. 00000100:00000001:0.0:1788880761.470712:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880761.470716:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306397647 00000100:00000001:0.0:1788880761.470721:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306392527 00000100:00000001:0.0:1788880761.470724:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306397647 00000100:00000010:0.0:1788880761.470726:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009ac8d70b. 00000100:00000001:0.0:1788880761.470729:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880761.470731:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880761.470734:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880761.470737:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880761.470739:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880761.470771:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880761.470773:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880761.470784:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880761.470788:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000629edf55. 00000020:00000040:2.0:1788880761.470790:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880761.470792:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880761.470793:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880761.470796:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880761.470800:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f47bc5fc. 00000800:00000200:1.0:1788880761.470801:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880761.470814:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880761.470817:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000100:00000001:3.0:1788880761.470821:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880761.470822:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:1.0:1788880761.470824:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880761.470825:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880761.470834:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880761.470841:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000400:00000200:1.0:1788880761.470842:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000115221d8 00000100:00000001:2.0:1788880761.470843:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880761.470847:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000115221d8. 00000100:00000040:2.0:1788880761.470850:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000629edf55] to pc [ptlrpcd_00_02+2] req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880761.470853:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880761.470860:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880777 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880761.470870:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880761.470871:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880761.470874:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880761.470877:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880761.470880:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002084a2b4. 00000100:00000040:1.0:1788880761.470881:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880777 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880761.470883:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880761.470884:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880761.470887:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880761.470888:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880761.470891:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880761.470894:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880761.470895:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880761.470897:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880761.470900:0:201279:0:(events.c:68:request_out_callback()) Process leaving 02000000:00000010:2.0:1788880761.470900:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 00000100:00000040:3.0:1788880761.470904:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880761.470905:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880761.470911:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880761.470918:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880761.470918:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880761.470920:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002084a2b4] to pc [ptlrpcd_00_03+3] req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880761.470927:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880761.470931:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880761.470943:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880761.470944:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880761.470947:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880761.470947:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880761.470950:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880761.470952:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11730) 00000100:00000001:1.0:1788880761.470954:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880761.470960:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880761.470967:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880761.470969:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880761.470972:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880761.470975:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880761.470979:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880761.470983:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780683264:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880761.470983:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880761.470986:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880761.470988:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880761.470989:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880761.470992:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880761.470993:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880761.470998:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000010:1.0:1788880761.470998:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000cf77ae66. 02000000:00000001:3.0:1788880761.470998:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880761.471001:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000001:1.0:1788880761.471003:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880761.471003:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880761.471004:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880761.471007:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:1.0:1788880761.471010:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000066432675. 02000000:00000001:3.0:1788880761.471010:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880761.471011:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880761.471016:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880761.471017:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780683136:192.168.204.111@tcp:400: 00000400:00000010:1.0:1788880761.471018:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000200:1.0:1788880761.471023:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780683264, portal 10 00000100:00000001:3.0:1788880761.471024:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880761.471027:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880761.471029:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938429184 : -126823771122432 : ffff8ca78a2c5500) 02000000:00000001:3.0:1788880761.471030:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880761.471033:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880761.471036:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880761.471038:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880761.471041:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e0fe5497. 00000100:00000001:1.0:1788880761.471046:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880761.471048:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880761.471050:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000066e8f89d. 00000100:00000200:1.0:1788880761.471054:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780683264, offset 0 02000000:00000001:3.0:1788880761.471055:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880761.471061:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009f5de6c9. 00000400:00000010:3.0:1788880761.471067:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000999bcd04. 00000400:00000200:1.0:1788880761.471071:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880761.471077:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:3.0:1788880761.471077:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780683136, portal 10 00000100:00000001:3.0:1788880761.471083:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880761.471084:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880761.471086:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938432768 : -126823771118848 : ffff8ca78a2c6300) 00000400:00000200:1.0:1788880761.471088:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880761.471092:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880761.471097:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:3.0:1788880761.471097:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880761.471103:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [411:411:256:4294967295] 192.168.204.111@tcp LPNI seq info [411:411:8:4294967295] 00000100:00000001:3.0:1788880761.471116:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880761.471118:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788880761.471124:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880761.471129:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880761.471130:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 00000800:00000010:1.0:1788880761.471133:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000100:00000200:3.0:1788880761.471136:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780683136, offset 0 00000800:00000200:1.0:1788880761.471141:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880761.471147:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880761.471148:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880761.471153:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880761.471160:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880761.471164:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880761.471164:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788880761.471169:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 14693 usecs (15 jiffies) 00000400:00000200:3.0:1788880761.471171:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880761.471174:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306392591 00000400:00000200:3.0:1788880761.471177:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880761.471178:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306397647 00000100:00000001:1.0:1788880761.471183:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306392591 00000400:00000200:3.0:1788880761.471183:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880761.471187:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306397647 00000100:00000001:1.0:1788880761.471191:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306392591 00000400:00000200:3.0:1788880761.471192:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [412:412:255:4294967295] 192.168.204.111@tcp LPNI seq info [412:412:7:4294967295] 00000100:00000001:1.0:1788880761.471194:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306397647 00000100:00000001:1.0:1788880761.471198:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306392591 00000100:00000001:1.0:1788880761.471201:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306397647 00000100:00000001:1.0:1788880761.471203:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880761.471207:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880761.471210:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880761.471212:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880761.471213:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880761.471219:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:3.0:1788880761.471221:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880761.471226:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880761.471226:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:3.0:1788880761.471226:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000800:00000010:0.0:1788880761.471232:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000100:00000001:1.0:1788880761.471248:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880761.471250:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880761.471251:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880761.471253:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880761.471270:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880761.471274:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880761.471277:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000066e8f89d 00000400:00000010:0.0:1788880761.471278:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000066e8f89d. 00000100:00000001:0.0:1788880761.471281:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880761.471284:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880761.471296:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880761.471298:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880761.471304:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880761.471305:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880761.471318:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880761.471325:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880761.471327:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880761.471334:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880761.471336:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 211577 usecs (212 jiffies) 00000100:00000001:3.0:1788880761.471339:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306392789 00000100:00000001:3.0:1788880761.471341:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306397647 00000100:00000001:3.0:1788880761.471342:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306392789 00000100:00000001:3.0:1788880761.471345:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306397647 00000100:00000001:3.0:1788880761.471346:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306392789 00000100:00000001:3.0:1788880761.471349:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306397647 00000100:00000001:3.0:1788880761.471351:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306392789 00000100:00000001:3.0:1788880761.471353:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306397647 00000100:00000001:3.0:1788880761.471354:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880761.471356:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880761.471359:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880761.471361:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880761.471365:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880761.471366:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880761.471368:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880761.471369:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880761.471371:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880761.471373:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880761.471399:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880761.471403:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:2.0:1788880761.471406:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880761.471409:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880761.471412:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 00000400:00000010:2.0:1788880761.471414:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000100:00000001:2.0:1788880761.471416:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880761.471419:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880761.471425:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880761.471427:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880761.471431:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880761.471432:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880761.473572:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880761.473584:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880761.473588:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880761.473591:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880761.473599:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880761.473610:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0c700 00000400:00000200:1.0:1788880761.473618:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xda5 [1] + 224 00000800:00000001:1.0:1788880761.473623:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880761.473626:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880761.473628:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880761.473632:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880761.473637:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880761.473642:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880761.473652:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880761.473671:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880761.473678:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880761.473681:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880761.473683:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880761.473684:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880761.473688:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880761.473694:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c780 00000400:00000200:1.0:1788880761.473700:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xdb5 [1] + 224 00000800:00000001:1.0:1788880761.473703:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880761.473706:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880761.473715:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880761.473718:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880761.473720:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880761.473724:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:0.0:1788880761.473728:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880761.473730:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880761.473732:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000048e2c403. 00000400:00000200:0.0:1788880761.473737:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000400:00000010:0.0:1788880761.473739:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000001:0.0:1788880761.473743:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880761.473744:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880761.473748:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880761.473750:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880761.473760:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880761.473776:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880761.473783:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880761.473785:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880761.473788:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880761.473788:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788880761.473791:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880761.473794:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:1.0:1788880761.473795:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880761.473797:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880761.473801:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880761.473801:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880761.473817:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=139 v=5 (5 5 5 5) 00000800:00000001:1.0:1788880761.473817:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880761.473821:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880761.473822:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=139 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880761.473827:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880761.473830:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880761.473832:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00010000:00000001:0.0:1788880761.473834:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880761.473837:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:0.0:1788880761.473838:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880761.473841:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880761.473843:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880761.473843:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c800 00000400:00000001:3.0:1788880761.473846:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880761.473849:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:0.0:1788880761.473851:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880761.473852:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009f5de6c9. 00000400:00000200:3.0:1788880761.473856:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000999bcd04 00000400:00000010:3.0:1788880761.473859:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000999bcd04. 00000100:00000001:3.0:1788880761.473863:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788880761.473868:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xdad [1] + 224 00000100:00000200:3.0:1788880761.473870:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880761.473876:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880761.473882:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:1.0:1788880761.473884:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880761.473885:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:1.0:1788880761.473889:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880761.473890:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880761.473900:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880761.473900:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880761.473908:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880761.473910:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880761.473916:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880761.473925:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780683008:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880761.473935:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880761.473937:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880761.473939:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880761.473941:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880761.473942:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880761.473943:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880761.473946:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880761.473949:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:0.0:1788880761.473950:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000005a8bb1de x1875776780683008/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880761.473952:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880761.473956:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880761.473963:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=129 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880761.473969:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=129 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880761.473972:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880761.473973:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880761.473975:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880761.473977:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880761.473979:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880761.473981:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880761.473982:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:0.0:1788880761.473986:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880761.473986:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880761.473988:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880761.473991:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000115c4ce2. 00000100:00000040:3.0:1788880761.473993:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880761.473997:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880761.474002:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880761.474003:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000020:00000040:0.0:1788880761.474005:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880761.474007:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000020:00000001:0.0:1788880761.474010:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880761.474010:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788880761.474015:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 00000100:00000040:3.0:1788880761.474016:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880761.474021:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880761.474023:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880761.474025:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788880761.474027:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00100000:3.0:1788880761.474028:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780683136:192.168.204.111@tcp:400: 00000100:00000010:0.0:1788880761.474029:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000005a8bb1de. 00000100:00000001:3.0:1788880761.474036:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880761.474037:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880761.474039:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880761.474040:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880761.474040:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880761.474045:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880761.474047:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000629edf55 x1875776780683136/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880761.474051:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000001:1.0:1788880761.474053:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:2.0:1788880761.474054:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880761.474055:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788880761.474058:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880761.474059:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:1.0:1788880761.474059:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880761.474062:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000066432675. 00000100:00000001:0.0:1788880761.474064:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880761.474069:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000100:00000001:3.0:1788880761.474073:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000010:2.0:1788880761.474074:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 02000000:00000001:3.0:1788880761.474076:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880761.474079:0:201288:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000010:3.0:1788880761.474079:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e0fe5497. 02000000:00000001:3.0:1788880761.474084:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:2.0:1788880761.474086:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880761.474087:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880761.474089:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880761.474094:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880761.474096:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f47bc5fc. 02000000:00000001:3.0:1788880761.474101:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880761.474103:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880761.474105:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000629edf55. 00000100:00000200:2.0:1788880761.474106:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880761.474110:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880761.474113:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880761.474117:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880761.474121:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880761.474122:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880761.474124:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880761.474127:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880761.474135:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880761.474138:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880761.474140:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880761.474142:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880761.474143:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880761.474145:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880761.474147:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880761.474151:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=139 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880761.474154:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=96 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880761.474156:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880761.474157:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880761.474159:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880761.474162:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880761.474168:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880761.474171:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880761.474172:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880761.474173:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880761.474174:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880761.474176:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880761.474182:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880761.474183:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880761.474184:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880761.474187:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880761.474193:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780683264:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880761.474197:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880761.474199:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880761.474201:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002084a2b4 x1875776780683264/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880777 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880761.474220:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880761.474221:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880761.474229:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000cf77ae66. 02000000:00000001:2.0:1788880761.474256:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880761.474258:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880761.474259:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880761.474261:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880761.474262:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 02000000:00000001:2.0:1788880761.474265:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880761.474266:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880761.474267:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002084a2b4. 00000100:00000001:2.0:1788880761.474269:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880761.474270:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880761.474272:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880761.474275:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880761.474276:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880761.474278:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880761.474279:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880761.534350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880761.534356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880761.534362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880761.534363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880761.534432:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880761.534437:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880761.534440:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880762.558308:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880762.558315:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880762.558321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880762.558328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880762.558360:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880762.558364:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880762.558367:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880763.582333:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880763.582339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880763.582343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880763.582344:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880763.582402:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880763.582406:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880763.582409:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880764.606352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880764.606358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880764.606363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880764.606364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880764.606414:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880764.606418:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880764.606421:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880765.630809:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880765.630818:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880765.630820:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880765.630823:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880765.630828:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880765.630830:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880765.630832:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880765.630834:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880765.630838:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880765.630839:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880765.630891:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880765.630893:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880765.630896:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880766.590324:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306397647, at 4306397696 00000100:00000040:1.0:1788880766.590324:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306397647, at 4306397696 00000100:00000040:2.0:1788880766.590324:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306397647, at 4306397696 00000100:00000010:2.0:1788880766.590339:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d5dcfb55. 00000100:00000010:0.0:1788880766.590341:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001d590ced. 00000100:00000010:1.0:1788880766.590345:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005b8bebe2. 00000100:00000040:2.0:1788880766.590353:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880766.590359:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880766.590376:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002084a2b4. 00000020:00000040:2.0:1788880766.590379:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880766.590384:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880766.590387:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:3.0:1788880766.590387:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306397647, at 4306397696 02000000:00000001:2.0:1788880766.590392:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880766.590396:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009ac8d70b. 02000000:00000010:2.0:1788880766.590402:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 00000100:00000001:2.0:1788880766.590413:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880766.590423:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880766.590433:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880766.590435:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880766.590440:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002084a2b4] to pc [ptlrpcd_00_00+0] req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880766.590465:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880766.590467:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880766.590471:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880766.590473:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000884e74b8. 00000020:00000040:2.0:1788880766.590475:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880766.590477:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880766.590479:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880766.590481:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880766.590485:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:2.0:1788880766.590488:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880766.590491:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880766.590498:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880766.590500:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880766.590503:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000884e74b8] to pc [ptlrpcd_00_01+1] req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880766.590506:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788880766.590511:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:2.0:1788880766.590514:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880766.590517:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880766.590521:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880766.590527:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880766.590528:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000010:2.0:1788880766.590530:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 00000100:00000001:0.0:1788880766.590532:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:2.0:1788880766.590536:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880766.590537:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880766.590541:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880766.590542:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880766.590544:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788880766.590546:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880766.590547:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880766.590552:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880766.590552:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f9812869. 00000100:00000001:2.0:1788880766.590558:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880766.590564:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880766.590566:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880766.590567:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880766.590572:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880766.590575:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788880766.590576:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880766.590577:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880766.590581:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880766.590583:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_02+2] req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880766.590591:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780683520:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880766.590597:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880766.590603:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880766.590605:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880766.590606:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880766.590609:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880766.590612:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880766.590613:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000010:0.0:1788880766.590619:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b4722d49. 00000100:00000040:2.0:1788880766.590619:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11735) 02000000:00000001:0.0:1788880766.590624:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880766.590624:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880766.590627:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880766.590628:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880766.590629:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880766.590630:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000048e2c403. 00000100:00000001:1.0:1788880766.590632:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880766.590632:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880766.590637:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880766.590641:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 00000100:00000040:1.0:1788880766.590645:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880766.590648:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780683520, portal 10 00000100:00000001:0.0:1788880766.590654:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880766.590656:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938423808 : -126823771127808 : ffff8ca78a2c4000) 00000100:00000001:1.0:1788880766.590659:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880766.590662:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880766.590666:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880766.590667:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880766.590670:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880766.590676:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880766.590678:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00100000:1.0:1788880766.590682:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780683648:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880766.590683:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880766.590686:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000100:00000001:1.0:1788880766.590688:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000200:0.0:1788880766.590690:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780683520, offset 0 02000000:00000001:1.0:1788880766.590692:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880766.590695:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880766.590696:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880766.590705:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880766.590718:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000010:1.0:1788880766.590724:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000356bd8da. 00000400:00000200:0.0:1788880766.590725:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880766.590728:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880766.590732:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005fdfe59c. 00000400:00000200:0.0:1788880766.590735:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880766.590739:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:1.0:1788880766.590739:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 00000400:00000200:0.0:1788880766.590744:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:1.0:1788880766.590745:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780683648, portal 10 00000400:00000200:0.0:1788880766.590747:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [413:413:256:4294967295] 192.168.204.111@tcp LPNI seq info [413:413:8:4294967295] 00000100:00000001:1.0:1788880766.590751:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880766.590753:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000400:00000200:0.0:1788880766.590759:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:1.0:1788880766.590763:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880766.590769:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880766.590776:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c93f60a2. 00000100:00000001:1.0:1788880766.590777:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880766.590782:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880766.590786:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e249bdf9. 00000800:00000200:0.0:1788880766.590787:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:1.0:1788880766.590790:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780683648, offset 0 00000400:00000200:1.0:1788880766.590800:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880766.590801:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880766.590807:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880766.590811:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1788880766.590815:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c93f60a2 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880766.590816:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880766.590819:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880766.590824:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880766.590828:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [414:414:255:4294967295] 192.168.204.111@tcp LPNI seq info [414:414:7:4294967295] 00000400:00000200:1.0:1788880766.590835:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880766.590837:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880766.590839:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880766.590844:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 209888 usecs (210 jiffies) 00000800:00000010:1.0:1788880766.590844:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000800:00000200:1.0:1788880766.590850:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880766.590851:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306397906 00000800:00000200:1.0:1788880766.590855:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:0.0:1788880766.590857:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001d590ced. 00000800:00000200:1.0:1788880766.590858:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880766.590863:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000100:00000001:1.0:1788880766.590863:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880766.590865:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 14605 usecs (15 jiffies) 00000100:00000001:1.0:1788880766.590868:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306397711 00000100:00000001:0.0:1788880766.590869:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306397906 00000100:00000001:1.0:1788880766.590871:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306397906 00000100:00000010:0.0:1788880766.590874:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000007d3948ef. 00000100:00000001:0.0:1788880766.590879:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880766.590880:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306397711 00000100:00000001:0.0:1788880766.590885:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306397906 00000100:00000001:1.0:1788880766.590885:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306397906 00000800:00000200:2.0:1788880766.590885:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:0.0:1788880766.590892:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006050354c. 00000100:00000001:1.0:1788880766.590892:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306397711 00000800:00000010:2.0:1788880766.590893:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c93f60a2. 00000100:00000001:0.0:1788880766.590899:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000400:00000200:2.0:1788880766.590900:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880766.590903:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306397906 00000100:00000001:1.0:1788880766.590904:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306397906 00000100:00000010:0.0:1788880766.590908:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000da3f50d1. 00000100:00000001:1.0:1788880766.590910:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306397711 00000400:00000200:2.0:1788880766.590910:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880766.590912:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880766.590918:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880766.590919:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306397906 00000100:00000001:0.0:1788880766.590920:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880766.590920:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000100:00000001:1.0:1788880766.590924:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880766.590925:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00000100:00000001:0.0:1788880766.590926:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880766.590929:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880766.590932:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880766.590932:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880766.590934:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880766.590939:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880766.590941:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880766.590945:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880766.590949:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880766.590954:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880766.590957:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880766.590960:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880766.590965:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880766.590965:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880766.590972:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880766.590972:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880766.590974:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880766.590978:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880766.590981:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880766.590986:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880766.590992:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880766.590994:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780683392:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880766.590995:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880766.591000:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880766.591003:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880766.591005:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880766.591007:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880766.591010:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000082966425. 02000000:00000001:0.0:1788880766.591012:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880766.591016:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000cb4ffedc. 00000400:00000010:0.0:1788880766.591019:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002aa6ba25. 00000100:00000200:0.0:1788880766.591020:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780683392, portal 25 00000100:00000001:0.0:1788880766.591022:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880766.591023:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938429184 : -126823771122432 : ffff8ca78a2c5500) 00000100:00000040:0.0:1788880766.591026:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880782 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880766.591029:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880766.591032:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000010:2.0:1788880766.591032:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000100:00000040:0.0:1788880766.591036:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880766.591038:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:0.0:1788880766.591039:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000100:00000200:0.0:1788880766.591043:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780683392, offset 0 00000400:00000200:2.0:1788880766.591047:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880766.591050:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880766.591053:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e249bdf9 00000400:00000010:2.0:1788880766.591056:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e249bdf9. 00000400:00000200:0.0:1788880766.591057:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880766.591063:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880766.591063:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:0.0:1788880766.591068:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:2.0:1788880766.591071:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880766.591072:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880766.591075:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880766.591080:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [415:415:255:4294967295] 192.168.204.111@tcp LPNI seq info [415:415:7:4294967295] 00000400:00000200:0.0:1788880766.591091:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880766.591092:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880766.591097:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880766.591107:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880766.591109:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880766.591112:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000010:0.0:1788880766.591113:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000004e0f6846. 00000800:00000200:0.0:1788880766.591120:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880766.591125:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880766.591128:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000004e0f6846 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880766.591134:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880766.591138:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306402767 00000100:00000001:0.0:1788880766.591143:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306397906 00000100:00000001:0.0:1788880766.591148:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306402767 00000100:00000001:0.0:1788880766.591150:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306397906 00000100:00000001:0.0:1788880766.591152:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306402767 00000100:00000001:0.0:1788880766.591155:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306397906 00000100:00000001:0.0:1788880766.591157:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306402767 00000100:00000001:0.0:1788880766.591161:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306397906 00000100:00000001:0.0:1788880766.591163:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880766.591165:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880766.591167:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880766.591169:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880766.591172:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880766.591173:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880766.591176:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880766.591177:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880766.591179:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880766.591181:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880766.591181:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880766.591187:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000004e0f6846. 00000400:00000200:2.0:1788880766.591192:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880766.591196:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880766.591199:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000400:00000010:2.0:1788880766.591201:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:2.0:1788880766.591204:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880766.591207:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880782 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880766.591214:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880766.591216:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880782 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880766.591222:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880766.591223:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880766.592606:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880766.592617:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880766.592621:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880766.592622:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880766.592629:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880766.592639:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c900 00000400:00000200:1.0:1788880766.592645:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xdbd [1] + 224 00000800:00000001:1.0:1788880766.592650:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880766.592652:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880766.592653:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880766.592655:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880766.592658:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880766.592661:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880766.592668:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880766.592687:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880766.592693:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880766.592716:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880766.592718:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880766.592721:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880766.592724:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880766.592728:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0c980 00000400:00000200:1.0:1788880766.592731:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xdc5 [1] + 224 00000800:00000001:1.0:1788880766.592734:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880766.592736:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880766.592739:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880766.592739:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880766.592743:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880766.592744:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880766.592748:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:0.0:1788880766.592750:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:1.0:1788880766.592751:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880766.592753:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880766.592756:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000048e2c403. 00000400:00000200:0.0:1788880766.592760:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000400:00000010:0.0:1788880766.592762:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000040:1.0:1788880766.592765:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880766.592766:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880766.592775:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880766.592778:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:0.0:1788880766.592784:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880766.592789:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880766.592791:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880766.592792:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880766.592793:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880766.592796:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880766.592798:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880766.592800:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880766.592800:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880766.592802:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880766.592804:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880766.592808:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880766.592810:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=134 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880766.592815:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=134 v=5 (1 1 1 1) 00000400:00000200:1.0:1788880766.592817:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880766.592818:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880766.592820:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880766.592824:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880766.592827:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880766.592827:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0c880 00000100:00000001:0.0:1788880766.592830:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880766.592830:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880766.592833:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xdcd [1] + 224 00000100:00000040:0.0:1788880766.592834:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880766.592835:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880766.592838:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000800:00000001:1.0:1788880766.592840:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880766.592841:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880766.592842:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880766.592845:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:0.0:1788880766.592846:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880766.592847:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880766.592848:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005fdfe59c. 00000400:00000200:1.0:1788880766.592852:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880766.592855:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000400:00000200:1.0:1788880766.592860:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:3.0:1788880766.592860:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000001:3.0:1788880766.592866:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880766.592867:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880766.592868:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880766.592871:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880766.592872:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880766.592875:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880766.592879:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880766.592881:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880766.592910:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880766.592912:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780683520:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880766.592939:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:3.0:1788880766.592940:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880766.592949:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880766.592952:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880766.592954:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880766.592956:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:1.0:1788880766.592957:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880766.592958:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880766.592961:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880766.592962:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880766.592963:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880766.592968:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880766.592970:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:3.0:1788880766.592970:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=144 v=5 (5 5 5 5) 00000400:00000010:0.0:1788880766.592973:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000cb4ffedc. 00000100:00001000:3.0:1788880766.592975:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=101 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880766.592979:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:0.0:1788880766.592981:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002aa6ba25 00000100:00000001:3.0:1788880766.592982:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880766.592984:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002aa6ba25. 00010000:00000001:3.0:1788880766.592986:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880766.592987:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00010000:00080000:3.0:1788880766.592991:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880766.592993:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880766.593017:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880766.593018:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880766.593024:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788880766.593026:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880766.593027:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880766.593029:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880766.593031:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880766.593032:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880766.593035:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880766.593035:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880766.593037:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880766.593039:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880766.593042:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880766.593042:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880766.593044:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=144 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880766.593050:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=144 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880766.593054:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880766.593057:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880766.593059:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00010000:00000001:0.0:1788880766.593061:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880766.593064:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00010000:00000001:0.0:1788880766.593065:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880766.593067:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880766.593069:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880766.593073:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880766.593077:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880766.593090:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880766.593092:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880766.593094:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880766.593101:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880766.593101:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780683648:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880766.593111:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880766.593121:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00100000:0.0:1788880766.593124:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780683392:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880766.593128:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780683648/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880766.593132:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880766.593137:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880766.593140:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:0.0:1788880766.593142:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000884e74b8 x1875776780683520/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880766.593144:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880766.593148:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000356bd8da. 02000000:00000001:3.0:1788880766.593153:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880766.593156:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880766.593165:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000020:00000040:3.0:1788880766.593166:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880766.593169:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880766.593171:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b4722d49. 00000020:00000001:3.0:1788880766.593173:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:0.0:1788880766.593176:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000010:3.0:1788880766.593177:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f9812869. 00000020:00000001:0.0:1788880766.593179:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880766.593182:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:3.0:1788880766.593183:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000020:00000001:0.0:1788880766.593185:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:3.0:1788880766.593187:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000010:0.0:1788880766.593188:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 00000100:00000010:3.0:1788880766.593190:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 02000000:00000001:0.0:1788880766.593193:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880766.593195:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:0.0:1788880766.593196:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880766.593199:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000884e74b8. 00000100:00000001:3.0:1788880766.593199:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880766.593203:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880766.593205:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880766.593207:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880766.593208:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880766.593210:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880766.593212:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880766.593215:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002084a2b4 x1875776780683392/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880782 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880766.593216:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880766.593219:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880766.593227:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880766.593229:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880766.593231:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000082966425. 02000000:00000001:0.0:1788880766.593257:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880766.593258:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880766.593259:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880766.593261:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880766.593262:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 02000000:00000001:0.0:1788880766.593264:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880766.593265:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880766.593266:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002084a2b4. 00000100:00000001:0.0:1788880766.593273:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880766.593274:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880766.593275:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880766.593279:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880766.593280:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880766.593282:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880766.593284:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880766.654341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880766.654348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880766.654354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880766.654355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880766.654401:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880766.654404:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880766.654407:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880767.678351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880767.678358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880767.678363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880767.678366:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880767.678368:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880767.678371:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880767.678374:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880767.678376:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880767.678378:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880767.678380:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880767.678435:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880767.678439:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880767.678450:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880768.702359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880768.702367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880768.702376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880768.702379:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880768.702417:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880768.702420:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880768.702422:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880769.726352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880769.726359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880769.726366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880769.726367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880769.726420:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880769.726425:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880769.726428:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880770.750339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880770.750346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880770.750357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880770.750359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880770.750397:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880770.750399:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880770.750402:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880771.710323:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306402767, at 4306402816 00000100:00000040:0.0:1788880771.710329:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306402767, at 4306402816 00000100:00000040:3.0:1788880771.710329:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880771.710342:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306402767, at 4306402816 00000100:00000010:2.0:1788880771.710344:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006050354c. 00000100:00000001:3.0:1788880771.710345:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880771.710347:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001d590ced. 00000100:00000010:1.0:1788880771.710354:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000007d3948ef. 00000100:00000010:3.0:1788880771.710360:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000020:00000040:3.0:1788880771.710369:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880771.710376:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880771.710379:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880771.710385:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880771.710395:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:3.0:1788880771.710402:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880771.710413:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880771.710437:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880771.710439:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880771.710444:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_03+3] req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880771.710469:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880771.710471:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880771.710473:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880771.710475:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880771.710478:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880771.710479:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880771.710480:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880771.710482:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880771.710485:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880771.710485:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:3.0:1788880771.710492:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880771.710494:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880771.710498:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880771.710500:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880771.710507:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880771.710512:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880771.710514:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880771.710518:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880771.710522:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_00+0] req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880771.710523:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880771.710564:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880771.710568:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880771.710574:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880771.710577:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880771.710579:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880771.710583:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880771.710583:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880771.710593:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880771.710595:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000100:00100000:2.0:1788880771.710598:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780683776:192.168.204.111@tcp:400: 00000020:00000040:3.0:1788880771.710600:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880771.710603:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880771.710605:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880771.710606:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00100000:1.0:1788880771.710619:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:1.0:1788880771.710625:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880771.710625:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880771.710626:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000010:3.0:1788880771.710630:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:0.0:1788880771.710635:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880771.710637:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880771.710638:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880771.710639:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880771.710640:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880771.710643:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880771.710643:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880771.710647:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880771.710654:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880771.710658:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880771.710663:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:2.0:1788880771.710664:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008fc66eec. 00000100:00000001:3.0:1788880771.710664:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880771.710666:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880771.710668:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880771.710671:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880771.710675:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880771.710680:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_01+1] req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880771.710687:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001b82b6de. 00000400:00000010:2.0:1788880771.710701:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ac9b6c42. 00000100:00000001:1.0:1788880771.710711:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000200:2.0:1788880771.710713:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780683776, portal 25 00000100:00000001:1.0:1788880771.710714:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880771.710714:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880771.710723:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880771.710725:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880771.710725:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:3.0:1788880771.710727:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880771.710730:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 02000000:00000001:1.0:1788880771.710731:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880771.710735:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880771.710735:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11740) 00000100:00000001:0.0:1788880771.710746:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788880771.710749:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780683904:192.168.204.111@tcp:400: 00000100:00100000:0.0:1788880771.710752:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000040:2.0:1788880771.710755:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880787 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880771.710756:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306402767, at 4306402816 00000100:00000001:1.0:1788880771.710757:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880771.710760:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880771.710765:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:3.0:1788880771.710771:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000da3f50d1. 00000100:00000001:0.0:1788880771.710775:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:1.0:1788880771.710776:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880771.710783:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880771.710789:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880771.710790:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000abce1aea. 00000100:00000001:2.0:1788880771.710791:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880771.710795:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880771.710795:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880771.710802:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880771.710803:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880771.710806:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:1.0:1788880771.710806:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000400:00000010:2.0:1788880771.710810:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000023cb0718. 00000400:00000010:1.0:1788880771.710817:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000200:2.0:1788880771.710817:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780683776, offset 0 00000100:00000040:0.0:1788880771.710820:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880771.710826:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780683904, portal 10 00000100:00000001:1.0:1788880771.710834:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788880771.710838:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880771.710840:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000001:0.0:1788880771.710853:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880771.710854:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880771.710858:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880771.710859:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880771.710870:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880771.710879:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880771.710879:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880771.710883:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880771.710890:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880771.710894:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880771.710894:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:1.0:1788880771.710902:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000002fa6ff6. 00000100:00100000:0.0:1788880771.710904:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780684032:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880771.710907:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:1.0:1788880771.710909:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780683904, offset 0 00000100:00000001:0.0:1788880771.710913:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880771.710913:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788880771.710921:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880771.710925:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880771.710929:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [416:416:256:4294967295] 192.168.204.111@tcp LPNI seq info [416:416:8:4294967295] 00000400:00000200:1.0:1788880771.710933:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880771.710936:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880771.710944:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000010b36294. 02000000:00000001:0.0:1788880771.710948:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880771.710955:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000cb4ffedc. 00000400:00000200:2.0:1788880771.710955:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:0.0:1788880771.710962:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002aa6ba25. 00000100:00000200:0.0:1788880771.710966:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780684032, portal 10 00000100:00000001:0.0:1788880771.710968:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880771.710969:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000800:00000200:2.0:1788880771.710974:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880771.710978:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880771.710982:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880771.710982:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c52725cf. 00000400:00000200:1.0:1788880771.710998:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880771.710999:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880771.711008:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880771.711012:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880771.711016:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880771.711022:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:0.0:1788880771.711028:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880771.711029:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:0.0:1788880771.711036:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 00000400:00000200:1.0:1788880771.711037:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [417:417:255:4294967295] 192.168.204.111@tcp LPNI seq info [417:417:7:4294967295] 00000800:00000200:2.0:1788880771.711038:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c52725cf type 1, nob 320 niov 1 nkiov 1 00000100:00000200:0.0:1788880771.711049:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780684032, offset 0 00000400:00000200:0.0:1788880771.711066:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880771.711066:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880771.711071:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880771.711078:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306407887 00000800:00000200:1.0:1788880771.711083:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880771.711090:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:2.0:1788880771.711090:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000076127301. 00000800:00000010:1.0:1788880771.711091:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000400:00000200:0.0:1788880771.711101:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880771.711105:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:1.0:1788880771.711106:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880771.711114:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880771.711119:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306407887 00000800:00000200:1.0:1788880771.711120:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880771.711124:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:1.0:1788880771.711129:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000010:2.0:1788880771.711131:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000017cf871b. 00000400:00000200:0.0:1788880771.711135:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880771.711135:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880771.711138:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:1.0:1788880771.711145:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 208220 usecs (209 jiffies) 00000100:00000001:2.0:1788880771.711147:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306407887 00000100:00000001:1.0:1788880771.711153:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306403025 00000400:00000200:0.0:1788880771.711154:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [418:418:254:4294967295] 192.168.204.111@tcp LPNI seq info [418:418:6:4294967295] 00000100:00000010:2.0:1788880771.711156:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000247083fe. 00000100:00000001:1.0:1788880771.711164:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306407887 00000800:00000200:3.0:1788880771.711169:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880771.711172:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306403025 00000800:00000010:3.0:1788880771.711177:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c52725cf. 00000100:00000001:1.0:1788880771.711178:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306407887 00000100:00000001:2.0:1788880771.711178:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880771.711184:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306403025 00000400:00000200:3.0:1788880771.711186:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880771.711189:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306407887 00000100:00000001:1.0:1788880771.711190:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306407887 00000100:00000001:1.0:1788880771.711196:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306403025 00000400:00000200:0.0:1788880771.711203:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:3.0:1788880771.711203:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000010:1.0:1788880771.711209:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000100:00000001:1.0:1788880771.711219:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788880771.711224:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000023cb0718 00000100:00000001:1.0:1788880771.711225:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788880771.711228:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000023cb0718. 00000100:00000001:1.0:1788880771.711229:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880771.711230:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306403025 00000100:00000001:3.0:1788880771.711251:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880771.711257:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880787 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880771.711263:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880771.711267:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880771.711273:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880771.711276:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880771.711277:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880771.711281:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880771.711285:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880771.711286:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c93f60a2. 00000100:00000001:2.0:1788880771.711286:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880771.711293:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880771.711297:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880771.711301:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880771.711303:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880771.711308:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880771.711311:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:0.0:1788880771.711314:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880771.711316:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:0.0:1788880771.711321:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c93f60a2 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880771.711322:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880771.711328:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880771.711331:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880771.711331:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880771.711334:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880771.711339:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880787 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880771.711340:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 14508 usecs (15 jiffies) 00000100:00000001:0.0:1788880771.711353:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306402832 00000100:00000001:3.0:1788880771.711354:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880771.711359:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880771.711362:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306407887 00000100:00000001:0.0:1788880771.711367:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306402832 00000100:00000001:0.0:1788880771.711370:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306407887 00000100:00000001:0.0:1788880771.711372:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306402832 00000100:00000001:0.0:1788880771.711375:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306407887 00000800:00000200:3.0:1788880771.711375:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880771.711382:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000100:00000001:0.0:1788880771.711383:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306402832 00000400:00000200:3.0:1788880771.711388:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880771.711390:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306407887 00000100:00000001:0.0:1788880771.711393:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880771.711400:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880771.711401:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880771.711407:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000002fa6ff6 00000100:00000001:0.0:1788880771.711409:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880771.711410:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000002fa6ff6. 00000100:00000001:0.0:1788880771.711420:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880771.711420:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880771.711423:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880771.711429:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880771.711434:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880771.711439:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880771.711441:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880771.711468:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880771.711470:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c93f60a2. 00000400:00000200:3.0:1788880771.711473:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880771.711475:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880771.711477:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000400:00000010:3.0:1788880771.711478:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000001:3.0:1788880771.711480:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880771.711482:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880771.711486:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880771.711488:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880771.711491:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880771.711492:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880771.712634:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880771.712647:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880771.712651:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880771.712653:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880771.712660:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880771.712672:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0ca00 00000400:00000200:2.0:1788880771.712680:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xdd5 [1] + 224 00000800:00000001:2.0:1788880771.712692:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880771.712731:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880771.712733:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880771.712736:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880771.712740:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880771.712742:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880771.712749:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880771.712762:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880771.712772:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880771.712774:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880771.712775:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880771.712776:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880771.712780:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880771.712790:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0cb00 00000400:00000200:2.0:1788880771.712795:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xde5 [1] + 224 00000800:00000001:2.0:1788880771.712800:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880771.712802:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880771.712804:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880771.712805:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880771.712811:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:0.0:1788880771.712815:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880771.712818:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:0.0:1788880771.712821:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880771.712823:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001b82b6de. 00000400:00000200:0.0:1788880771.712828:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ac9b6c42 00000400:00000010:0.0:1788880771.712830:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ac9b6c42. 00000100:00000001:2.0:1788880771.712834:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880771.712835:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880771.712852:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880771.712853:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880771.712927:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880771.712927:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880771.712980:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880771.712983:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880771.712986:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880771.712989:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880771.712989:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880771.712990:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880771.712994:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880771.712996:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880771.713002:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=149 v=5 (5 5 5 5) 00000800:00000001:2.0:1788880771.713003:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880771.713010:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=149 v=5 (1 1 1 1) 00000800:00000001:2.0:1788880771.713013:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880771.713017:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880771.713021:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880771.713023:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880771.713027:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880771.713030:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880771.713036:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00010000:00000001:0.0:1788880771.713037:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880771.713037:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:0.0:1788880771.713046:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880771.713047:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880771.713054:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ca80 00000400:00000001:3.0:1788880771.713054:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880771.713057:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880771.713057:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000cb4ffedc. 00000400:00000200:3.0:1788880771.713063:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002aa6ba25 00000400:00000010:3.0:1788880771.713065:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002aa6ba25. 00000100:00000040:0.0:1788880771.713069:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880771.713071:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880771.713078:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xddd [1] + 224 00000100:00000200:3.0:1788880771.713080:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880771.713098:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880771.713108:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880771.713115:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880771.713132:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880771.713133:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880771.713139:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880771.713139:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880771.713141:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:2.0:1788880771.713147:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880771.713148:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880771.713152:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880771.713156:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880771.713160:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880771.713164:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:0.0:1788880771.713166:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880771.713167:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:2.0:1788880771.713171:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880771.713173:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880771.713182:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880771.713194:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=149 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880771.713207:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=106 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880771.713212:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880771.713219:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:0.0:1788880771.713220:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780683776:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880771.713220:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880771.713223:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880771.713225:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788880771.713230:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00080000:3.0:1788880771.713266:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880771.713274:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880771.713277:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880771.713285:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880771.713287:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788880771.713288:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880771.713290:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:3.0:1788880771.713293:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:1.0:1788880771.713294:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880771.713295:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000010:1.0:1788880771.713302:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000100:00000001:3.0:1788880771.713302:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880771.713307:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880771.713308:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780683776/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880771.713309:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000100:00000040:3.0:1788880771.713315:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880771.713317:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:1.0:1788880771.713325:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880771.713335:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880771.713368:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880771.713369:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880771.713374:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:0.0:1788880771.713377:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880771.713377:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880771.713380:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880771.713382:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008fc66eec. 00000100:00000040:3.0:1788880771.713383:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880771.713396:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880771.713404:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880771.713411:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00100000:3.0:1788880771.713425:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780684032:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880771.713426:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:0.0:1788880771.713427:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880771.713430:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880771.713433:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788880771.713434:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 00000100:00000001:1.0:1788880771.713440:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880771.713443:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:1.0:1788880771.713445:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:0.0:1788880771.713446:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880771.713448:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:3.0:1788880771.713449:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780684032/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880771.713452:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880771.713452:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880771.713456:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880771.713463:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880771.713464:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 02000000:00000001:3.0:1788880771.713464:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880771.713468:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000010b36294. 00000100:00001000:1.0:1788880771.713473:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=139 v=5 (5 5 5 5) 02000000:00000001:3.0:1788880771.713476:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880771.713477:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00001000:1.0:1788880771.713479:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=139 v=5 (1 1 1 1) 00000020:00000001:3.0:1788880771.713483:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880771.713484:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880771.713484:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000020:00000040:3.0:1788880771.713486:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880771.713489:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880771.713495:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:3.0:1788880771.713496:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:1.0:1788880771.713499:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880771.713499:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 00000100:00000001:0.0:1788880771.713504:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880771.713504:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:3.0:1788880771.713506:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:1.0:1788880771.713508:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:3.0:1788880771.713509:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880771.713514:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000010:3.0:1788880771.713514:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:0.0:1788880771.713516:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880771.713517:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880771.713521:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880771.713522:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880771.713525:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880771.713526:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880771.713533:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880771.713534:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880771.713543:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880771.713543:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880771.713546:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880771.713551:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880771.713553:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880771.713555:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880771.713563:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880771.713565:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880771.713571:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880771.713581:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780683904:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880771.713586:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880771.713588:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880771.713591:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780683904/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880787 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880771.713595:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880771.713596:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880771.713597:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000abce1aea. 02000000:00000001:1.0:1788880771.713600:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880771.713601:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880771.713601:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880771.713603:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880771.713604:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:1.0:1788880771.713606:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880771.713607:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880771.713607:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:1.0:1788880771.713623:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880771.713624:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880771.713625:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880771.713628:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880771.713629:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880771.713630:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880771.713632:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880771.774323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880771.774330:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880771.774335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880771.774336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880771.774379:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880771.774384:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880771.774387:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880772.798322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880772.798327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880772.798332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880772.798348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880772.798379:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880772.798383:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880772.798385:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880773.822349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880773.822358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880773.822364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880773.822365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880773.822403:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880773.822408:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880773.822411:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880774.846316:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880774.846321:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880774.846325:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880774.846326:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880774.846351:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880774.846355:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880774.846358:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880775.870385:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880775.870394:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880775.870396:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880775.870399:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880775.870404:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880775.870407:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880775.870408:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880775.870410:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880775.870414:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880775.870414:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880775.870469:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880775.870475:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880775.870480:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880776.830299:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306407887, at 4306407936 00000100:00000040:3.0:1788880776.830305:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306407887, at 4306407936 00000100:00000040:1.0:1788880776.830309:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306407887, at 4306407936 00000100:00000010:2.0:1788880776.830316:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000247083fe. 00000100:00000010:3.0:1788880776.830319:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000010:1.0:1788880776.830321:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000017cf871b. 00000100:00000040:0.0:1788880776.830468:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306407887, at 4306407936 00000100:00000010:0.0:1788880776.830475:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000076127301. 00000100:00000040:0.0:1788880776.830485:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880776.830490:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880776.830496:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ff0be673. 00000020:00000040:0.0:1788880776.830500:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880776.830505:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880776.830508:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880776.830523:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880776.830533:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b016e4f4. 00000100:00000001:0.0:1788880776.830541:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880776.830553:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880776.830578:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880776.830581:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880776.830587:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ff0be673] to pc [ptlrpcd_00_02+2] req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880776.830629:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880776.830634:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880776.830637:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880776.830643:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880776.830643:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880776.830648:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.830661:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880776.830664:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880776.830668:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880776.830670:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880776.830673:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880776.830676:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780684160:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880776.830680:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880776.830685:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880776.830685:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880776.830687:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880776.830690:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880776.830693:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880776.830697:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006b32fd51. 00000020:00000040:0.0:1788880776.830700:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880776.830703:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880776.830705:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880776.830708:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000010:1.0:1788880776.830709:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b31f2b6a. 02000000:00000001:0.0:1788880776.830711:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880776.830714:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880776.830719:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c93f60a2. 00000400:00000010:1.0:1788880776.830723:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000001:0.0:1788880776.830724:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880776.830730:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:1.0:1788880776.830735:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000001:0.0:1788880776.830741:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000200:1.0:1788880776.830743:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780684160, portal 25 00000100:00000001:0.0:1788880776.830744:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.830748:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880776.830751:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006b32fd51] to pc [ptlrpcd_00_03+3] req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880776.830752:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849359616 : -126822860192000 : ffff8ca7c0780700) 00000100:00000040:1.0:1788880776.830761:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880792 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880776.830770:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880776.830773:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880776.830775:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000100:00000200:1.0:1788880776.830777:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780684160, offset 0 00000400:00000200:1.0:1788880776.830783:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880776.830796:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880776.830801:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880776.830806:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880776.830807:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880776.830810:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880776.830811:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880776.830814:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788880776.830816:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:2.0:1788880776.830822:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880776.830824:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [419:419:256:4294967295] 192.168.204.111@tcp LPNI seq info [419:419:8:4294967295] 00000100:00000001:2.0:1788880776.830832:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880776.830835:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880776.830841:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880776.830843:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880776.830845:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880776.830849:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780684288:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880776.830857:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880776.830857:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880776.830860:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880776.830863:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880776.830868:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880776.830868:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000010:1.0:1788880776.830874:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 02000000:00000010:2.0:1788880776.830879:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cd0104b. 00000800:00000200:1.0:1788880776.830884:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:2.0:1788880776.830885:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880776.830892:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000702dddb0. 00000800:00000200:1.0:1788880776.830896:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880776.830901:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000400:00000010:2.0:1788880776.830904:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000200:2.0:1788880776.830908:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780684288, portal 10 00000100:00000001:2.0:1788880776.830911:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880776.830912:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849362304 : -126822860189312 : ffff8ca7c0781180) 00000100:00000040:2.0:1788880776.830918:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880776.830920:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.830927:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306413007 00000100:00000001:2.0:1788880776.830929:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:1.0:1788880776.830933:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000100:00000040:2.0:1788880776.830939:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880776.830940:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000010:2.0:1788880776.830943:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000200:2.0:1788880776.830946:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780684288, offset 0 00000100:00000001:1.0:1788880776.830952:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306413007 00000400:00000200:2.0:1788880776.830955:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880776.830959:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000100:00000001:1.0:1788880776.830963:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:2.0:1788880776.830963:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880776.830970:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880776.830972:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880776.830974:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880776.830976:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880776.830979:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [420:420:255:4294967295] 192.168.204.111@tcp LPNI seq info [420:420:7:4294967295] 00000100:00000001:1.0:1788880776.830983:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306413007 00000400:00000200:2.0:1788880776.830989:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:1.0:1788880776.830991:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000100:00000001:1.0:1788880776.830996:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:2.0:1788880776.830996:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880776.831000:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880776.831002:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306413007 00000800:00000010:2.0:1788880776.831003:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5d1e8ed. 00000800:00000010:3.0:1788880776.831004:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000100:00000010:1.0:1788880776.831008:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000400:00000200:3.0:1788880776.831010:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:2.0:1788880776.831011:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880776.831014:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:2.0:1788880776.831019:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880776.831019:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880776.831020:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880776.831022:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5d1e8ed type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880776.831025:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000100:00000001:1.0:1788880776.831026:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880776.831028:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.831029:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.831032:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880776.831034:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 206582 usecs (207 jiffies) 00000100:00000001:1.0:1788880776.831038:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880776.831038:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306408143 00000100:00000001:1.0:1788880776.831041:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880776.831041:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306413007 00000100:00000001:1.0:1788880776.831045:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880776.831046:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880776.831047:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.831048:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880776.831054:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306408143 00000100:00000001:2.0:1788880776.831057:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306413007 00000100:00000001:2.0:1788880776.831059:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306408143 00000100:00000001:2.0:1788880776.831061:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306413007 00000100:00000001:2.0:1788880776.831063:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306408143 00000100:00000001:2.0:1788880776.831064:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306413007 00000100:00000001:2.0:1788880776.831066:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880776.831068:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880776.831070:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880776.831072:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880776.831098:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880776.831100:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880776.831100:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00000100:00000001:0.0:1788880776.831103:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880776.831104:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000010:0.0:1788880776.831106:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000451fe31a. 00000020:00000040:0.0:1788880776.831109:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000200:3.0:1788880776.831109:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880792 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880776.831111:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880776.831113:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880776.831115:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.831115:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880776.831118:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880776.831118:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000010:0.0:1788880776.831120:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e494c472. 00000100:00000001:1.0:1788880776.831122:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.831126:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880776.831126:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880792 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880776.831129:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880776.831134:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880776.831142:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880776.831143:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880776.831146:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000451fe31a] to pc [ptlrpcd_00_00+0] req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880776.831169:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880776.831172:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880776.831175:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880776.831180:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880776.831181:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11745) 00000100:00000001:1.0:1788880776.831185:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880776.831187:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880776.831192:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880776.831194:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880776.831197:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.831209:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880776.831212:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880776.831214:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880776.831216:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880776.831218:0:201279:0:(events.c:68:request_out_callback()) Process leaving 02000000:00000001:1.0:1788880776.831219:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880776.831224:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880776.831227:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780684416:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880776.831252:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880776.831255:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:1.0:1788880776.831256:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880776.831258:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880776.831261:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880776.831264:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000010:1.0:1788880776.831266:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000006b7f624f. 00000100:00000001:2.0:1788880776.831268:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880776.831270:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880776.831272:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:1.0:1788880776.831276:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005972c901. 00000100:00000001:2.0:1788880776.831276:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880776.831281:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880776.831283:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000030b8c48c. 00000800:00000200:3.0:1788880776.831283:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880776.831287:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5d1e8ed. 00000100:00000200:1.0:1788880776.831289:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780684416, portal 10 00000100:00000001:1.0:1788880776.831293:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788880776.831294:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880776.831295:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849366784 : -126822860184832 : ffff8ca7c0782300) 00000400:00000200:3.0:1788880776.831301:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:1.0:1788880776.831304:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880776.831307:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:3.0:1788880776.831311:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:1.0:1788880776.831315:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880776.831317:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000040:1.0:1788880776.831319:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880776.831321:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000200:1.0:1788880776.831324:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780684416, offset 0 00000100:00000200:3.0:1788880776.831324:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880776.831330:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880776.831337:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880776.831339:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:1.0:1788880776.831340:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880776.831342:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880776.831344:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880776.831345:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880776.831346:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [421:421:255:4294967295] 192.168.204.111@tcp LPNI seq info [421:421:7:4294967295] 00000400:00000200:1.0:1788880776.831351:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880776.831353:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880776.831355:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000749ed09e. 00000800:00000200:1.0:1788880776.831358:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880776.831360:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880776.831362:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000749ed09e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880776.831364:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880776.831369:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 14415 usecs (15 jiffies) 00000100:00000001:1.0:1788880776.831371:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306407952 00000100:00000001:1.0:1788880776.831373:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306413007 00000100:00000001:1.0:1788880776.831374:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306407952 00000100:00000001:1.0:1788880776.831376:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306413007 00000100:00000001:1.0:1788880776.831378:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306407952 00000100:00000001:1.0:1788880776.831380:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306413007 00000100:00000001:1.0:1788880776.831382:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306407952 00000100:00000040:3.0:1788880776.831382:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880776.831385:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306413007 00000100:00000001:1.0:1788880776.831388:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880776.831391:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.831394:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.831397:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880776.831402:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880776.831402:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.831404:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880776.831408:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880776.831408:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880776.831410:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880776.831412:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.831413:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880776.831441:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880776.831444:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000749ed09e. 00000400:00000200:3.0:1788880776.831446:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880776.831448:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880776.831450:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:3.0:1788880776.831451:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:3.0:1788880776.831453:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880776.831454:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880776.831460:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880776.831461:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880776.831465:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880776.831466:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880776.833137:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880776.833146:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880776.833153:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880776.833155:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880776.833160:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880776.833168:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0cb80 00000400:00000200:3.0:1788880776.833174:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xded [1] + 224 00000800:00000001:3.0:1788880776.833180:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880776.833184:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880776.833186:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880776.833188:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880776.833192:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880776.833197:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880776.833207:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880776.833224:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880776.833231:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880776.833251:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880776.833253:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880776.833255:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880776.833259:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880776.833265:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0cc00 00000100:00000001:1.0:1788880776.833266:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880776.833270:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880776.833270:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xdf5 [1] + 224 00000800:00000001:3.0:1788880776.833275:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880776.833278:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788880776.833278:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880776.833282:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:1.0:1788880776.833283:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880776.833287:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:3.0:1788880776.833289:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880776.833292:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000400:00000010:1.0:1788880776.833295:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:1.0:1788880776.833298:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880776.833302:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880776.833303:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880776.833308:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880776.833321:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880776.833326:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880776.833335:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880776.833343:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880776.833349:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880776.833352:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880776.833355:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880776.833358:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880776.833360:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880776.833361:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:1.0:1788880776.833366:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=154 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880776.833371:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=154 v=5 (1 1 1 1) 00000800:00000001:3.0:1788880776.833372:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880776.833374:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880776.833377:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880776.833379:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880776.833381:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:3.0:1788880776.833382:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:1.0:1788880776.833384:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880776.833386:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880776.833389:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880776.833395:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880776.833396:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880776.833398:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880776.833399:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788880776.833402:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880776.833405:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:3.0:1788880776.833405:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0cc80 00000400:00000010:2.0:1788880776.833408:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000702dddb0. 00000400:00000200:2.0:1788880776.833412:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000100:00000001:1.0:1788880776.833413:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880776.833416:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:1.0:1788880776.833417:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880776.833420:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880776.833421:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:3.0:1788880776.833424:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xdfd [1] + 224 00000100:00000040:1.0:1788880776.833425:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880776.833428:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880776.833433:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880776.833438:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1788880776.833443:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ff0be673 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780684160:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880776.833443:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:2.0:1788880776.833451:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880776.833451:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880776.833452:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880776.833455:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880776.833456:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:3.0:1788880776.833459:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880776.833461:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880776.833462:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ff0be673 x1875776780684160/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880776.833463:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880776.833467:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880776.833470:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880776.833477:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880776.833479:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.833481:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:3.0:1788880776.833481:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880776.833482:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880776.833484:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880776.833488:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b31f2b6a. 00000100:00001000:2.0:1788880776.833488:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=144 v=5 (5 5 5 5) 02000000:00000001:1.0:1788880776.833493:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880776.833495:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880776.833496:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880776.833498:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880776.833499:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b016e4f4. 00000100:00001000:2.0:1788880776.833502:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=144 v=5 (1 1 1 1) 02000000:00000001:1.0:1788880776.833503:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880776.833503:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880776.833508:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:3.0:1788880776.833508:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880776.833522:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880776.833523:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00010000:00000001:2.0:1788880776.833524:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:3.0:1788880776.833524:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:1.0:1788880776.833528:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ff0be673. 00010000:00000001:2.0:1788880776.833528:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880776.833535:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880776.833537:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880776.833538:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880776.833539:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880776.833541:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880776.833542:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880776.833546:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880776.833547:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880776.833550:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880776.833551:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880776.833555:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000001:0.0:1788880776.833556:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880776.833559:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880776.833560:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880776.833564:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005972c901. 00000100:00000001:1.0:1788880776.833565:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880776.833566:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880776.833570:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000030b8c48c 00000100:00000001:1.0:1788880776.833570:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880776.833571:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000010:0.0:1788880776.833575:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000030b8c48c. 00000100:00000001:2.0:1788880776.833575:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880776.833581:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:2.0:1788880776.833583:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880776.833588:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880776.833607:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006b32fd51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780684288:192.168.204.111@tcp:400: 00000100:00000200:0.0:1788880776.833610:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880776.833617:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880776.833623:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880776.833623:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880776.833627:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880776.833629:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006b32fd51 x1875776780684288/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880776.833631:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880776.833634:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880776.833636:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880776.833639:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880776.833644:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880776.833644:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880776.833647:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880776.833649:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cd0104b. 02000000:00000001:2.0:1788880776.833654:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:0.0:1788880776.833656:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=154 v=5 (5 5 5 5) 00000020:00000001:2.0:1788880776.833657:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880776.833659:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00001000:0.0:1788880776.833662:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=111 v=5 (1 1 0 0) 00000020:00000001:2.0:1788880776.833664:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880776.833667:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c93f60a2. 00000100:00000001:0.0:1788880776.833668:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880776.833669:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880776.833671:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880776.833676:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880776.833678:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880776.833681:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880776.833685:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006b32fd51. 00010000:00000001:0.0:1788880776.833690:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880776.833691:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880776.833695:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880776.833695:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880776.833698:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880776.833699:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880776.833700:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880776.833703:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880776.833706:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880776.833706:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880776.833710:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880776.833716:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880776.833717:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880776.833719:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880776.833722:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880776.833724:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880776.833729:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880776.833736:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780684416:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880776.833740:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880776.833742:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880776.833744:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000451fe31a x1875776780684416/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880792 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880776.833750:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880776.833751:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880776.833753:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000006b7f624f. 02000000:00000001:0.0:1788880776.833756:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880776.833757:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880776.833757:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880776.833759:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880776.833764:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e494c472. 02000000:00000001:0.0:1788880776.833766:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880776.833766:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880776.833767:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000451fe31a. 00000100:00000001:0.0:1788880776.833769:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880776.833770:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880776.833771:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880776.833775:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880776.833776:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880776.833778:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880776.833779:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880776.894372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880776.894380:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880776.894385:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880776.894387:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880776.894409:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880776.894414:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880776.894418:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880777.918358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880777.918364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880777.918368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880777.918370:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880777.918372:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880777.918374:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880777.918376:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880777.918378:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880777.918379:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880777.918381:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880777.918412:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880777.918415:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880777.918417:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880778.942339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880778.942346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880778.942353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880778.942354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880778.942370:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880778.942374:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880778.942381:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880779.966383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880779.966390:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880779.966396:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880779.966397:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880779.966426:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880779.966432:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880779.966435:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880780.990304:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880780.990310:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880780.990315:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880780.990316:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880780.990344:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880780.990348:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880780.990351:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880781.950411:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306413007, at 4306413056 00000100:00000040:0.0:1788880781.950412:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306413007, at 4306413056 00000100:00000040:1.0:1788880781.950412:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306413007, at 4306413056 00000100:00000040:3.0:1788880781.950412:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306413007, at 4306413056 00000100:00000010:2.0:1788880781.950453:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000010:0.0:1788880781.950460:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000010:3.0:1788880781.950464:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000010:1.0:1788880781.950466:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000040:3.0:1788880781.950493:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880781.950500:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880781.950506:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788880781.950510:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880781.950515:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880781.950518:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880781.950522:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880781.950530:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:3.0:1788880781.950549:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880781.950560:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880781.950571:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880781.950574:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880781.950581:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_01+1] req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880781.950608:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880781.950611:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880781.950616:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880781.950620:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880781.950622:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880781.950625:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880781.950626:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880781.950628:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880781.950629:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880781.950631:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:0.0:1788880781.950635:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880781.950635:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880781.950639:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880781.950641:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880781.950642:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:0.0:1788880781.950647:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880781.950653:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880781.950659:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880781.950662:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880781.950668:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788880781.950670:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_02+2] req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880781.950672:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880781.950679:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880781.950683:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880781.950686:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880781.950690:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780684544:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880781.950695:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880781.950696:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880781.950700:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880781.950701:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880781.950706:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880781.950706:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:0.0:1788880781.950711:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:3.0:1788880781.950712:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880781.950716:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000010:0.0:1788880781.950717:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000482afcf7. 00000100:00000001:1.0:1788880781.950719:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880781.950721:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880781.950722:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880781.950723:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880781.950724:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880781.950727:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880781.950731:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000048e2c403. 02000000:00000001:3.0:1788880781.950733:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880781.950735:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880781.950736:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788880781.950740:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000fbbca8a. 02000000:00000010:3.0:1788880781.950740:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:2.0:1788880781.950743:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880781.950747:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880781.950749:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780684544, portal 25 00000100:00000001:0.0:1788880781.950756:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:3.0:1788880781.950758:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880781.950760:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000001:1.0:1788880781.950762:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880781.950764:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880781.950770:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880781.950774:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880781.950774:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880781.950777:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880781.950779:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880797 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880781.950780:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880781.950785:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_03+3] req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880781.950797:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780684672:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880781.950806:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880781.950809:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880781.950811:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880781.950813:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880781.950816:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000978ba7c2. 00000100:00000001:3.0:1788880781.950816:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880781.950817:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880781.950822:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880781.950822:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880781.950826:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880781.950828:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:1.0:1788880781.950829:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000400:00000010:0.0:1788880781.950832:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e73139ca. 00000100:00000040:3.0:1788880781.950834:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11750) 00000400:00000010:1.0:1788880781.950838:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000200:0.0:1788880781.950839:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780684544, offset 0 00000100:00000200:1.0:1788880781.950843:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780684672, portal 10 00000100:00000001:1.0:1788880781.950847:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880781.950852:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880781.950853:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880781.950854:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000100:00000001:2.0:1788880781.950859:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880781.950863:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880781.950867:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880781.950872:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880781.950884:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880781.950893:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880781.950894:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788880781.950894:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880781.950903:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000065ad0e62. 00000400:00000200:0.0:1788880781.950908:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:1.0:1788880781.950913:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780684672, offset 0 00000400:00000200:0.0:1788880781.950916:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880781.950922:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880781.950928:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880781.950932:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:0.0:1788880781.950935:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [422:422:256:4294967295] 192.168.204.111@tcp LPNI seq info [422:422:8:4294967295] 00000100:00000001:2.0:1788880781.950936:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880781.950948:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788880781.950960:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:0.0:1788880781.950965:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880781.950966:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:2.0:1788880781.950966:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000010:0.0:1788880781.950975:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000fc8f9b7c. 02000000:00000001:2.0:1788880781.950975:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880781.950979:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880781.950991:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00100000:2.0:1788880781.950991:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780684800:192.168.204.111@tcp:400: 00000800:00000200:0.0:1788880781.950992:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880781.951009:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:0.0:1788880781.951014:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880781.951014:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:2.0:1788880781.951016:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880781.951023:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880781.951024:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000fc8f9b7c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880781.951025:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:2.0:1788880781.951036:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880781.951038:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [423:423:255:4294967295] 192.168.204.111@tcp LPNI seq info [423:423:7:4294967295] 02000000:00000010:2.0:1788880781.951047:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000006eecdb68. 00000100:00000001:0.0:1788880781.951051:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880781.951054:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880781.951060:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306418127 00000400:00000200:1.0:1788880781.951061:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:2.0:1788880781.951065:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000702dddb0. 00000100:00000010:0.0:1788880781.951069:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001d590ced. 00000800:00000200:1.0:1788880781.951075:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880781.951077:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000001:0.0:1788880781.951079:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000010:1.0:1788880781.951082:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000100:00000200:2.0:1788880781.951087:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780684800, portal 10 00000100:00000001:0.0:1788880781.951088:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306418127 00000800:00000200:1.0:1788880781.951094:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880781.951097:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000010:0.0:1788880781.951099:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ffa8a4fd. 00000100:00000001:2.0:1788880781.951102:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000001:0.0:1788880781.951104:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:1.0:1788880781.951106:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880781.951114:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306418127 00000800:00000200:1.0:1788880781.951114:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:2.0:1788880781.951120:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880781.951123:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008921ad2e. 00000100:00000001:1.0:1788880781.951124:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880781.951133:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000040:1.0:1788880781.951134:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 204970 usecs (205 jiffies) 00000100:00000001:0.0:1788880781.951143:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306418127 00000100:00000001:1.0:1788880781.951144:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306413261 00000100:00000001:2.0:1788880781.951146:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880781.951150:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306418127 00000100:00000010:0.0:1788880781.951157:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001a3d5793. 00000100:00000040:2.0:1788880781.951157:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880781.951158:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306413261 00000800:00000200:3.0:1788880781.951160:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000010:2.0:1788880781.951163:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cc594c. 00000100:00000001:0.0:1788880781.951165:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880781.951166:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306418127 00000800:00000010:3.0:1788880781.951166:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000fc8f9b7c. 00000100:00000200:2.0:1788880781.951169:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780684800, offset 0 00000100:00000001:1.0:1788880781.951173:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306413261 00000100:00000001:0.0:1788880781.951174:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880781.951174:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880781.951181:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306418127 00000100:00000001:0.0:1788880781.951183:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880781.951183:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880781.951191:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306413261 00000400:00000200:3.0:1788880781.951192:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880781.951194:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880781.951196:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306418127 00000100:00000001:0.0:1788880781.951200:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880781.951201:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880781.951205:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880781.951211:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880781.951215:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880781.951216:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880781.951218:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880781.951218:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880781.951223:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880781.951225:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880781.951225:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880781.951257:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880781.951260:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880781.951263:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880781.951267:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880781.951268:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880781.951274:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880781.951276:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880781.951282:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880781.951287:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [424:424:254:4294967295] 192.168.204.111@tcp LPNI seq info [424:424:6:4294967295] 00000100:00000001:1.0:1788880781.951288:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880781.951292:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880781.951297:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880781.951310:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880781.951320:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880781.951325:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008eccda10. 00000800:00000200:2.0:1788880781.951332:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880781.951338:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e73139ca 00000400:00000010:3.0:1788880781.951349:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e73139ca. 00000800:00000200:2.0:1788880781.951355:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880781.951357:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:2.0:1788880781.951361:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008eccda10 type 1, nob 320 niov 1 nkiov 1 00000100:00000200:3.0:1788880781.951366:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880797 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880781.951369:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880781.951377:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 14322 usecs (15 jiffies) 00000100:00000001:2.0:1788880781.951385:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306413072 00000100:00000001:3.0:1788880781.951388:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880781.951390:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306418127 00000100:00000001:2.0:1788880781.951394:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306413072 00000100:00000040:3.0:1788880781.951396:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880797 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880781.951401:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306418127 00000100:00000001:2.0:1788880781.951408:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306413072 00000100:00000001:2.0:1788880781.951412:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306418127 00000100:00000001:2.0:1788880781.951427:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306413072 00000100:00000001:2.0:1788880781.951433:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306418127 00000100:00000001:3.0:1788880781.951434:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880781.951436:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880781.951440:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880781.951440:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880781.951449:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880781.951453:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880781.951460:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880781.951463:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:3.0:1788880781.951466:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880781.951468:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:3.0:1788880781.951469:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000100:00000001:2.0:1788880781.951472:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880781.951476:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880781.951476:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880781.951484:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880781.951484:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880781.951496:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000065ad0e62 00000400:00000010:3.0:1788880781.951498:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000065ad0e62. 00000100:00000001:3.0:1788880781.951502:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880781.951513:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880781.951520:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880781.951522:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880781.951526:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880781.951528:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880781.951567:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880781.951570:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008eccda10. 00000400:00000200:3.0:1788880781.951573:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880781.951576:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880781.951578:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cc594c 00000400:00000010:3.0:1788880781.951581:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cc594c. 00000100:00000001:3.0:1788880781.951583:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880781.951586:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880781.951591:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880781.951593:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880781.951598:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880781.951599:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880781.952722:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880781.952731:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880781.952734:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880781.952736:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880781.952742:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880781.952753:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0cd00 00000400:00000200:3.0:1788880781.952760:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xe05 [1] + 224 00000800:00000001:3.0:1788880781.952835:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880781.952847:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880781.952849:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880781.952851:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880781.952855:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880781.952859:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880781.952872:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880781.952882:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880781.952887:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880781.952889:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880781.952907:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880781.952909:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880781.952914:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880781.952915:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880781.952916:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000048e2c403. 00000400:00000200:0.0:1788880781.952919:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000fbbca8a 00000400:00000010:0.0:1788880781.952920:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000fbbca8a. 00000100:00000001:0.0:1788880781.952922:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880781.952926:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880781.952935:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880781.952941:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880781.952943:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880781.952946:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880781.952949:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880781.952951:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880781.952953:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880781.952956:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880781.952961:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=9 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880781.952965:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=9 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880781.952967:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880781.952968:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880781.952970:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880781.952978:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880781.952981:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880781.952986:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880781.952993:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880781.952996:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880781.952997:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880781.953001:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880781.953008:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780684544:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880781.953014:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880781.953016:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880781.953019:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780684544/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880781.953026:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880781.953029:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880781.953032:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000482afcf7. 00000800:00000001:3.0:1788880781.953034:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:0.0:1788880781.953035:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880781.953038:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880781.953040:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000800:00000001:3.0:1788880781.953041:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:0.0:1788880781.953043:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000800:00000001:3.0:1788880781.953046:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000010:0.0:1788880781.953047:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 02000000:00000001:0.0:1788880781.953051:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880781.953052:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880781.953053:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:0.0:1788880781.953057:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880781.953059:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880781.953060:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880781.953067:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880781.953073:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880781.953081:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880781.953083:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880781.953085:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880781.953086:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880781.953119:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0cd80 00000400:00000200:3.0:1788880781.953124:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe0d [1] + 224 00000800:00000001:3.0:1788880781.953189:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880781.953194:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880781.953196:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880781.953199:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880781.953202:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880781.953205:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880781.953212:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880781.953228:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880781.953264:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880781.953267:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880781.953270:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880781.953272:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880781.953274:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:1.0:1788880781.953277:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000400:00000010:1.0:1788880781.953278:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:1.0:1788880781.953281:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880781.953285:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880781.953292:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788880781.953298:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880781.953298:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880781.953302:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880781.953306:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880781.953307:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880781.953309:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880781.953310:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880781.953311:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880781.953311:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880781.953313:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880781.953316:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880781.953321:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=149 v=5 (5 5 5 5) 00000400:00000200:3.0:1788880781.953321:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:1.0:1788880781.953326:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=149 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880781.953329:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880781.953329:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880781.953331:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880781.953334:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880781.953337:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880781.953338:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880781.953339:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880781.953340:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880781.953341:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ce00 00000100:00000040:1.0:1788880781.953347:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880781.953351:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe15 [1] + 224 00000100:00000001:1.0:1788880781.953360:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:3.0:1788880781.953361:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880781.953362:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880781.953364:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880781.953367:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880781.953369:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880781.953372:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880781.953380:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880781.953387:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:1.0:1788880781.953388:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780684672:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880781.953393:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880781.953395:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880781.953400:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880781.953405:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780684672/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880781.953409:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880781.953412:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880781.953426:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880781.953429:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000978ba7c2. 02000000:00000001:1.0:1788880781.953433:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880781.953435:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880781.953436:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880781.953437:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880781.953438:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:1.0:1788880781.953439:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880781.953440:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880781.953441:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000010:1.0:1788880781.953442:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:1.0:1788880781.953446:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880781.953446:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880781.953448:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880781.953450:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880781.953451:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880781.953453:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880781.953454:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880781.953455:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880781.953460:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880781.953474:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880781.953476:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880781.953483:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880781.953484:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880781.953486:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000702dddb0. 00000400:00000200:2.0:1788880781.953490:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788880781.953491:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:2.0:1788880781.953494:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880781.953498:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880781.953518:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880781.953523:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880781.953524:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880781.953526:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880781.953528:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880781.953529:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880781.953531:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880781.953533:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880781.953538:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=9 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880781.953540:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=116 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880781.953542:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880781.953543:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880781.953544:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880781.953547:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880781.953551:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880781.953553:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880781.953554:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880781.953555:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880781.953556:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880781.953558:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880781.953561:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880781.953563:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880781.953564:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880781.953566:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880781.953570:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780684800:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880781.953573:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880781.953578:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880781.953579:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780684800/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880797 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880781.953584:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880781.953585:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880781.953587:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000006eecdb68. 02000000:00000001:2.0:1788880781.953590:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880781.953591:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880781.953592:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880781.953594:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880781.953596:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 02000000:00000001:2.0:1788880781.953598:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880781.953599:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880781.953600:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:2.0:1788880781.953603:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880781.953604:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880781.953605:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880781.953608:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880781.953609:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880781.953610:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880781.953612:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880782.014376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880782.014383:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880782.014389:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880782.014391:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880782.014426:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880782.014433:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880782.014436:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880783.038349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880783.038356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880783.038363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880783.038364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880783.038387:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880783.038391:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880783.038404:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880784.062340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880784.062346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880784.062353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880784.062369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880784.062413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880784.062418:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880784.062421:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880785.086342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880785.086348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880785.086352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880785.086353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880785.086392:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880785.086395:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880785.086398:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880786.110413:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880786.110421:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880786.110423:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880786.110426:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880786.110431:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880786.110433:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880786.110435:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880786.110437:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880786.110441:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880786.110442:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880786.110482:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880786.110485:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880786.110487:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880787.070343:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880787.070343:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306418127, at 4306418176 00000100:00000040:2.0:1788880787.070343:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306418127, at 4306418176 00000100:00000010:1.0:1788880787.070357:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ffa8a4fd. 00000100:00000001:0.0:1788880787.070365:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880787.070366:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008921ad2e. 00000100:00000010:0.0:1788880787.070370:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000451fe31a. 00000020:00000040:0.0:1788880787.070377:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880787.070381:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880787.070384:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880787.070388:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880787.070396:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000044c69dd2. 00000100:00000001:0.0:1788880787.070403:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880787.070414:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:3.0:1788880787.070415:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306418127, at 4306418176 00000100:00000010:3.0:1788880787.070424:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001a3d5793. 00000100:00000001:0.0:1788880787.070429:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880787.070433:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880787.070441:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000451fe31a] to pc [ptlrpcd_00_00+0] req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880787.070469:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880787.070472:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880787.070475:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880787.070477:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:0.0:1788880787.070478:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000009ee4c6bf. 00000100:00000001:1.0:1788880787.070481:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:0.0:1788880787.070482:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880787.070485:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880787.070487:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880787.070488:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880787.070490:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:1.0:1788880787.070491:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880787.070493:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880787.070500:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880787.070501:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000013bf63f4. 00000100:00000001:1.0:1788880787.070501:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880787.070504:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880787.070506:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880787.070508:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880787.070511:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880787.070514:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880787.070523:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788880787.070526:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780684928:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880787.070532:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880787.070538:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880787.070539:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880787.070542:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880787.070543:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880787.070547:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000009ee4c6bf] to pc [ptlrpcd_00_01+1] req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880787.070547:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880787.070553:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b31f2b6a. 02000000:00000001:1.0:1788880787.070556:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880787.070563:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000001:0.0:1788880787.070567:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880787.070572:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000040:0.0:1788880787.070577:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:1.0:1788880787.070578:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780684928, portal 25 00000100:00000001:2.0:1788880787.070580:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880787.070582:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880787.070583:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880787.070586:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849366784 : -126822860184832 : ffff8ca7c0782300) 00000100:00000001:0.0:1788880787.070587:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1788880787.070587:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000010:0.0:1788880787.070593:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000007f91037. 00000100:00000040:2.0:1788880787.070593:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880787.070595:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880803 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1788880787.070599:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880787.070605:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880787.070609:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880787.070612:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880787.070619:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000087d24527. 00000100:00000001:2.0:1788880787.070619:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880787.070620:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880787.070622:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880787.070625:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880787.070626:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880787.070626:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880787.070628:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788880787.070630:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000ae22117. 02000000:00000001:2.0:1788880787.070632:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880787.070633:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780684928, offset 0 00000100:00000040:0.0:1788880787.070635:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788880787.070641:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780685056:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880787.070648:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880787.070649:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880787.070652:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880787.070652:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880787.070654:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880787.070658:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880787.070659:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000007f91037] to pc [ptlrpcd_00_02+2] req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880787.070662:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880787.070667:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008c6f5069. 00000400:00000200:1.0:1788880787.070669:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:2.0:1788880787.070672:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880787.070676:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:2.0:1788880787.070678:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000702dddb0. 00000400:00000200:1.0:1788880787.070680:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880787.070684:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:2.0:1788880787.070685:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000001:0.0:1788880787.070686:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880787.070688:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:0.0:1788880787.070691:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:2.0:1788880787.070692:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780685056, portal 10 00000100:00000001:3.0:1788880787.070697:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880787.070698:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:1.0:1788880787.070698:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [425:425:256:4294967295] 192.168.204.111@tcp LPNI seq info [425:425:8:4294967295] 00000100:00000001:3.0:1788880787.070702:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880787.070703:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880787.070707:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11755) 00000100:00000001:2.0:1788880787.070707:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849358720 : -126822860192896 : ffff8ca7c0780380) 00000100:00000001:3.0:1788880787.070707:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880787.070717:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880787.070718:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880787.070725:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880787.070733:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306418127, at 4306418176 00000800:00000200:1.0:1788880787.070742:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880787.070743:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001d590ced. 00000100:00000001:2.0:1788880787.070743:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880787.070748:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880787.070750:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880787.070751:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880787.070752:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000400:00000010:2.0:1788880787.070753:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001e963647. 02000000:00000001:3.0:1788880787.070754:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880787.070757:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788880787.070758:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780685056, offset 0 02000000:00000001:3.0:1788880787.070761:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880787.070765:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00100000:3.0:1788880787.070768:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780685184:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880787.070770:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880787.070775:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:1.0:1788880787.070778:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:3.0:1788880787.070778:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880787.070781:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880787.070782:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880787.070784:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:3.0:1788880787.070791:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880787.070792:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880787.070794:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880787.070796:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880787.070797:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d653e4f6. 00000100:00000001:1.0:1788880787.070798:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880787.070800:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880787.070803:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880787.070806:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306423247 00000400:00000200:2.0:1788880787.070807:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:3.0:1788880787.070809:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009f5de6c9. 00000100:00000010:1.0:1788880787.070813:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000400:00000200:2.0:1788880787.070813:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:3.0:1788880787.070815:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000999bcd04. 00000100:00000001:1.0:1788880787.070819:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:2.0:1788880787.070821:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [426:426:255:4294967295] 192.168.204.111@tcp LPNI seq info [426:426:7:4294967295] 00000100:00000200:3.0:1788880787.070821:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780685184, portal 10 00000100:00000001:1.0:1788880787.070824:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306423247 00000100:00000001:3.0:1788880787.070825:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880787.070828:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849371264 : -126822860180352 : ffff8ca7c0783480) 00000100:00000010:1.0:1788880787.070830:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000100:00000001:1.0:1788880787.070833:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:3.0:1788880787.070837:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880787.070843:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306423247 00000400:00000200:2.0:1788880787.070843:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:1.0:1788880787.070848:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000100:00000001:1.0:1788880787.070853:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880787.070857:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880787.070858:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306423247 00000800:00000200:2.0:1788880787.070858:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880787.070863:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880787.070865:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000800:00000010:2.0:1788880787.070865:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 00000400:00000010:3.0:1788880787.070866:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000100:00000001:1.0:1788880787.070869:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000200:3.0:1788880787.070871:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780685184, offset 0 00000100:00000001:1.0:1788880787.070873:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880787.070876:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880787.070877:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880787.070878:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880787.070882:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000400:00000200:3.0:1788880787.070883:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880787.070884:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880787.070884:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880787.070888:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788880787.070889:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880787.070892:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880787.070895:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880787.070896:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:2.0:1788880787.070897:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 203381 usecs (204 jiffies) 00000400:00000200:0.0:1788880787.070898:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880787.070900:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:3.0:1788880787.070902:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880787.070903:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306418380 00000100:00000001:1.0:1788880787.070906:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880787.070907:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306423247 00000100:00000001:1.0:1788880787.070910:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880787.070910:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880787.070911:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880787.070912:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306418380 00000100:00000001:1.0:1788880787.070914:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880787.070918:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306423247 00000400:00000200:3.0:1788880787.070918:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880787.070920:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880787.070921:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306418380 00000100:00000001:2.0:1788880787.070925:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306423247 00000100:00000001:2.0:1788880787.070927:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306418380 00000100:00000001:2.0:1788880787.070928:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306423247 00000100:00000001:2.0:1788880787.070929:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880787.070930:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880787.070932:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880787.070933:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880787.070936:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880787.070937:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880787.070938:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880787.070943:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880787.070945:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880787.070947:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880787.070960:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880787.070963:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [427:427:254:4294967295] 192.168.204.111@tcp LPNI seq info [427:427:6:4294967295] 00000400:00000200:3.0:1788880787.070971:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880787.070975:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880787.070977:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000ae22117 00000800:00000010:3.0:1788880787.070979:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000400:00000010:0.0:1788880787.070981:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000ae22117. 00000100:00000001:0.0:1788880787.070986:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:3.0:1788880787.070988:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:0.0:1788880787.070993:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880803 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880787.070993:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880787.070996:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880787.070998:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880787.071000:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 14235 usecs (15 jiffies) 00000100:00000001:3.0:1788880787.071003:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306418191 00000100:00000001:3.0:1788880787.071007:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306423247 00000100:00000001:3.0:1788880787.071014:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306418191 00000100:00000001:0.0:1788880787.071015:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880787.071016:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306423247 00000100:00000001:3.0:1788880787.071019:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306418191 00000100:00000001:3.0:1788880787.071020:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306423247 00000100:00000040:0.0:1788880787.071023:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880803 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880787.071023:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306418191 00000100:00000001:3.0:1788880787.071026:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306423247 00000100:00000001:3.0:1788880787.071029:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880787.071032:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880787.071036:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880787.071040:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880787.071043:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880787.071045:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880787.071046:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880787.071048:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880787.071052:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880787.071053:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880787.071054:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880787.071056:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880787.071066:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880787.071069:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000400:00000200:0.0:1788880787.071072:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880787.071076:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880787.071079:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001e963647 00000400:00000010:0.0:1788880787.071081:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001e963647. 00000100:00000001:0.0:1788880787.071084:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880787.071087:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880787.071106:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880787.071109:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880787.071114:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880787.071115:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880787.071151:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880787.071154:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:0.0:1788880787.071157:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880787.071161:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880787.071163:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00000400:00000010:0.0:1788880787.071165:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000100:00000001:0.0:1788880787.071167:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880787.071170:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880787.071177:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880787.071179:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880787.071195:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880787.071197:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880787.072588:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880787.072599:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880787.072603:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880787.072605:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880787.072612:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880787.072658:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0ce80 00000400:00000200:3.0:1788880787.072666:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xe1d [1] + 224 00000800:00000001:3.0:1788880787.072671:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880787.072677:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880787.072679:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880787.072683:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880787.072686:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880787.072691:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880787.072700:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880787.072725:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880787.072732:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880787.072735:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880787.072814:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880787.072815:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880787.072818:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880787.072822:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880787.072824:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788880787.072825:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880787.072829:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:3.0:1788880787.072831:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880787.072833:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:1.0:1788880787.072840:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000400:00000200:3.0:1788880787.072840:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000010:1.0:1788880787.072844:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:1.0:1788880787.072850:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:3.0:1788880787.072852:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0cf80 00000100:00000200:1.0:1788880787.072857:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880787.072863:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe2d [1] + 224 00000100:00000200:1.0:1788880787.072870:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880787.072877:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880787.072879:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880787.072881:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880787.072884:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880787.072887:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880787.072889:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880787.072891:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880787.072904:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=15 v=5 (5 5 5 5) 00000800:00000001:3.0:1788880787.072905:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:1.0:1788880787.072912:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=15 v=5 (1 1 1 1) 00000800:00000001:3.0:1788880787.072912:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880787.072917:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880787.072919:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880787.072923:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880787.072923:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00010000:00000001:1.0:1788880787.072929:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880787.072930:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:1.0:1788880787.072934:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880787.072936:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880787.072939:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880787.072949:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880787.072952:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880787.072972:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880787.072975:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880787.072977:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880787.072982:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880787.072994:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000451fe31a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780684928:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880787.073000:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880787.073000:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880787.073003:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:3.0:1788880787.073007:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788880787.073008:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000451fe31a x1875776780684928/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880787.073012:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880787.073020:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880787.073023:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880787.073026:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b31f2b6a. 02000000:00000001:1.0:1788880787.073031:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880787.073033:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880787.073035:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880787.073038:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:1.0:1788880787.073038:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880787.073042:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:1.0:1788880787.073043:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000044c69dd2. 00000400:00000001:0.0:1788880787.073047:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:1.0:1788880787.073049:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000001:0.0:1788880787.073050:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880787.073053:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009f5de6c9. 02000000:00000001:1.0:1788880787.073053:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880787.073056:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000451fe31a. 00000400:00000200:0.0:1788880787.073059:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000999bcd04 00000400:00000010:0.0:1788880787.073062:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000999bcd04. 00000100:00000001:1.0:1788880787.073063:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880787.073066:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880787.073068:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880787.073073:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880787.073079:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880787.073079:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880787.073082:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880787.073088:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880787.073091:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880787.073095:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880787.073104:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880787.073105:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880787.073107:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880787.073109:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880787.073110:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880787.073112:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880787.073114:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880787.073118:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=15 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880787.073122:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=122 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880787.073124:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880787.073126:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880787.073128:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880787.073131:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880787.073138:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880787.073141:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880787.073143:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880787.073145:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880787.073147:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880787.073156:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880787.073157:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880787.073162:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:3.0:1788880787.073164:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880787.073165:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880787.073167:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880787.073169:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1788880787.073171:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880787.073173:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:0.0:1788880787.073182:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780685184:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880787.073185:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880787.073189:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880787.073191:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880787.073193:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000007f91037 x1875776780685184/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880787.073196:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880787.073197:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880787.073199:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d653e4f6. 02000000:00000001:0.0:1788880787.073200:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880787.073201:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880787.073202:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880787.073203:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880787.073204:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000087d24527. 02000000:00000001:0.0:1788880787.073206:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880787.073207:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880787.073207:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000007f91037. 00000100:00000001:0.0:1788880787.073209:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880787.073210:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880787.073210:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880787.073213:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880787.073214:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880787.073215:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880787.073217:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880787.073258:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0cf00 00000400:00000200:3.0:1788880787.073263:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe25 [1] + 224 00000800:00000001:3.0:1788880787.073267:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880787.073277:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880787.073279:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880787.073283:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880787.073286:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880787.073289:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880787.073298:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880787.073327:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880787.073333:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880787.073335:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880787.073353:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880787.073357:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880787.073360:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880787.073361:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880787.073363:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000702dddb0. 00000400:00000200:2.0:1788880787.073367:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788880787.073369:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:2.0:1788880787.073371:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880787.073375:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880787.073385:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880787.073391:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880787.073392:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880787.073394:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880787.073396:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880787.073397:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880787.073398:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880787.073400:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880787.073404:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=155 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880787.073406:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=155 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880787.073408:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880787.073410:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880787.073411:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880787.073413:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880787.073416:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880787.073417:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880787.073418:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880787.073419:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880787.073422:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880787.073429:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880787.073436:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880787.073437:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880787.073440:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880787.073445:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000009ee4c6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780685056:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880787.073448:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880787.073449:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880787.073451:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000009ee4c6bf x1875776780685056/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880803 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880787.073454:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880787.073455:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880787.073457:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008c6f5069. 02000000:00000001:2.0:1788880787.073459:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880787.073460:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880787.073460:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880787.073462:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880787.073462:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000013bf63f4. 02000000:00000001:2.0:1788880787.073464:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880787.073465:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880787.073465:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000009ee4c6bf. 00000100:00000001:2.0:1788880787.073467:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880787.073468:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880787.073469:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880787.073471:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880787.073472:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880787.073473:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880787.073474:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880787.134330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880787.134337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880787.134342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880787.134343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880787.134378:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880787.134381:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880787.134383:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880788.158345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880788.158351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880788.158355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880788.158357:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880788.158359:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880788.158362:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880788.158364:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880788.158365:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880788.158366:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880788.158367:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880788.158399:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880788.158404:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880788.158406:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880789.182350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880789.182357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880789.182363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880789.182364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880789.182389:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880789.182411:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880789.182414:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880790.206799:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880790.206804:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880790.206809:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880790.206810:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880790.206939:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880790.206943:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880790.206946:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880791.230291:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880791.230297:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880791.230304:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880791.230305:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880791.230357:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880791.230360:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880791.230363:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880792.190345:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306423247, at 4306423296 00000100:00000040:3.0:1788880792.190350:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306423247, at 4306423296 00000100:00000010:1.0:1788880792.190361:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000040:0.0:1788880792.190368:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306423247, at 4306423296 00000100:00000010:3.0:1788880792.190368:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000010:0.0:1788880792.190379:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000040:2.0:1788880792.190460:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880792.190465:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880792.190472:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 00000020:00000040:2.0:1788880792.190476:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880792.190480:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880792.190482:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880792.190486:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880792.190493:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 00000100:00000001:2.0:1788880792.190498:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880792.190510:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880792.190518:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880792.190520:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880792.190524:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_03+3] req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880792.190549:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880792.190551:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880792.190553:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880792.190555:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000020:00000040:2.0:1788880792.190556:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880792.190558:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880792.190559:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880792.190560:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880792.190562:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 00000100:00000001:2.0:1788880792.190565:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880792.190569:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880792.190586:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880792.190590:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880792.190591:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880792.190598:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880792.190601:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000040:2.0:1788880792.190606:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_00+0] req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880792.190607:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.190608:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880792.190635:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880792.190638:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880792.190641:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880792.190643:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880792.190647:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880792.190658:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880792.190668:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880792.190671:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880792.190671:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880792.190678:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880792.190678:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880792.190682:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880792.190690:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880792.190692:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880792.190700:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880792.190712:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780685312:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880792.190720:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880792.190736:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880792.190738:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880792.190743:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880792.190743:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880792.190748:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880792.190749:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880792.190756:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:1.0:1788880792.190757:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880792.190763:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780685440:192.168.204.111@tcp:400: 02000000:00000010:3.0:1788880792.190763:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000358be29b. 02000000:00000001:3.0:1788880792.190769:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880792.190772:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880792.190777:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880792.190779:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880792.190781:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:3.0:1788880792.190781:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000071e96305. 02000000:00000010:1.0:1788880792.190789:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005578e134. 02000000:00000001:1.0:1788880792.190794:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880792.190794:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000400:00000010:1.0:1788880792.190802:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000200:3.0:1788880792.190805:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780685312, portal 25 00000100:00000001:3.0:1788880792.190809:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880792.190810:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938435456 : -126823771116160 : ffff8ca78a2c6d80) 00000100:00000040:3.0:1788880792.190818:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880808 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880792.190825:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880792.190828:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880792.190832:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a165c7f9. 00000100:00000200:3.0:1788880792.190835:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780685312, offset 0 00000400:00000200:3.0:1788880792.190843:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880792.190863:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880792.190869:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880792.190874:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880792.190878:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880792.190882:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880792.190886:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [428:428:256:4294967295] 192.168.204.111@tcp LPNI seq info [428:428:8:4294967295] 00000400:00000200:3.0:1788880792.190898:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880792.190905:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880792.190911:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000800:00000200:3.0:1788880792.190918:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880792.190927:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880792.190932:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880792.190951:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.190956:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306428367 00000100:00000010:3.0:1788880792.190960:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000067a34a59. 00000100:00000001:3.0:1788880792.190971:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788880792.190980:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306428367 00000100:00000010:3.0:1788880792.190985:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a5ad6403. 00000100:00000001:3.0:1788880792.190989:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880792.190992:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306428367 00000100:00000001:2.0:1788880792.191000:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.191001:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306423247 00000100:00000040:2.0:1788880792.191008:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880792.191009:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306428367 00000100:00000010:3.0:1788880792.191016:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000025d9193b. 00000100:00000001:2.0:1788880792.191018:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880792.191022:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000010:2.0:1788880792.191025:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000100:00000001:3.0:1788880792.191029:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000020:00000040:2.0:1788880792.191031:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880792.191041:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880792.191041:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880792.191046:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880792.191051:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880792.191052:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880792.191069:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e53a8f34. 00000100:00000001:3.0:1788880792.191069:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880792.191079:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880792.191086:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880792.191091:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880792.191093:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880792.191096:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_01+1] req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880792.191103:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880792.191106:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000200:0.0:1788880792.191108:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:2.0:1788880792.191109:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880792.191113:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:0.0:1788880792.191115:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000040:2.0:1788880792.191118:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11760) 00000100:00000001:3.0:1788880792.191118:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880792.191123:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880792.191124:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880792.191133:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880792.191134:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880792.191143:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a165c7f9 00000100:00000001:3.0:1788880792.191145:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:0.0:1788880792.191146:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a165c7f9. 00000100:00000001:3.0:1788880792.191148:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880792.191151:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880792.191154:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880792.191157:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880792.191160:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00100000:3.0:1788880792.191161:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780685568:192.168.204.111@tcp:400: 00000100:00000200:0.0:1788880792.191168:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880808 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880792.191168:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880792.191176:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:1.0:1788880792.191178:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 02000000:00000001:3.0:1788880792.191179:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880792.191183:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:3.0:1788880792.191191:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880792.191193:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880808 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880792.191194:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780685440, portal 10 02000000:00000010:3.0:1788880792.191197:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000fdf771c7. 00000100:00000001:1.0:1788880792.191206:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880792.191210:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880792.191210:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 00000100:00000001:0.0:1788880792.191231:0:201279:0:(events.c:68:request_out_callback()) Process leaving 02000000:00000001:3.0:1788880792.191293:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880792.191299:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000162a13e6. 00000100:00000040:1.0:1788880792.191303:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880792.191307:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000001:1.0:1788880792.191310:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880792.191316:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880792.191323:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:3.0:1788880792.191324:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780685568, portal 10 00000100:00000200:1.0:1788880792.191326:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780685440, offset 0 00000100:00000001:3.0:1788880792.191330:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880792.191333:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000400:00000200:1.0:1788880792.191337:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880792.191347:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880792.191351:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880792.191364:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880792.191373:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880792.191378:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880792.191384:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880792.191390:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:3.0:1788880792.191392:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880792.191399:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [429:429:256:4294967295] 192.168.204.111@tcp LPNI seq info [429:429:8:4294967295] 00000400:00000010:3.0:1788880792.191400:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e78d7264. 00000100:00000200:3.0:1788880792.191412:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780685568, offset 0 00000400:00000200:1.0:1788880792.191420:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:3.0:1788880792.191430:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880792.191440:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880792.191445:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880792.191448:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000800:00000200:1.0:1788880792.191462:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880792.191468:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:1.0:1788880792.191472:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880792.191475:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880792.191477:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880792.191482:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880792.191488:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880792.191491:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [430:430:255:4294967295] 192.168.204.111@tcp LPNI seq info [430:430:7:4294967295] 00000400:00000200:3.0:1788880792.191501:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880792.191507:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880792.191513:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000800:00000200:3.0:1788880792.191518:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880792.191522:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880792.191524:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880792.191527:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000040:1.0:1788880792.191529:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 201820 usecs (202 jiffies) 00000100:00000001:3.0:1788880792.191533:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880792.191535:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306423499 00000100:00000001:1.0:1788880792.191542:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306428367 00000100:00000040:3.0:1788880792.191543:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 14145 usecs (15 jiffies) 00000100:00000001:1.0:1788880792.191547:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306423499 00000100:00000001:1.0:1788880792.191552:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306428367 00000100:00000001:3.0:1788880792.191552:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306423312 00000100:00000001:1.0:1788880792.191566:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306423499 00000100:00000001:3.0:1788880792.191566:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306428367 00000100:00000001:3.0:1788880792.191580:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306423312 00000100:00000001:1.0:1788880792.191581:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306428367 00000100:00000001:3.0:1788880792.191598:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306428367 00000100:00000001:1.0:1788880792.191599:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306423499 00000100:00000001:3.0:1788880792.191610:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306423312 00000100:00000001:1.0:1788880792.191611:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306428367 00000100:00000001:1.0:1788880792.191614:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880792.191616:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.191623:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306428367 00000100:00000001:1.0:1788880792.191624:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.191628:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306423312 00000100:00000001:1.0:1788880792.191630:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880792.191634:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306428367 00000100:00000001:3.0:1788880792.191637:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880792.191644:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.191646:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.191648:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880792.191649:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880792.191653:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880792.191654:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880792.191656:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880792.191658:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880792.191659:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880792.191664:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.191665:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880792.191669:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.191669:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880792.191671:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.191676:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880792.191738:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880792.191742:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000400:00000200:2.0:1788880792.191746:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880792.191750:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880792.191759:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:2.0:1788880792.191761:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:2.0:1788880792.191764:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880792.191767:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880792.191774:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880792.191776:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880792.191780:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880792.191781:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880792.191847:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880792.191849:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:2.0:1788880792.191851:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880792.191854:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880792.191856:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e78d7264 00000400:00000010:2.0:1788880792.191857:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e78d7264. 00000100:00000001:2.0:1788880792.191859:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880792.191861:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880792.191868:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880792.191870:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880792.191875:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880792.191877:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880792.192632:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880792.192642:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880792.192645:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880792.192648:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880792.192654:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880792.192663:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0d000 00000400:00000200:3.0:1788880792.192724:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xe35 [1] + 224 00000800:00000001:3.0:1788880792.192732:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880792.192738:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880792.192740:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880792.192744:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880792.192748:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880792.192752:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880792.192762:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880792.192779:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880792.192784:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880792.192787:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880792.192825:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880792.192829:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880792.192835:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880792.192837:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880792.192840:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000071e96305. 00000400:00000200:0.0:1788880792.192844:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000400:00000010:0.0:1788880792.192846:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000001:0.0:1788880792.192850:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880792.192855:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880792.192865:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880792.192871:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880792.192872:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880792.192875:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880792.192876:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880792.192878:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880792.192880:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880792.192882:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880792.192890:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=20 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880792.192903:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=20 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880792.192906:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880792.192907:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880792.192910:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880792.192913:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880792.192916:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880792.192920:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880792.192927:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880792.192930:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880792.192932:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880792.192936:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880792.192942:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780685312:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880792.192946:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880792.192947:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880792.192949:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000175072aa x1875776780685312/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880792.192953:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880792.192956:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880792.192958:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000358be29b. 02000000:00000001:0.0:1788880792.192960:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880792.192962:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880792.192964:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880792.192966:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880792.192968:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 02000000:00000001:0.0:1788880792.192971:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880792.192972:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880792.192973:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000175072aa. 00000100:00000001:0.0:1788880792.192977:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880792.192978:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880792.192979:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880792.192982:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880792.192986:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880792.192988:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880792.192989:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880792.193435:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880792.193444:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880792.193446:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880792.193448:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880792.193453:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880792.193462:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d080 00000400:00000200:0.0:1788880792.193467:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe3d [1] + 224 00000800:00000001:0.0:1788880792.193471:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880792.193477:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880792.193481:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880792.193484:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880792.193487:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880792.193489:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880792.193497:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880792.193511:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880792.193516:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880792.193518:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880792.193557:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880792.193561:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880792.193567:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880792.193569:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880792.193571:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:1.0:1788880792.193575:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000400:00000010:1.0:1788880792.193593:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:1.0:1788880792.193597:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880792.193603:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880792.193612:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880792.193619:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880792.193621:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880792.193623:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880792.193626:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880792.193628:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880792.193630:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880792.193633:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880792.193640:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=10 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880792.193644:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=10 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880792.193646:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880792.193648:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880792.193650:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880792.193654:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880792.193657:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880792.193659:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880792.193661:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880792.193662:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880792.193667:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880792.193674:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880792.193676:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880792.193678:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880792.193696:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880792.193705:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:1.0:1788880792.193706:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780685440:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880792.193712:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880792.193713:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880792.193716:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880792.193718:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880792.193723:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880792.193724:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880792.193730:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780685440/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880792.193733:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d100 00000400:00000200:2.0:1788880792.193743:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe41 [1] + 224 00000100:00000001:1.0:1788880792.193747:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000800:00000001:2.0:1788880792.193748:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:1.0:1788880792.193757:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000800:00000001:2.0:1788880792.193760:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880792.193761:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005578e134. 00000400:00000200:2.0:1788880792.193763:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1788880792.193768:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000200:2.0:1788880792.193769:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000020:00000001:1.0:1788880792.193773:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880792.193774:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000040:1.0:1788880792.193776:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000200:2.0:1788880792.193779:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788880792.193782:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880792.193785:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 02000000:00000001:1.0:1788880792.193792:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:2.0:1788880792.193793:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880792.193796:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880792.193799:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000100:00000001:2.0:1788880792.193814:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880792.193815:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880792.193818:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880792.193821:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880792.193821:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880792.193826:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880792.193827:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880792.193830:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880792.193833:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880792.193835:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.193841:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880792.193845:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880792.193849:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880792.193851:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880792.193853:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000162a13e6. 00000400:00000200:3.0:1788880792.193856:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000400:00000010:3.0:1788880792.193858:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000001:3.0:1788880792.193861:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880792.193865:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880792.193875:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880792.193882:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880792.193883:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880792.193884:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880792.193886:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880792.193887:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880792.193888:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880792.193890:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880792.193894:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=20 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880792.193896:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=127 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880792.193898:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880792.193899:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880792.193901:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880792.193904:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880792.193910:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.193912:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880792.193913:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880792.193914:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880792.193915:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880792.193918:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880792.193924:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880792.193932:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880792.193934:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880792.193937:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880792.193943:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780685568:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880792.193947:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880792.193949:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880792.193951:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780685568/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880808 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880792.193956:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880792.193958:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880792.193959:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000fdf771c7. 02000000:00000001:3.0:1788880792.193962:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880792.193963:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880792.193964:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880792.193966:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880792.193967:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e53a8f34. 02000000:00000001:3.0:1788880792.193970:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880792.193971:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880792.193977:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:3.0:1788880792.193979:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880792.193980:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880792.193981:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880792.193985:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880792.193986:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880792.193987:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880792.193989:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880792.254340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880792.254345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880792.254349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880792.254350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880792.254380:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880792.254383:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880792.254385:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880793.278357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880793.278364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880793.278369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880793.278371:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880793.278434:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880793.278438:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880793.278441:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880794.302393:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880794.302400:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880794.302406:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880794.302407:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880794.303247:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880794.303251:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880794.303254:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880795.326339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880795.326347:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880795.326350:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880795.326358:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880795.326363:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880795.326381:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880795.326383:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880795.326386:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880795.326389:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880795.326391:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880795.326456:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880795.326459:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880795.326461:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880796.350337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880796.350343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880796.350348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880796.350349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880796.350404:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880796.350407:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880796.350409:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880797.310325:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306428367, at 4306428416 00000100:00000040:3.0:1788880797.310326:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306428367, at 4306428416 00000100:00000040:2.0:1788880797.310328:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306428367, at 4306428416 00000100:00000010:1.0:1788880797.310341:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a5ad6403. 00000100:00000010:2.0:1788880797.310342:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:3.0:1788880797.310352:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000025d9193b. 00000100:00000040:0.0:1788880797.310452:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880797.310458:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880797.310465:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000007f91037. 00000020:00000040:0.0:1788880797.310469:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880797.310482:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880797.310485:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880797.310489:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880797.310496:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000087d24527. 00000100:00000001:0.0:1788880797.310504:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880797.310513:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880797.310534:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880797.310537:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880797.310542:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000007f91037] to pc [ptlrpcd_00_02+2] req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880797.310583:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880797.310587:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880797.310589:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880797.310594:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880797.310597:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880797.310602:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880797.310603:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880797.310605:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880797.310609:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880797.310612:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880797.310614:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880797.310616:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780685696:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880797.310619:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880797.310624:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880797.310626:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880797.310628:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880797.310631:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007c59c7b9. 02000000:00000001:3.0:1788880797.310633:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880797.310637:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000162a13e6. 00000100:00000001:0.0:1788880797.310640:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880797.310642:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000040:0.0:1788880797.310644:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:3.0:1788880797.310648:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780685696, portal 25 00000100:00000001:0.0:1788880797.310649:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880797.310652:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000010:0.0:1788880797.310655:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000937cf312. 00000100:00000001:3.0:1788880797.310655:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849371264 : -126822860180352 : ffff8ca7c0783480) 00000100:00000040:3.0:1788880797.310660:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880813 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1788880797.310661:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880797.310666:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880797.310670:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880797.310671:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880797.310674:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880797.310679:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000bd68c00a. 00000100:00000040:3.0:1788880797.310679:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880797.310682:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000003539bb5a. 00000100:00000001:0.0:1788880797.310685:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880797.310685:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780685696, offset 0 00000100:00000040:0.0:1788880797.310695:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:3.0:1788880797.310695:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880797.310706:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880797.310708:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880797.310710:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:0.0:1788880797.310714:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000937cf312] to pc [ptlrpcd_00_03+3] req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880797.310717:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880797.310724:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880797.310732:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880797.310735:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880797.310738:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880797.310739:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [431:431:256:4294967295] 192.168.204.111@tcp LPNI seq info [431:431:8:4294967295] 00000100:00000040:0.0:1788880797.310746:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:3.0:1788880797.310751:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880797.310758:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000800:00000200:3.0:1788880797.310759:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880797.310763:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000a2475c3c. 00000800:00000010:3.0:1788880797.310764:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:2.0:1788880797.310767:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:0.0:1788880797.310769:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000800:00000200:3.0:1788880797.310773:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880797.310774:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880797.310775:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880797.310779:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880797.310779:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880797.310781:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880797.310784:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:2.0:1788880797.310787:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880797.310788:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003806e31f. 00000800:00000200:3.0:1788880797.310789:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880797.310801:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880797.310802:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880797.310805:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880797.310806:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880797.310809:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880797.310812:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880797.310812:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306433487 02000000:00000001:2.0:1788880797.310816:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880797.310821:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306428367 02000000:00000001:2.0:1788880797.310823:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880797.310827:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306433487 00000100:00000001:0.0:1788880797.310828:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880797.310831:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880797.310833:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000025d9193b. 00000100:00100000:2.0:1788880797.310837:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780685824:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880797.310839:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a2475c3c] to pc [ptlrpcd_00_00+0] req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880797.310839:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880797.310843:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306433487 00000100:00000001:2.0:1788880797.310847:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000010:3.0:1788880797.310849:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002ad0ec0e. 02000000:00000001:2.0:1788880797.310852:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880797.310854:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 02000000:00000001:2.0:1788880797.310856:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880797.310860:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306433487 02000000:00000001:2.0:1788880797.310862:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:3.0:1788880797.310865:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b0b8cfdf. 02000000:00000010:2.0:1788880797.310867:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b2a347c7. 00000100:00000001:3.0:1788880797.310868:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 02000000:00000001:2.0:1788880797.310873:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880797.310873:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880797.310878:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880797.310879:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880797.310881:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880797.310882:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000702dddb0. 00000100:00000040:0.0:1788880797.310883:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880797.310884:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880797.310888:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000040:0.0:1788880797.310890:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880797.310892:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:2.0:1788880797.310894:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780685824, portal 10 00000100:00000001:3.0:1788880797.310895:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:0.0:1788880797.310897:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11766) 00000100:00000001:2.0:1788880797.310899:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880797.310900:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880797.310903:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880797.310904:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849367680 : -126822860183936 : ffff8ca7c0782680) 00000100:00000001:3.0:1788880797.310905:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880797.310912:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880797.310913:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880797.310914:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880797.310918:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:1.0:1788880797.310923:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880797.310923:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880797.310926:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00100000:3.0:1788880797.310927:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788880797.310929:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880797.310930:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880797.310933:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880797.310934:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880797.310935:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:2.0:1788880797.310936:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880797.310938:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880797.310941:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880797.310942:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000003539bb5a 00000400:00000010:2.0:1788880797.310943:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c3a5d4ae. 00000100:00000001:3.0:1788880797.310943:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:1.0:1788880797.310945:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000003539bb5a. 00000100:00000001:3.0:1788880797.310946:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880797.310951:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000040:3.0:1788880797.310952:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880797.310954:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780685824, offset 0 00000100:00000200:1.0:1788880797.310957:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880813 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880797.310968:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880797.310969:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880797.310972:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880797.310976:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:2.0:1788880797.310978:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880797.310978:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880797.310983:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880797.310985:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788880797.310986:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:3.0:1788880797.310987:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780685952:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880797.310991:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880797.310994:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880813 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880797.310994:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880797.310995:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880797.310997:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880797.311002:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880797.311002:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880797.311008:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880797.311010:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880797.311013:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:2.0:1788880797.311014:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [432:432:255:4294967295] 192.168.204.111@tcp LPNI seq info [432:432:7:4294967295] 00000400:00000200:2.0:1788880797.311023:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880797.311028:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880797.311032:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000800:00000200:2.0:1788880797.311036:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880797.311042:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880797.311045:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880797.311062:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880797.311063:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000063883087. 00000100:00000040:2.0:1788880797.311067:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 200285 usecs (201 jiffies) 02000000:00000001:3.0:1788880797.311068:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880797.311073:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306428617 00000400:00000010:3.0:1788880797.311074:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f65244a7. 00000100:00000001:2.0:1788880797.311077:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306433487 00000400:00000010:3.0:1788880797.311080:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000100:00000001:2.0:1788880797.311081:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306428617 00000100:00000001:2.0:1788880797.311086:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306433487 00000100:00000200:3.0:1788880797.311087:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780685952, portal 10 00000100:00000001:2.0:1788880797.311088:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306428617 00000100:00000001:3.0:1788880797.311091:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880797.311093:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306433487 00000100:00000001:3.0:1788880797.311094:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849361408 : -126822860190208 : ffff8ca7c0780e00) 00000100:00000001:2.0:1788880797.311096:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306428617 00000100:00000001:2.0:1788880797.311100:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306433487 00000100:00000001:2.0:1788880797.311103:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880797.311104:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880797.311108:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880797.311112:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880797.311116:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880797.311131:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880797.311134:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788880797.311136:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880797.311137:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880797.311137:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000010:3.0:1788880797.311141:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 00000800:00000010:0.0:1788880797.311142:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000100:00000001:2.0:1788880797.311143:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:3.0:1788880797.311146:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780685952, offset 0 00000100:00000001:2.0:1788880797.311147:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880797.311148:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880797.311150:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880797.311153:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880797.311157:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880797.311161:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880797.311167:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880797.311168:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c3a5d4ae 00000400:00000200:3.0:1788880797.311171:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:0.0:1788880797.311172:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c3a5d4ae. 00000400:00000200:3.0:1788880797.311176:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880797.311179:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880797.311185:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880797.311192:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880797.311194:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880797.311199:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880797.311200:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:3.0:1788880797.311212:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880797.311216:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880797.311218:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [433:433:255:4294967295] 192.168.204.111@tcp LPNI seq info [433:433:7:4294967295] 00000400:00000200:3.0:1788880797.311223:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880797.311226:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880797.311229:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000800:00000200:3.0:1788880797.311247:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880797.311251:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880797.311253:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880797.311264:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880797.311266:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 14061 usecs (15 jiffies) 00000100:00000001:3.0:1788880797.311269:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306428432 00000100:00000001:3.0:1788880797.311271:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306433487 00000100:00000001:3.0:1788880797.311273:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306428432 00000100:00000001:3.0:1788880797.311274:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306433487 00000100:00000001:3.0:1788880797.311276:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306428432 00000100:00000001:3.0:1788880797.311277:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306433487 00000100:00000001:3.0:1788880797.311279:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306428432 00000100:00000001:3.0:1788880797.311280:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306433487 00000100:00000001:3.0:1788880797.311282:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880797.311284:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880797.311286:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880797.311288:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880797.311339:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880797.311342:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:2.0:1788880797.311346:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880797.311350:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880797.311353:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 00000400:00000010:2.0:1788880797.311355:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 00000100:00000001:2.0:1788880797.311358:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880797.311361:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880797.311369:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880797.311372:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880797.311384:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880797.311386:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880797.312305:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880797.312314:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880797.312317:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880797.312319:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880797.312326:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880797.312336:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0d180 00000400:00000200:1.0:1788880797.312341:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xe4d [1] + 224 00000800:00000001:1.0:1788880797.312347:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880797.312352:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880797.312360:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880797.312364:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880797.312368:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880797.312372:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880797.312381:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880797.312395:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880797.312402:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880797.312406:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880797.312408:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880797.312411:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880797.312413:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880797.312415:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880797.312421:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d280 00000400:00000200:1.0:1788880797.312426:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe5d [1] + 224 00000800:00000001:1.0:1788880797.312429:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880797.312434:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880797.312435:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880797.312438:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880797.312440:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880797.312442:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880797.312450:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880797.312456:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880797.312462:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880797.312464:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880797.312466:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000162a13e6. 00000100:00000001:1.0:1788880797.312468:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:3.0:1788880797.312470:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000400:00000010:3.0:1788880797.312473:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000001:3.0:1788880797.312475:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:1.0:1788880797.312480:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1788880797.312480:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880797.312484:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880797.312494:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880797.312501:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880797.312503:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880797.312504:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880797.312506:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880797.312508:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880797.312510:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000001:2.0:1788880797.312513:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880797.312514:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000001:2.0:1788880797.312517:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880797.312518:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880797.312520:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000f65244a7. 00000400:00000200:2.0:1788880797.312524:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000400:00000010:2.0:1788880797.312525:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:2.0:1788880797.312528:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880797.312532:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880797.312539:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880797.312546:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880797.312547:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880797.312549:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880797.312550:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880797.312552:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880797.312553:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880797.312556:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880797.312562:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=25 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880797.312566:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=132 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880797.312569:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880797.312570:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880797.312572:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880797.312577:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880797.312583:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880797.312589:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:3.0:1788880797.312590:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880797.312592:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00001000:3.0:1788880797.312595:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=25 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880797.312596:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880797.312599:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880797.312601:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=25 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880797.312604:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:0.0:1788880797.312611:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:2.0:1788880797.312612:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880797.312618:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880797.312621:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:0.0:1788880797.312622:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880797.312625:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880797.312626:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880797.312628:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880797.312632:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880797.312640:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880797.312641:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00010000:00000001:3.0:1788880797.312646:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880797.312649:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880797.312653:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880797.312654:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d200 00000100:00000040:3.0:1788880797.312662:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880797.312664:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe55 [1] + 224 00000100:00100000:2.0:1788880797.312665:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000a2475c3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780685952:192.168.204.111@tcp:400: 00000800:00000001:0.0:1788880797.312674:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880797.312675:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880797.312680:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880797.312681:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:0.0:1788880797.312682:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880797.312685:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:0.0:1788880797.312686:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:2.0:1788880797.312687:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000a2475c3c x1875776780685952/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880797.312689:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880797.312696:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:3.0:1788880797.312698:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880797.312704:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880797.312710:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:0.0:1788880797.312711:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880797.312714:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880797.312719:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000063883087. 02000000:00000001:2.0:1788880797.312725:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00100000:3.0:1788880797.312726:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780685696:192.168.204.111@tcp:400: 00000020:00000001:2.0:1788880797.312729:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880797.312733:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:0.0:1788880797.312735:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:2.0:1788880797.312737:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880797.312741:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003806e31f. 02000000:00000001:2.0:1788880797.312748:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880797.312748:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880797.312751:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880797.312753:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000010:2.0:1788880797.312754:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000a2475c3c. 00000100:00000040:3.0:1788880797.312759:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000007f91037 x1875776780685696/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880797.312760:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880797.312760:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880797.312764:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880797.312769:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880797.312772:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880797.312774:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000800:00000001:0.0:1788880797.312777:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880797.312777:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:3.0:1788880797.312777:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880797.312781:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880797.312782:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007c59c7b9. 00000100:00000001:2.0:1788880797.312785:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880797.312787:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880797.312788:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880797.312790:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880797.312793:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880797.312794:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:3.0:1788880797.312796:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880797.312798:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:3.0:1788880797.312799:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000087d24527. 00000400:00000001:1.0:1788880797.312803:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:3.0:1788880797.312804:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000001:1.0:1788880797.312806:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:3.0:1788880797.312807:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000010:1.0:1788880797.312809:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000702dddb0. 00000100:00000010:3.0:1788880797.312810:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000007f91037. 00000100:00000001:3.0:1788880797.312813:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880797.312814:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880797.312817:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880797.312820:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000100:00000001:3.0:1788880797.312822:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:1.0:1788880797.312824:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:3.0:1788880797.312825:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880797.312829:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880797.312829:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880797.312833:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880797.312836:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880797.312846:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880797.312853:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880797.312854:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880797.312856:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880797.312857:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880797.312859:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880797.312860:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880797.312862:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880797.312865:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=15 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880797.312868:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=15 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880797.312870:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880797.312871:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880797.312873:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880797.312875:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880797.312877:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880797.312878:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880797.312880:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880797.312881:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880797.312883:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880797.312889:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880797.312891:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880797.312892:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880797.312895:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880797.312901:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780685824:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880797.312911:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880797.312912:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880797.312915:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000937cf312 x1875776780685824/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880813 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880797.312921:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880797.312922:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880797.312923:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b2a347c7. 02000000:00000001:1.0:1788880797.312926:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880797.312928:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880797.312929:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880797.312931:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880797.312932:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000bd68c00a. 02000000:00000001:1.0:1788880797.312934:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880797.312935:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880797.312936:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000937cf312. 00000100:00000001:1.0:1788880797.312944:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880797.312945:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880797.312947:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880797.312950:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880797.312951:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880797.312953:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880797.312954:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880797.374347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880797.374353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880797.374358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880797.374361:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880797.374363:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880797.374365:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880797.374367:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880797.374369:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880797.374371:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880797.374372:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880797.374418:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880797.374421:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880797.374423:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880798.398366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880798.398372:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880798.398376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880798.398376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880798.398435:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880798.398440:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880798.398444:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880799.422341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880799.422346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880799.422351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880799.422352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880799.422385:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880799.422390:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880799.422394:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880800.446278:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880800.446284:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880800.446288:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880800.446289:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880800.446337:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880800.446342:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880800.446346:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880801.470304:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880801.470309:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880801.470314:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880801.470315:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880801.470365:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880801.470368:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880801.470370:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880802.430327:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880802.430327:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306433487, at 4306433536 00000100:00000040:2.0:1788880802.430327:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306433487, at 4306433536 00000100:00000040:3.0:1788880802.430327:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306433487, at 4306433536 00000100:00000001:0.0:1788880802.430337:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880802.430341:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000025d9193b. 00000100:00000010:2.0:1788880802.430343:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002ad0ec0e. 00000100:00000010:3.0:1788880802.430343:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b0b8cfdf. 00000100:00000010:0.0:1788880802.430346:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000802a821. 00000020:00000040:0.0:1788880802.430350:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880802.430354:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880802.430357:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880802.430361:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880802.430368:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000005f53e16f. 00000100:00000001:0.0:1788880802.430374:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880802.430385:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880802.430396:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880802.430399:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880802.430405:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000802a821] to pc [ptlrpcd_00_01+1] req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880802.430437:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880802.430447:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880802.430451:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880802.430452:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880802.430453:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880802.430457:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000040:1.0:1788880802.430460:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880802.430461:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b8846eac. 00000020:00000040:0.0:1788880802.430466:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880802.430470:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880802.430471:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:0.0:1788880802.430472:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880802.430474:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880802.430475:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880802.430478:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000010:0.0:1788880802.430480:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000050c5542b. 02000000:00000001:1.0:1788880802.430481:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880802.430484:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880802.430485:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880802.430488:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780686080:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880802.430492:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880802.430493:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880802.430498:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880802.430501:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880802.430505:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880802.430507:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000010:1.0:1788880802.430509:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004d9329ab. 00000100:00000001:0.0:1788880802.430510:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880802.430514:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880802.430518:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b8846eac] to pc [ptlrpcd_00_02+2] req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880802.430521:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1788880802.430522:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000001:3.0:1788880802.430526:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880802.430532:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000200:1.0:1788880802.430541:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780686080, portal 25 00000100:00000001:1.0:1788880802.430544:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880802.430553:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849370368 : -126822860181248 : ffff8ca7c0783100) 00000100:00000001:0.0:1788880802.430554:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880802.430558:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880802.430562:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880818 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880802.430563:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880802.430567:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:0.0:1788880802.430568:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006f591845. 00000100:00000001:1.0:1788880802.430569:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880802.430572:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:0.0:1788880802.430573:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:1.0:1788880802.430576:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880802.430577:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880802.430578:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:1.0:1788880802.430580:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000487fae38. 02000000:00000001:0.0:1788880802.430581:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000200:1.0:1788880802.430584:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780686080, offset 0 02000000:00000001:0.0:1788880802.430585:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880802.430585:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880802.430592:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b7399401. 00000400:00000200:1.0:1788880802.430594:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880802.430600:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880802.430602:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880802.430605:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880802.430607:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:1.0:1788880802.430610:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880802.430610:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880802.430618:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880802.430623:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000400:00000200:1.0:1788880802.430623:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880802.430626:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880802.430627:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880802.430629:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:0.0:1788880802.430637:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006f591845] to pc [ptlrpcd_00_03+3] req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880802.430637:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [434:434:256:4294967295] 192.168.204.111@tcp LPNI seq info [434:434:8:4294967295] 00000400:00000200:1.0:1788880802.430646:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880802.430651:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880802.430657:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000100:00000001:0.0:1788880802.430661:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880802.430664:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000200:1.0:1788880802.430664:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:3.0:1788880802.430667:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880802.430671:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:3.0:1788880802.430672:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880802.430677:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:0.0:1788880802.430678:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11771) 00000100:00100000:3.0:1788880802.430680:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780686208:192.168.204.111@tcp:400: 00000800:00000200:1.0:1788880802.430681:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880802.430685:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880802.430688:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880802.430689:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306433487, at 4306433536 02000000:00000001:3.0:1788880802.430690:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880802.430696:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:0.0:1788880802.430697:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000067a34a59. 02000000:00000010:3.0:1788880802.430700:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e0fe5497. 02000000:00000001:3.0:1788880802.430704:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880802.430708:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000162a13e6. 00000100:00000001:0.0:1788880802.430709:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880802.430709:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880802.430712:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:3.0:1788880802.430713:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000001:0.0:1788880802.430715:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880802.430715:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306438607 00000100:00000200:3.0:1788880802.430719:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780686208, portal 10 00000100:00000040:0.0:1788880802.430721:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880802.430722:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000100:00000001:3.0:1788880802.430725:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880802.430728:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364096 : -126822860187520 : ffff8ca7c0781880) 00000100:00000001:1.0:1788880802.430729:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788880802.430735:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880802.430737:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880802.430738:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306438607 02000000:00000001:0.0:1788880802.430739:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1788880802.430742:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880802.430743:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:1.0:1788880802.430745:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 02000000:00000001:0.0:1788880802.430747:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880802.430751:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880802.430755:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306438607 00000100:00000001:3.0:1788880802.430756:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:1.0:1788880802.430760:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000100:00000040:3.0:1788880802.430764:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00100000:0.0:1788880802.430765:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006f591845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780686336:192.168.204.111@tcp:400: 00000800:00000200:2.0:1788880802.430765:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880802.430766:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000010:3.0:1788880802.430769:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b6d0d6b5. 00000800:00000010:2.0:1788880802.430771:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000100:00000001:0.0:1788880802.430775:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000200:3.0:1788880802.430775:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780686208, offset 0 00000100:00000001:1.0:1788880802.430777:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306438607 00000400:00000200:2.0:1788880802.430779:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:0.0:1788880802.430781:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:1.0:1788880802.430782:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000400:00000200:3.0:1788880802.430785:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880802.430786:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880802.430786:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:2.0:1788880802.430790:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:0.0:1788880802.430792:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880802.430793:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880802.430796:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880802.430798:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880802.430799:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000072dc05f6. 00000100:00000001:1.0:1788880802.430800:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880802.430801:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000487fae38 00000400:00000200:3.0:1788880802.430804:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880802.430806:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880802.430806:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000487fae38. 00000400:00000200:3.0:1788880802.430808:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880802.430809:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880802.430813:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880802.430813:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000400:00000010:0.0:1788880802.430814:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000048e2c403. 00000400:00000200:3.0:1788880802.430814:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880802.430818:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880802.430820:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880818 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880802.430821:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000fbbca8a. 00000100:00000001:1.0:1788880802.430822:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880802.430824:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880802.430829:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780686336, portal 10 00000100:00000001:1.0:1788880802.430830:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880802.430833:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880802.430835:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:2.0:1788880802.430838:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880818 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880802.430839:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849363200 : -126822860188416 : ffff8ca7c0781500) 00000100:00000001:2.0:1788880802.430847:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880802.430850:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880802.430851:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:3.0:1788880802.430851:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880802.430859:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880802.430864:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [435:435:255:4294967295] 192.168.204.111@tcp LPNI seq info [435:435:7:4294967295] 00000100:00000001:0.0:1788880802.430875:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880802.430878:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880802.430879:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000200:0.0:1788880802.430883:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780686336, offset 0 00000400:00000200:0.0:1788880802.430889:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880802.430892:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880802.430898:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880802.430903:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000400:00000200:0.0:1788880802.430906:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880802.430910:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:3.0:1788880802.430911:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880802.430915:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880802.430921:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880802.430922:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880802.430927:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788880802.430927:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880802.430934:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [436:436:255:4294967295] 192.168.204.111@tcp LPNI seq info [436:436:7:4294967295] 00000100:00000001:3.0:1788880802.430942:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880802.430946:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788880802.430948:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 198765 usecs (199 jiffies) 00000800:00000200:0.0:1788880802.430954:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880802.430958:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ae4abc27. 00000800:00000200:0.0:1788880802.430962:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880802.430967:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880802.430970:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ae4abc27 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880802.430980:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880802.430984:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13966 usecs (14 jiffies) 00000800:00000200:1.0:1788880802.430987:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880802.430988:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306433550 00000800:00000010:1.0:1788880802.430991:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:0.0:1788880802.430993:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306438607 00000400:00000200:1.0:1788880802.430995:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880802.430997:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306433550 00000400:00000200:1.0:1788880802.431001:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880802.431003:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306438607 00000400:00000200:1.0:1788880802.431005:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b6d0d6b5 00000100:00000001:0.0:1788880802.431007:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306433550 00000100:00000001:0.0:1788880802.431010:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306438607 00000400:00000010:1.0:1788880802.431014:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b6d0d6b5. 00000100:00000001:0.0:1788880802.431015:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306433550 00000100:00000001:1.0:1788880802.431018:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880802.431020:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306438607 00000100:00000200:1.0:1788880802.431022:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880802.431042:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880802.431044:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880802.431048:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880802.431048:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880802.431053:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880802.431058:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880802.431059:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880802.431061:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880802.431072:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880802.431075:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ae4abc27. 00000100:00000001:0.0:1788880802.431077:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788880802.431078:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880802.431079:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:1.0:1788880802.431083:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880802.431084:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880802.431086:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000100:00000001:0.0:1788880802.431087:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:1.0:1788880802.431089:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788880802.431091:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880802.431092:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880802.431095:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880802.431101:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880802.431107:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880802.431108:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880802.431112:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880802.431113:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880802.431126:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306433735 00000100:00000001:3.0:1788880802.431130:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306438607 00000100:00000001:3.0:1788880802.431132:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306433735 00000100:00000001:3.0:1788880802.431134:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306438607 00000100:00000001:3.0:1788880802.431199:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306433735 00000100:00000001:3.0:1788880802.431202:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306438607 00000100:00000001:3.0:1788880802.431205:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306433735 00000100:00000001:3.0:1788880802.431207:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306438607 00000100:00000001:3.0:1788880802.431209:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880802.431212:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880802.431213:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880802.431215:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880802.431218:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880802.431218:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880802.431220:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880802.431221:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880802.431221:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880802.431222:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880802.432365:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880802.432373:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880802.432376:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880802.432378:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880802.432385:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880802.432395:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0d300 00000400:00000200:0.0:1788880802.432403:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xe65 [1] + 224 00000800:00000001:0.0:1788880802.432409:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880802.432437:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880802.432440:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880802.432443:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880802.432447:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880802.432451:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880802.432459:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880802.432474:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880802.432480:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880802.432483:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880802.432485:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880802.432487:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880802.432492:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880802.432498:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d380 00000100:00000001:1.0:1788880802.432505:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880802.432509:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880802.432512:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe6d [1] + 224 00000800:00000001:0.0:1788880802.432518:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880802.432519:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880802.432522:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:0.0:1788880802.432524:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880802.432524:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:0.0:1788880802.432529:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880802.432529:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000400:00000010:1.0:1788880802.432533:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000400:00000200:0.0:1788880802.432535:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880802.432537:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880802.432540:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880802.432544:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880802.432548:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880802.432569:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880802.432574:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880802.432581:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880802.432584:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880802.432588:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880802.432591:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880802.432595:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880802.432597:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880802.432599:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880802.432600:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880802.432606:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:1.0:1788880802.432608:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=30 v=5 (5 5 5 5) 00000800:00000001:0.0:1788880802.432611:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880802.432613:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=30 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880802.432617:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880802.432619:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880802.432619:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:1.0:1788880802.432623:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880802.432626:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880802.432626:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880802.432631:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880802.432632:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880802.432635:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880802.432636:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000162a13e6. 00000100:00000040:1.0:1788880802.432639:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880802.432640:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000400:00000010:3.0:1788880802.432642:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000001:3.0:1788880802.432646:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880802.432651:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880802.432658:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880802.432675:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880802.432678:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880802.432679:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880802.432690:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880802.432697:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780686080:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880802.432702:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880802.432704:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880802.432706:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000802a821 x1875776780686080/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880802.432713:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880802.432715:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880802.432717:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004d9329ab. 02000000:00000001:1.0:1788880802.432722:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880802.432724:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880802.432725:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880802.432728:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880802.432730:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000005f53e16f. 02000000:00000001:1.0:1788880802.432734:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880802.432736:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880802.432740:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000010:1.0:1788880802.432743:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000802a821. 00000100:00000001:3.0:1788880802.432743:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880802.432747:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880802.432749:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880802.432750:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880802.432751:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880802.432754:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880802.432757:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880802.432758:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880802.432762:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880802.432765:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880802.432766:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880802.432768:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880802.432770:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880802.432771:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880802.432779:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880802.432781:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880802.432784:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880802.432789:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:3.0:1788880802.432789:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=20 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880802.432796:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=20 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880802.432797:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880802.432798:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880802.432800:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:2.0:1788880802.432803:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d400 00010000:00000001:3.0:1788880802.432803:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880802.432806:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880802.432809:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880802.432811:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880802.432813:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880802.432814:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe75 [1] + 224 00000100:00000040:3.0:1788880802.432818:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880802.432821:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880802.432826:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880802.432828:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000800:00000001:2.0:1788880802.432829:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880802.432830:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880802.432835:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:3.0:1788880802.432836:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880802.432844:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880802.432848:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880802.432853:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880802.432869:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780686208:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880802.432873:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880802.432880:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880802.432885:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b8846eac x1875776780686208/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880802.432888:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880802.432906:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880802.432909:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880802.432911:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e0fe5497. 02000000:00000001:3.0:1788880802.432913:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880802.432914:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880802.432915:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880802.432916:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880802.432917:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000050c5542b. 02000000:00000001:3.0:1788880802.432919:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880802.432920:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:3.0:1788880802.432920:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880802.432923:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b8846eac. 00000100:00000001:3.0:1788880802.432925:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880802.432926:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880802.432927:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880802.432928:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880802.432930:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000001:2.0:1788880802.432933:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880802.432934:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880802.432939:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880802.432941:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880802.432955:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880802.432958:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880802.432961:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880802.432963:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880802.432964:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000048e2c403. 00000400:00000200:0.0:1788880802.432968:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000fbbca8a 00000400:00000010:0.0:1788880802.432969:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000fbbca8a. 00000100:00000001:0.0:1788880802.432971:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880802.432975:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880802.432982:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880802.432989:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880802.432990:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880802.432991:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880802.432992:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880802.432993:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880802.432995:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880802.432996:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880802.432999:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=30 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880802.433007:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=137 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880802.433009:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880802.433010:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880802.433012:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880802.433015:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880802.433021:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880802.433034:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880802.433035:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880802.433037:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880802.433038:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880802.433041:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880802.433047:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880802.433049:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880802.433050:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880802.433052:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880802.433059:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006f591845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780686336:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880802.433062:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880802.433064:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880802.433066:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006f591845 x1875776780686336/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880818 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880802.433071:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880802.433073:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880802.433075:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000072dc05f6. 02000000:00000001:0.0:1788880802.433077:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880802.433078:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880802.433079:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880802.433082:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880802.433083:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b7399401. 02000000:00000001:0.0:1788880802.433089:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880802.433091:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880802.433092:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006f591845. 00000100:00000001:0.0:1788880802.433094:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880802.433095:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880802.433097:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880802.433100:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880802.433100:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880802.433102:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880802.433103:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880802.494335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880802.494340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880802.494344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880802.494345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880802.494387:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880802.494391:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880802.494393:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880803.518339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880803.518362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880803.518369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880803.518370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880803.518441:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880803.518444:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880803.518454:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880804.542351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880804.542358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880804.542364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880804.542365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880804.542458:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880804.542466:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880804.542470:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880805.566383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880805.566393:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880805.566395:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880805.566398:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880805.566403:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880805.566406:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880805.566407:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880805.566409:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880805.566413:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880805.566413:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880805.566468:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880805.566473:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880805.566476:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880806.590364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880806.590372:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880806.590380:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880806.590381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880806.590437:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880806.590442:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880806.590445:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880807.550376:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306438607, at 4306438656 00000100:00000040:1.0:1788880807.550377:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306438607, at 4306438656 00000100:00000040:0.0:1788880807.550383:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306438607, at 4306438656 00000100:00000010:1.0:1788880807.550393:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000010:2.0:1788880807.550395:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000040:3.0:1788880807.550395:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306438607, at 4306438656 00000100:00000010:0.0:1788880807.550399:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:3.0:1788880807.550414:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000040:1.0:1788880807.550430:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880807.550437:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880807.550445:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002dcbd5d0. 00000020:00000040:1.0:1788880807.550450:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880807.550454:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880807.550457:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880807.550461:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880807.550469:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:1.0:1788880807.550476:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880807.550486:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880807.550496:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880807.550510:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880807.550520:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002dcbd5d0] to pc [ptlrpcd_00_00+0] req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880807.550550:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880807.550552:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880807.550555:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880807.550557:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880807.550559:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880807.550561:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880807.550562:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880807.550565:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880807.550569:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:1.0:1788880807.550572:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880807.550575:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880807.550576:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880807.550580:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:2.0:1788880807.550581:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:1.0:1788880807.550583:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880807.550584:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880807.550586:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880807.550587:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880807.550593:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000008b25f66] to pc [ptlrpcd_00_01+1] req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880807.550596:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880807.550600:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880807.550604:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880807.550606:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880807.550609:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880807.550609:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880807.550613:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880807.550617:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880807.550618:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:1.0:1788880807.550621:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001cf64abe. 00000100:00000001:3.0:1788880807.550621:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880807.550625:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:1.0:1788880807.550627:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880807.550631:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880807.550634:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880807.550637:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880807.550642:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:1.0:1788880807.550646:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880807.550650:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880807.550660:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880807.550664:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880807.550670:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880807.550671:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880807.550673:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880807.550673:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880807.550678:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001cf64abe] to pc [ptlrpcd_00_02+2] req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880807.550679:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880807.550686:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780686592:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880807.550701:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880807.550707:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880807.550709:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880807.550709:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880807.550714:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880807.550715:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880807.550720:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000010:2.0:1788880807.550720:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ab1d1f2f. 02000000:00000001:2.0:1788880807.550725:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880807.550727:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11776) 00000400:00000010:2.0:1788880807.550731:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ded9c590. 00000100:00000001:0.0:1788880807.550732:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880807.550735:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880807.550737:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:2.0:1788880807.550740:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008f790e4a. 00000100:00000040:0.0:1788880807.550743:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880807.550747:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780686592, portal 10 00000100:00000001:1.0:1788880807.550748:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880807.550751:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880807.550754:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880807.550754:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880807.550759:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880807.550759:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 02000000:00000001:0.0:1788880807.550763:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880807.550766:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880807.550767:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880807.550771:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780686720:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880807.550771:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880807.550775:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880807.550779:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880807.550781:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880807.550784:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880807.550788:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:0.0:1788880807.550789:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003a17641d. 02000000:00000001:0.0:1788880807.550792:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880807.550794:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880807.550795:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000048e2c403. 00000400:00000010:2.0:1788880807.550798:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001c5b8d0f. 00000400:00000010:0.0:1788880807.550800:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000fbbca8a. 00000100:00000200:2.0:1788880807.550802:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780686592, offset 0 00000100:00000200:0.0:1788880807.550804:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780686720, portal 10 00000100:00000001:0.0:1788880807.550807:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880807.550808:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938054144 : -126823771497472 : ffff8ca78a269c00) 00000400:00000200:2.0:1788880807.550811:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880807.550814:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880807.550825:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:2.0:1788880807.550828:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:0.0:1788880807.550832:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880807.550834:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000096cf0c4a. 00000100:00000200:0.0:1788880807.550837:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780686720, offset 0 00000400:00000200:2.0:1788880807.550837:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880807.550845:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880807.550846:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880807.550852:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880807.550855:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880807.550858:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [437:437:256:4294967295] 192.168.204.111@tcp LPNI seq info [437:437:8:4294967295] 00000400:00000200:2.0:1788880807.550867:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880807.550872:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880807.550876:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880807.550879:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000400:00000200:0.0:1788880807.550881:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880807.550886:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880807.550889:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880807.550892:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880807.550897:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788880807.550900:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880807.550901:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [438:438:255:4294967295] 192.168.204.111@tcp LPNI seq info [438:438:7:4294967295] 00000800:00000200:2.0:1788880807.550907:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880807.550913:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880807.550917:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880807.550920:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b7399401. 00000100:00000001:2.0:1788880807.550923:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880807.550924:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880807.550942:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880807.550945:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b7399401 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:2.0:1788880807.550946:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 197268 usecs (198 jiffies) 00000100:00000001:0.0:1788880807.550949:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880807.550952:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306438854 00000100:00000040:0.0:1788880807.550953:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13880 usecs (14 jiffies) 00000100:00000001:0.0:1788880807.550958:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306438670 00000100:00000010:2.0:1788880807.550958:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ccadc63b. 00000100:00000001:2.0:1788880807.550964:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000100:00000001:2.0:1788880807.550968:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306438854 00000100:00000001:0.0:1788880807.550969:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306438854 00000100:00000001:0.0:1788880807.550973:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306438670 00000100:00000010:2.0:1788880807.550974:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000247083fe. 00000800:00000200:1.0:1788880807.550977:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880807.550978:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000800:00000010:1.0:1788880807.550982:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000100:00000001:0.0:1788880807.550984:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306438854 00000100:00000001:2.0:1788880807.550984:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306438854 00000400:00000200:1.0:1788880807.550986:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880807.550991:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306438670 00000100:00000010:2.0:1788880807.550991:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000668fd8a. 00000400:00000200:1.0:1788880807.550995:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880807.550997:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000400:00000200:1.0:1788880807.551000:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001c5b8d0f 00000100:00000001:2.0:1788880807.551001:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306438854 00000100:00000001:0.0:1788880807.551002:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306438854 00000400:00000010:1.0:1788880807.551003:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001c5b8d0f. 00000100:00000001:0.0:1788880807.551007:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306438670 00000100:00000010:2.0:1788880807.551008:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f6d3b065. 00000100:00000001:1.0:1788880807.551010:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880807.551013:0:201286:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000200:1.0:1788880807.551014:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880807.551018:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880807.551019:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306438854 00000100:00000001:2.0:1788880807.551024:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880807.551026:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880807.551031:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880807.551031:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880807.551031:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880807.551036:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880807.551039:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880807.551041:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880807.551045:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880807.551050:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880807.551052:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880807.551053:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880807.551057:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880807.551058:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880807.551061:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880807.551064:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880807.551064:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880807.551068:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880807.551069:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880807.551074:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880807.551076:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880807.551078:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880807.551081:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780686464:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880807.551085:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880807.551087:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880807.551089:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880807.551090:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880807.551093:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000046cece22. 02000000:00000001:2.0:1788880807.551095:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880807.551103:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000017fdcf5e. 00000400:00000010:2.0:1788880807.551106:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000841b943f. 00000800:00000200:1.0:1788880807.551109:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000200:2.0:1788880807.551109:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780686464, portal 25 00000100:00000001:2.0:1788880807.551113:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000010:1.0:1788880807.551115:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b7399401. 00000100:00000001:2.0:1788880807.551115:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938047872 : -126823771503744 : ffff8ca78a268380) 00000400:00000200:1.0:1788880807.551121:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:2.0:1788880807.551122:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880823 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880807.551130:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880807.551134:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880807.551136:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000096cf0c4a 00000400:00000010:1.0:1788880807.551140:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000096cf0c4a. 00000100:00000040:2.0:1788880807.551140:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880807.551142:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000005509e00. 00000100:00000001:1.0:1788880807.551150:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880807.551151:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780686464, offset 0 00000100:00000200:1.0:1788880807.551154:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880807.551162:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788880807.551162:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880807.551167:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880807.551170:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880807.551177:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880807.551178:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880807.551180:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:2.0:1788880807.551183:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880807.551185:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880807.551187:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880807.551190:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [439:439:255:4294967295] 192.168.204.111@tcp LPNI seq info [439:439:7:4294967295] 00000400:00000200:2.0:1788880807.551196:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880807.551200:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880807.551203:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e53a8f34. 00000800:00000200:2.0:1788880807.551207:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880807.551211:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880807.551214:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e53a8f34 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880807.551223:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880807.551225:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306443727 00000100:00000001:2.0:1788880807.551229:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306438854 00000100:00000001:2.0:1788880807.551251:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306443727 00000100:00000001:2.0:1788880807.551255:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306438854 00000100:00000001:2.0:1788880807.551258:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306443728 00000100:00000001:2.0:1788880807.551266:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306438854 00000800:00000200:0.0:1788880807.551269:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880807.551269:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306443728 00000800:00000010:0.0:1788880807.551275:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e53a8f34. 00000100:00000001:2.0:1788880807.551276:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306438854 00000400:00000200:0.0:1788880807.551280:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880807.551281:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880807.551286:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880807.551289:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880807.551290:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880807.551292:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880807.551296:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000005509e00 00000100:00000001:2.0:1788880807.551298:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:0.0:1788880807.551300:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000005509e00. 00000100:00000001:2.0:1788880807.551302:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880807.551304:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880807.551306:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:0.0:1788880807.551310:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880823 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880807.551310:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880807.551312:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880807.551316:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880807.551324:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880807.551327:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880823 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880807.551334:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880807.551335:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880807.553084:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880807.553093:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880807.553096:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880807.553097:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880807.553102:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880807.553110:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d500 00000400:00000200:2.0:1788880807.553115:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe7d [1] + 224 00000800:00000001:2.0:1788880807.553119:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880807.553121:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880807.553122:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880807.553124:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880807.553127:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880807.553130:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880807.553137:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880807.553148:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880807.553153:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880807.553155:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880807.553160:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880807.553162:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880807.553165:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880807.553170:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d580 00000400:00000200:2.0:1788880807.553173:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe85 [1] + 224 00000800:00000001:2.0:1788880807.553175:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880807.553178:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880807.553179:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880807.553179:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880807.553182:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880807.553184:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:1.0:1788880807.553187:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880807.553190:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880807.553191:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ded9c590. 00000400:00000200:1.0:1788880807.553194:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008f790e4a 00000400:00000010:1.0:1788880807.553195:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008f790e4a. 00000100:00000001:1.0:1788880807.553197:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880807.553198:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880807.553201:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880807.553204:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880807.553224:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880807.553230:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880807.553247:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880807.553253:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880807.553257:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880807.553259:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880807.553260:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880807.553262:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880807.553265:0:201286:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788880807.553265:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880807.553267:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880807.553268:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880807.553271:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880807.553275:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880807.553276:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880807.553279:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880807.553281:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880807.553282:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880807.553286:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788880807.553286:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0d480 00000100:00001000:1.0:1788880807.553289:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=25 v=5 (5 5 5 5) 00000400:00000010:0.0:1788880807.553290:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000048e2c403. 00000100:00001000:1.0:1788880807.553294:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=25 v=5 (1 1 1 1) 00000400:00000200:0.0:1788880807.553296:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000fbbca8a 00000100:00000001:1.0:1788880807.553297:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000010:0.0:1788880807.553299:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000fbbca8a. 00000100:00000001:1.0:1788880807.553300:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880807.553303:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880807.553305:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:1.0:1788880807.553306:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880807.553309:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xe8d [1] + 224 00000100:00000001:1.0:1788880807.553310:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880807.553314:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:2.0:1788880807.553316:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880807.553319:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880807.553322:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000800:00000001:2.0:1788880807.553323:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880807.553325:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880807.553327:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:1.0:1788880807.553334:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880807.553335:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880807.553340:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880807.553344:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880807.553347:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880807.553347:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880807.553348:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880807.553350:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880807.553357:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880807.553371:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880807.553378:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880807.553381:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1788880807.553384:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780686592:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880807.553386:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880807.553389:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880807.553392:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880807.553394:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880807.553400:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880807.553407:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=35 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880807.553407:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:0.0:1788880807.553413:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=142 v=5 (1 1 0 0) 00000100:00000001:0.0:1788880807.553416:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880807.553418:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880807.553418:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880807.553418:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:0.0:1788880807.553420:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000001:1.0:1788880807.553422:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:2.0:1788880807.553423:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880807.553424:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000017fdcf5e. 00010000:00080000:0.0:1788880807.553428:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880807.553428:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000841b943f 00000400:00000010:1.0:1788880807.553431:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000841b943f. 00000100:00000001:1.0:1788880807.553436:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880807.553441:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880807.553442:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880807.553447:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880807.553449:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000200:1.0:1788880807.553450:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880807.553454:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880807.553457:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880807.553458:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880807.553461:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880807.553462:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880807.553463:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:0.0:1788880807.553464:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880807.553464:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880807.553467:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880807.553470:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880807.553474:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=35 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880807.553478:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=35 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880807.553482:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880807.553484:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880807.553485:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788880807.553486:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880807.553489:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880807.553491:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880807.553493:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880807.553495:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880807.553497:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880807.553500:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880807.553512:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780686720:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880807.553517:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880807.553519:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880807.553522:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880807.553522:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880807.553526:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880807.553527:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000001cf64abe x1875776780686720/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880807.553534:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880807.553557:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780686464:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880807.553560:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880807.553561:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880807.553565:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780686592/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880807.553567:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880807.553571:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880807.553572:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880807.553574:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880807.553575:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003a17641d. 02000000:00000010:1.0:1788880807.553576:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ab1d1f2f. 02000000:00000001:0.0:1788880807.553581:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:1.0:1788880807.553582:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880807.553585:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000001:1.0:1788880807.553585:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880807.553588:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000040:0.0:1788880807.553589:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880807.553590:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880807.553593:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 00000020:00000001:0.0:1788880807.553594:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880807.553597:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 02000000:00000001:1.0:1788880807.553597:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880807.553600:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:0.0:1788880807.553602:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000010:1.0:1788880807.553602:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:1.0:1788880807.553604:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:0.0:1788880807.553605:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880807.553607:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:0.0:1788880807.553608:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000001cf64abe. 00000100:00000001:1.0:1788880807.553609:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880807.553612:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002dcbd5d0 x1875776780686464/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880823 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880807.553614:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880807.553617:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880807.553622:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880807.553622:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880807.553624:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880807.553627:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000046cece22. 00000100:00000001:0.0:1788880807.553630:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880807.553632:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880807.553634:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880807.553636:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:1.0:1788880807.553637:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880807.553639:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880807.553641:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880807.553641:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880807.553643:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 02000000:00000001:1.0:1788880807.553646:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880807.553647:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880807.553647:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002dcbd5d0. 00000100:00000001:1.0:1788880807.553648:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880807.553649:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880807.553650:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880807.553653:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880807.553654:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880807.553656:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880807.553657:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880807.614351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880807.614357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880807.614363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880807.614366:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880807.614368:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880807.614371:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880807.614374:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880807.614376:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880807.614378:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880807.614380:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880807.614443:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880807.614447:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880807.614450:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880808.638328:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880808.638335:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880808.638340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880808.638341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880808.638398:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880808.638402:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880808.638405:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880809.662349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880809.662356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880809.662362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880809.662363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880809.662417:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880809.662421:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880809.662424:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880810.686302:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880810.686308:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880810.686313:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880810.686314:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880810.686366:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880810.686369:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880810.686372:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880811.710367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880811.710398:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880811.710406:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880811.710407:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880811.710455:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880811.710459:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880811.710462:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880812.670341:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306443728, at 4306443776 00000100:00000040:3.0:1788880812.670356:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306443728, at 4306443776 00000100:00000010:2.0:1788880812.670358:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000668fd8a. 00000100:00000010:3.0:1788880812.670372:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f6d3b065. 00000100:00000040:0.0:1788880812.670381:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306443727, at 4306443776 00000100:00000010:0.0:1788880812.670391:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ccadc63b. 00000100:00000040:1.0:1788880812.670515:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306443727, at 4306443776 00000100:00000010:1.0:1788880812.670524:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000247083fe. 00000100:00000040:2.0:1788880812.670525:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880812.670530:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880812.670537:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000020:00000040:2.0:1788880812.670541:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880812.670546:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880812.670549:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880812.670553:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880812.670561:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 00000100:00000001:2.0:1788880812.670568:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880812.670578:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880812.670590:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880812.670592:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880812.670598:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_03+3] req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880812.670627:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880812.670630:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880812.670634:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880812.670637:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000078d5a15d. 00000100:00000001:1.0:1788880812.670653:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880812.670653:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880812.670660:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880812.670660:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880812.670662:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880812.670662:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880812.670665:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880812.670667:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880812.670668:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880812.670674:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 00000100:00000040:1.0:1788880812.670681:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880812.670682:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880812.670691:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880812.670727:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880812.670727:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880812.670729:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880812.670733:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880812.670736:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000078d5a15d] to pc [ptlrpcd_00_00+0] req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880812.670744:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880812.670749:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880812.670753:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880812.670757:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780686848:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880812.670763:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880812.670764:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880812.670767:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880812.670770:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1788880812.670772:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:2.0:1788880812.670772:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880812.670778:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880812.670779:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880812.670784:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880812.670787:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880812.670788:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880812.670789:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788880812.670792:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880812.670792:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880812.670796:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880812.670797:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008e4ae4dd. 00000100:00000040:3.0:1788880812.670800:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880812.670802:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880812.670806:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880812.670808:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880812.670809:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 00000100:00000001:3.0:1788880812.670822:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:1.0:1788880812.670823:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000001:3.0:1788880812.670824:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880812.670825:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880812.670842:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880812.670846:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:3.0:1788880812.670846:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788880812.670848:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 02000000:00000001:3.0:1788880812.670853:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880812.670862:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780686848, portal 25 00000100:00000001:2.0:1788880812.670863:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:3.0:1788880812.670863:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780686976:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880812.670866:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880812.670872:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880812.670875:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880812.670878:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_00_01+1] req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880812.670880:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 02000000:00000001:3.0:1788880812.670884:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880812.670892:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880812.670897:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880812.670899:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880828 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880812.670901:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880812.670903:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000001f6835d8. 02000000:00000001:3.0:1788880812.670911:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880812.670912:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880812.670922:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880812.670926:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000162a13e6. 00000100:00000001:0.0:1788880812.670931:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880812.670935:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880812.670939:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:3.0:1788880812.670942:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000040:2.0:1788880812.670943:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11781) 00000100:00000040:0.0:1788880812.670953:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880812.670953:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:3.0:1788880812.670955:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780686976, portal 10 00000100:00000001:3.0:1788880812.670962:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:1.0:1788880812.670965:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880812.670973:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938426496 : -126823771125120 : ffff8ca78a2c4a80) 00000400:00000010:1.0:1788880812.670974:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000487fae38. 00000100:00000001:0.0:1788880812.670976:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880812.670982:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880812.670983:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780686848, offset 0 02000000:00000001:0.0:1788880812.670987:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1788880812.670989:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880812.670992:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880812.670996:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880812.671002:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780687104:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880812.671003:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880812.671018:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880812.671019:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880812.671023:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880812.671027:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880812.671027:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880812.671031:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:3.0:1788880812.671034:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b6d0d6b5. 02000000:00000010:0.0:1788880812.671035:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000edcbc44b. 00000100:00000200:3.0:1788880812.671039:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780686976, offset 0 00000400:00000200:1.0:1788880812.671040:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880812.671041:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880812.671054:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000048e2c403. 00000400:00000200:3.0:1788880812.671055:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880812.671056:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880812.671070:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:0.0:1788880812.671074:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000fbbca8a. 00000400:00000200:1.0:1788880812.671076:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:0.0:1788880812.671084:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780687104, portal 10 00000400:00000200:1.0:1788880812.671086:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880812.671094:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [440:440:256:4294967295] 192.168.204.111@tcp LPNI seq info [440:440:8:4294967295] 00000100:00000001:0.0:1788880812.671105:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880812.671109:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000400:00000200:1.0:1788880812.671115:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880812.671122:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880812.671128:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880812.671131:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880812.671141:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000749ed09e. 00000400:00000200:3.0:1788880812.671144:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880812.671150:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788880812.671156:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:0.0:1788880812.671161:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880812.671164:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880812.671165:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:0.0:1788880812.671167:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ff62fd67. 00000100:00000200:0.0:1788880812.671177:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780687104, offset 0 00000400:00000200:3.0:1788880812.671178:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880812.671186:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880812.671188:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880812.671191:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [441:441:255:4294967295] 192.168.204.111@tcp LPNI seq info [441:441:7:4294967295] 00000800:00000200:1.0:1788880812.671199:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000749ed09e type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880812.671213:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880812.671221:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880812.671225:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880812.671228:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880812.671229:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000400:00000200:0.0:1788880812.671231:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880812.671266:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880812.671268:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880812.671269:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306448848 00000400:00000200:0.0:1788880812.671276:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:1.0:1788880812.671281:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000800:00000200:3.0:1788880812.671282:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880812.671286:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788880812.671292:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880812.671293:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880812.671299:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [442:442:254:4294967295] 192.168.204.111@tcp LPNI seq info [442:442:6:4294967295] 00000100:00000001:3.0:1788880812.671303:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880812.671306:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306448848 00000100:00000040:3.0:1788880812.671309:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 195800 usecs (196 jiffies) 00000800:00000200:2.0:1788880812.671311:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:1.0:1788880812.671314:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000100:00000001:3.0:1788880812.671316:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306443973 00000800:00000010:2.0:1788880812.671319:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000749ed09e. 00000100:00000001:1.0:1788880812.671323:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880812.671326:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306448848 00000400:00000200:2.0:1788880812.671328:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880812.671330:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880812.671333:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306448848 00000100:00000001:3.0:1788880812.671335:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306443973 00000400:00000200:2.0:1788880812.671346:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000010:1.0:1788880812.671348:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000100:00000001:3.0:1788880812.671348:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306448848 00000800:00000200:0.0:1788880812.671357:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880812.671357:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000487fae38 00000100:00000001:3.0:1788880812.671358:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306443973 00000100:00000001:1.0:1788880812.671360:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000010:2.0:1788880812.671363:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000487fae38. 00000800:00000010:0.0:1788880812.671370:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000f9f1055. 00000100:00000001:1.0:1788880812.671371:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306448848 00000100:00000001:2.0:1788880812.671372:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880812.671372:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306448848 00000800:00000200:0.0:1788880812.671381:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:1.0:1788880812.671381:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000100:00000001:3.0:1788880812.671381:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306443973 00000100:00000200:2.0:1788880812.671382:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880828 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880812.671392:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:0.0:1788880812.671400:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880812.671404:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880812.671404:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306448848 00000800:00000200:0.0:1788880812.671405:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000f9f1055 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880812.671415:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880812.671421:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880812.671422:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880812.671423:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880812.671424:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880812.671433:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13798 usecs (14 jiffies) 00000100:00000001:1.0:1788880812.671433:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880812.671434:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880828 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880812.671435:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880812.671444:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306443791 00000100:00000001:1.0:1788880812.671444:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880812.671445:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880812.671451:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880812.671457:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880812.671458:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306448848 00000100:00000001:0.0:1788880812.671461:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306443791 00000100:00000001:0.0:1788880812.671463:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306448848 00000100:00000001:0.0:1788880812.671465:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306443791 00000100:00000001:1.0:1788880812.671465:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880812.671466:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880812.671470:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306448848 00000100:00000001:3.0:1788880812.671471:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:2.0:1788880812.671472:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880812.671473:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306443791 00000100:00000001:0.0:1788880812.671480:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306448848 00000100:00000001:1.0:1788880812.671480:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:2.0:1788880812.671480:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000100:00000001:3.0:1788880812.671481:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880812.671487:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880812.671492:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880812.671493:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880812.671494:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880812.671495:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880812.671500:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880812.671500:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880812.671501:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880812.671507:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880812.671509:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880812.671511:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880812.671511:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b6d0d6b5 00000400:00000010:2.0:1788880812.671517:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b6d0d6b5. 00000100:00000001:0.0:1788880812.671520:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880812.671523:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880812.671523:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880812.671526:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:2.0:1788880812.671531:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880812.671532:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880812.671536:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880812.671539:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880812.671541:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880812.671543:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880812.671544:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880812.671549:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880812.671553:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880812.671554:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880812.671603:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880812.671606:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000f9f1055. 00000400:00000200:2.0:1788880812.671615:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880812.671619:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880812.671621:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ff62fd67 00000400:00000010:2.0:1788880812.671623:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ff62fd67. 00000100:00000001:2.0:1788880812.671626:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880812.671629:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880812.671635:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880812.671637:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880812.671642:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880812.671643:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880812.673330:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880812.673342:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880812.673345:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880812.673347:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880812.673354:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880812.673367:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0d600 00000400:00000200:0.0:1788880812.673376:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xe95 [1] + 224 00000800:00000001:0.0:1788880812.673382:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880812.673385:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880812.673387:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880812.673390:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880812.673393:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880812.673404:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880812.673411:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880812.673426:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880812.673431:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880812.673433:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880812.673434:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880812.673435:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880812.673438:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880812.673441:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d680 00000400:00000200:0.0:1788880812.673445:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xe99 [1] + 224 00000800:00000001:0.0:1788880812.673448:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880812.673450:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880812.673451:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880812.673454:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880812.673456:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880812.673458:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880812.673464:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880812.673476:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880812.673479:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:0.0:1788880812.673480:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880812.673481:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880812.673484:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880812.673485:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880812.673487:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880812.673492:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:1.0:1788880812.673495:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:0.0:1788880812.673499:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d700 00000400:00000001:1.0:1788880812.673499:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880812.673505:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000100:00000001:3.0:1788880812.673507:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880812.673516:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880812.673518:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000400:00000010:1.0:1788880812.673523:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:1.0:1788880812.673528:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:0.0:1788880812.673533:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xea5 [1] + 224 00000800:00000001:0.0:1788880812.673538:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1788880812.673538:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880812.673541:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000200:1.0:1788880812.673542:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880812.673545:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000162a13e6. 00000800:00000001:0.0:1788880812.673547:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880812.673552:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000400:00000200:0.0:1788880812.673558:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:3.0:1788880812.673559:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000001:3.0:1788880812.673571:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:0.0:1788880812.673572:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:3.0:1788880812.673581:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880812.673583:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880812.673591:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880812.673611:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880812.673612:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880812.673642:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880812.673678:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880812.673680:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880812.673685:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880812.673686:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880812.673694:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880812.673695:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788880812.673696:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880812.673700:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:3.0:1788880812.673701:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:0.0:1788880812.673702:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880812.673705:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880812.673706:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:0.0:1788880812.673707:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880812.673711:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880812.673712:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880812.673722:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880812.673722:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880812.673728:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880812.673732:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880812.673735:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880812.673736:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:1.0:1788880812.673737:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=40 v=5 (5 5 5 5) 00000400:00000010:2.0:1788880812.673738:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000048e2c403. 00000100:00001000:3.0:1788880812.673738:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=30 v=5 (5 5 5 5) 00000400:00000200:2.0:1788880812.673751:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000fbbca8a 00000100:00001000:3.0:1788880812.673751:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=30 v=5 (1 1 1 1) 00000400:00000010:2.0:1788880812.673756:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000fbbca8a. 00000100:00000001:3.0:1788880812.673756:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880812.673758:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880812.673760:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=40 v=5 (1 1 1 1) 00010000:00000001:3.0:1788880812.673762:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880812.673763:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880812.673766:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880812.673772:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880812.673772:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880812.673773:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880812.673779:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880812.673781:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00010000:00000001:1.0:1788880812.673786:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880812.673786:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880812.673793:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880812.673797:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880812.673798:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880812.673798:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880812.673813:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880812.673817:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880812.673822:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880812.673822:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880812.673823:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880812.673842:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880812.673846:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880812.673850:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880812.673857:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880812.673871:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=40 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880812.673882:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=147 v=5 (1 1 0 0) 00000100:00000001:2.0:1788880812.673896:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880812.673901:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880812.673903:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880812.673904:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880812.673911:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00010000:00000001:2.0:1788880812.673911:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880812.673912:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880812.673917:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880812.673918:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00010000:00080000:2.0:1788880812.673919:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880812.673926:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880812.673927:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880812.673934:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880812.673938:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880812.673943:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000040:1.0:1788880812.673946:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880812.673946:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880812.673956:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880812.673956:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780686976:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880812.673975:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880812.673983:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880812.673997:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000078d5a15d x1875776780686976/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880812.674016:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880812.674024:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00100000:1.0:1788880812.674025:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780686848:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880812.674027:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880812.674038:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880812.674040:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880812.674046:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880812.674049:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1788880812.674050:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000001f6835d8. 00000100:00100000:2.0:1788880812.674056:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780687104:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880812.674057:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:3.0:1788880812.674058:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880812.674066:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880812.674066:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:1.0:1788880812.674071:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780686848/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:3.0:1788880812.674072:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880812.674073:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880812.674079:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780687104/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880828 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880812.674084:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880812.674090:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 02000000:00000001:3.0:1788880812.674100:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880812.674108:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880812.674108:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788880812.674109:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880812.674117:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:2.0:1788880812.674117:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000010:3.0:1788880812.674117:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000078d5a15d. 02000000:00000010:1.0:1788880812.674125:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008e4ae4dd. 02000000:00000010:2.0:1788880812.674126:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000edcbc44b. 00000100:00000001:3.0:1788880812.674127:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880812.674132:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880812.674138:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880812.674142:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:2.0:1788880812.674143:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880812.674145:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000020:00000001:1.0:1788880812.674153:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880812.674153:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880812.674154:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880812.674160:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000040:2.0:1788880812.674161:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880812.674169:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:1.0:1788880812.674172:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000020:00000001:2.0:1788880812.674173:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880812.674173:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880812.674181:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 02000000:00000010:2.0:1788880812.674181:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 02000000:00000001:2.0:1788880812.674192:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880812.674193:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880812.674193:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 02000000:00000001:1.0:1788880812.674198:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880812.674200:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:1.0:1788880812.674202:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788880812.674203:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:1.0:1788880812.674206:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000001:2.0:1788880812.674207:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880812.674217:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880812.674218:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880812.674220:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880812.674222:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880812.674223:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880812.674226:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880812.674229:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880812.674276:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880812.674278:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880812.674280:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880812.674282:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880812.734364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880812.734373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880812.734380:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880812.734381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880812.734446:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880812.734453:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880812.734457:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880813.758356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880813.758365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880813.758372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880813.758373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880813.758425:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880813.758431:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880813.758434:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880814.782333:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880814.782338:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880814.782358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880814.782359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880814.782415:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880814.782419:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880814.782422:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880815.806320:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880815.806329:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880815.806332:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880815.806335:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880815.806340:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880815.806343:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880815.806345:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880815.806347:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880815.806351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880815.806353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880815.806394:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880815.806397:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880815.806405:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880816.830334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880816.830339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880816.830343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880816.830344:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880816.830395:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880816.830399:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880816.830401:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880817.790308:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306448848, at 4306448896 00000100:00000040:3.0:1788880817.790316:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306448848, at 4306448896 00000100:00000040:0.0:1788880817.790319:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306448848, at 4306448896 00000100:00000040:2.0:1788880817.790323:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306448848, at 4306448896 00000100:00000010:1.0:1788880817.790327:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:0.0:1788880817.790328:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000010:2.0:1788880817.790330:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000010:3.0:1788880817.790330:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000040:2.0:1788880817.790345:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880817.790349:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880817.790358:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880817.790362:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880817.790366:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880817.790368:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880817.790373:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880817.790380:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 00000100:00000001:2.0:1788880817.790387:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880817.790396:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880817.790404:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880817.790405:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880817.790409:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_00_02+2] req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.790426:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880817.790438:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880817.790441:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880817.790443:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000020:00000040:2.0:1788880817.790445:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880817.790447:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880817.790449:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880817.790449:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880817.790451:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880817.790454:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008eccda10. 00000100:00000001:2.0:1788880817.790456:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880817.790458:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880817.790460:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880817.790461:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880817.790462:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880817.790465:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880817.790471:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880817.790473:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880817.790474:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880817.790478:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d838cab4] to pc [ptlrpcd_00_03+3] req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.790498:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880817.790501:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880817.790506:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880817.790507:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000010:2.0:1788880817.790508:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000020:00000040:2.0:1788880817.790511:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880817.790515:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880817.790515:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880817.790518:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880817.790520:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880817.790521:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880817.790522:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880817.790524:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880817.790527:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000010:2.0:1788880817.790527:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5d1e8ed. 00000100:00000001:1.0:1788880817.790528:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880817.790534:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880817.790534:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880817.790535:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880817.790543:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780687232:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880817.790545:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880817.790555:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880817.790561:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:0.0:1788880817.790562:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880817.790563:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880817.790564:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880817.790565:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880817.790566:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880817.790569:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880817.790572:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880817.790572:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fd6b7ee1] to pc [ptlrpcd_00_00+0] req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880817.790574:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000010:0.0:1788880817.790577:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a58a4768. 02000000:00000001:1.0:1788880817.790580:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880817.790584:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880817.790588:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780687360:192.168.204.111@tcp:400: 00000400:00000010:0.0:1788880817.790593:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006f1b1f6b. 00000100:00000001:1.0:1788880817.790593:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880817.790597:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880817.790601:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880817.790602:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 02000000:00000001:1.0:1788880817.790605:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:0.0:1788880817.790610:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780687232, portal 25 02000000:00000010:1.0:1788880817.790617:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000784e3b9. 00000100:00000001:2.0:1788880817.790618:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880817.790621:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880817.790623:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880817.790625:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880817.790628:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000400:00000010:1.0:1788880817.790629:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000040:2.0:1788880817.790630:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880817.790631:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880817.790635:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:1.0:1788880817.790637:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000040:2.0:1788880817.790637:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11786) 00000100:00000001:3.0:1788880817.790637:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:1.0:1788880817.790643:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780687360, portal 10 00000100:00000040:3.0:1788880817.790644:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.790647:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880817.790648:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880817.790650:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880817.790652:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000100:00000040:0.0:1788880817.790654:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880833 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880817.790660:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880817.790663:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880817.790666:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880817.790669:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880817.790672:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880817.790676:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880817.790678:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880817.790684:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00100000:3.0:1788880817.790686:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780687488:192.168.204.111@tcp:400: 00000400:00000010:0.0:1788880817.790689:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000001:1.0:1788880817.790689:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:0.0:1788880817.790695:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780687232, offset 0 00000100:00000001:3.0:1788880817.790695:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880817.790698:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880817.790698:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880817.790701:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880817.790703:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000400:00000200:0.0:1788880817.790704:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880817.790704:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880817.790710:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000051bfba0. 00000100:00000200:1.0:1788880817.790711:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780687360, offset 0 02000000:00000001:3.0:1788880817.790714:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880817.790719:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880817.790721:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880817.790725:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:3.0:1788880817.790725:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000162a13e6. 00000400:00000200:0.0:1788880817.790741:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:3.0:1788880817.790741:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000200:3.0:1788880817.790747:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780687488, portal 10 00000400:00000200:0.0:1788880817.790748:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880817.790752:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880817.790755:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880817.790758:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000400:00000200:0.0:1788880817.790759:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [443:443:256:4294967295] 192.168.204.111@tcp LPNI seq info [443:443:8:4294967295] 00000100:00000040:3.0:1788880817.790767:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880817.790780:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880817.790788:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880817.790792:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788880817.790798:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000001c4b7888. 00000400:00000200:1.0:1788880817.790801:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880817.790804:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880817.790807:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788880817.790810:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880817.790815:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:3.0:1788880817.790815:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000800:00000200:0.0:1788880817.790819:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:3.0:1788880817.790821:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780687488, offset 0 00000400:00000200:1.0:1788880817.790822:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880817.790831:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [444:444:255:4294967295] 192.168.204.111@tcp LPNI seq info [444:444:7:4294967295] 00000400:00000200:3.0:1788880817.790832:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880817.790838:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880817.790844:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000001c4b7888 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880817.790845:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880817.790853:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880817.790857:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880817.790858:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000100:00000001:0.0:1788880817.790860:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880817.790866:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880817.790867:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306453967 00000400:00000200:3.0:1788880817.790868:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:0.0:1788880817.790875:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000da3f50d1. 00000400:00000200:3.0:1788880817.790876:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880817.790878:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880817.790883:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:1.0:1788880817.790884:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880817.790884:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880817.790891:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306453967 00000100:00000001:1.0:1788880817.790891:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880817.790892:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788880817.790897:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 194355 usecs (195 jiffies) 00000100:00000010:0.0:1788880817.790898:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000007d3948ef. 00000400:00000200:3.0:1788880817.790899:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [445:445:254:4294967295] 192.168.204.111@tcp LPNI seq info [445:445:6:4294967295] 00000100:00000001:0.0:1788880817.790904:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880817.790904:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306449091 00000100:00000001:1.0:1788880817.790909:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306453967 00000100:00000001:0.0:1788880817.790910:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306453967 00000100:00000010:0.0:1788880817.790916:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000975d5882. 00000400:00000200:3.0:1788880817.790918:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880817.790919:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880817.790921:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306449091 00000800:00000010:2.0:1788880817.790925:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000001c4b7888. 00000100:00000001:1.0:1788880817.790927:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306453967 00000100:00000001:0.0:1788880817.790928:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:2.0:1788880817.790931:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:3.0:1788880817.790932:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880817.790934:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306453967 00000100:00000001:1.0:1788880817.790934:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306449091 00000800:00000010:3.0:1788880817.790940:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000100:00000001:1.0:1788880817.790941:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306453967 00000100:00000010:0.0:1788880817.790942:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001d590ced. 00000400:00000200:2.0:1788880817.790942:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880817.790947:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306449091 00000100:00000001:0.0:1788880817.790948:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:2.0:1788880817.790950:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000800:00000200:3.0:1788880817.790952:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:2.0:1788880817.790953:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:0.0:1788880817.790954:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880817.790955:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306453967 00000100:00000001:0.0:1788880817.790959:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880817.790961:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880817.790961:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:3.0:1788880817.790964:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880817.790965:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880817.790966:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880817.790969:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880817.790969:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880833 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880817.790971:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880817.790972:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880817.790977:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880817.790977:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880817.790979:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880817.790980:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880817.790985:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880817.790986:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880817.790986:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13721 usecs (14 jiffies) 00000100:00000001:2.0:1788880817.790987:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880817.790989:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:2.0:1788880817.790992:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880833 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880817.790993:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306448910 00000100:00000001:1.0:1788880817.790994:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880817.790997:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880817.790998:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306453967 00000100:00000001:0.0:1788880817.790999:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880817.791000:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880817.791002:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.791005:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306448910 00000100:00000001:1.0:1788880817.791006:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880817.791009:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.791012:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306453967 00000100:00000001:2.0:1788880817.791014:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.791017:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306448910 00000100:00000001:2.0:1788880817.791018:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880817.791022:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306453967 00000100:00000001:3.0:1788880817.791024:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306448910 00000100:00000001:3.0:1788880817.791026:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306453967 00000800:00000200:2.0:1788880817.791032:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880817.791036:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:2.0:1788880817.791037:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000100:00000001:3.0:1788880817.791040:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880817.791043:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880817.791043:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.791047:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880817.791051:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880817.791051:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880817.791053:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880817.791057:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880817.791058:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000100:00000001:3.0:1788880817.791058:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880817.791061:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880817.791062:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:3.0:1788880817.791063:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880817.791066:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880817.791070:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.791076:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880817.791083:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.791088:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880817.791089:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880817.791145:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880817.791147:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:2.0:1788880817.791150:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880817.791154:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880817.791156:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000400:00000010:2.0:1788880817.791157:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:2.0:1788880817.791160:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880817.791162:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.791169:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880817.791172:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.791178:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880817.791179:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880817.793478:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880817.793488:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880817.793491:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880817.793492:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880817.793498:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880817.793508:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d880 00000400:00000200:2.0:1788880817.793513:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xebd [1] + 224 00000800:00000001:2.0:1788880817.793517:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880817.793519:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880817.793521:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880817.793523:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880817.793527:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880817.793531:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880817.793538:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.793553:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880817.793559:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880817.793561:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880817.793563:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880817.793565:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880817.793567:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880817.793571:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0d780 00000400:00000200:2.0:1788880817.793575:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xead [1] + 224 00000800:00000001:2.0:1788880817.793578:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880817.793578:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880817.793581:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880817.793583:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880817.793584:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880817.793587:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:3.0:1788880817.793591:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880817.793594:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880817.793595:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000162a13e6. 00000400:00000200:3.0:1788880817.793599:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000400:00000010:3.0:1788880817.793601:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000001:2.0:1788880817.793605:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880817.793605:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880817.793610:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880817.793611:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880817.793628:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880817.793635:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.793641:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880817.793646:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880817.793647:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880817.793649:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880817.793652:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880817.793653:0:201285:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788880817.793656:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788880817.793658:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788880817.793659:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880817.793662:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880817.793666:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880817.793667:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 02000000:00000001:3.0:1788880817.793671:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880817.793676:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d800 00000100:00001000:3.0:1788880817.793679:0:201285:0:(import.c:2149:obd_at_measure()) add 0 to 0000000028cb2d55 time=45 v=5 (5 5 5 5) 00000100:00000001:0.0:1788880817.793682:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880817.793683:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xeb1 [1] + 224 00000100:00000001:0.0:1788880817.793685:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:3.0:1788880817.793687:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=152 v=5 (1 1 0 0) 00000800:00000001:2.0:1788880817.793688:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880817.793689:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880817.793692:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880817.793692:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000010:0.0:1788880817.793696:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006f1b1f6b. 00000800:00000001:2.0:1788880817.793696:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880817.793696:0:201285:0:(client.c:1493:ptlrpc_check_status()) Process leaving (rc=18446744073709551509 : -107 : ffffffffffffff95) 00000400:00000200:0.0:1788880817.793700:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000400:00000200:2.0:1788880817.793701:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880817.793702:0:201285:0:(recover.c:202:ptlrpc_request_handle_notconn()) Process entered 00000400:00000010:0.0:1788880817.793703:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000400:00000200:2.0:1788880817.793705:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880817.793707:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00080000:3.0:1788880817.793709:0:201285:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0001-mdc-ffff8ca787527000 of lustre-MDT0001_UUID@192.168.204.111@tcp abruptly disconnected: reconnecting 00000100:00000001:2.0:1788880817.793711:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880817.793713:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880817.793719:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788880817.793724:0:201285:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from FULL to DISCONN 00000100:00000200:0.0:1788880817.793740:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880817.793741:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:02000400:3.0:1788880817.793747:0:201285:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0001-mdc-ffff8ca787527000: Connection to lustre-MDT0001 (at 192.168.204.111@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00000001:0.0:1788880817.793753:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880817.793755:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880817.793758:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880817.793760:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880817.793760:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880817.793762:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880817.793764:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880817.793766:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880817.793766:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880817.793769:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=45 v=5 (5 5 5 5) 00000800:00000001:2.0:1788880817.793769:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880817.793773:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=45 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880817.793775:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880817.793781:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880817.793784:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880817.793786:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880817.793788:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880817.793793:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880817.793800:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880817.793802:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880817.793803:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880817.793807:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880817.793809:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880817.793812:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880817.793815:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880817.793818:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880817.793821:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000100:00100000:0.0:1788880817.793827:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780687232:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880817.793831:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000100:00000001:0.0:1788880817.793833:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880817.793835:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:0.0:1788880817.793837:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880817.793841:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:0.0:1788880817.793842:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780687232/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880817.793849:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880817.793851:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880817.793854:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880817.793856:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a58a4768. 02000000:00000001:0.0:1788880817.793861:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880817.793864:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000200:1.0:1788880817.793865:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1788880817.793866:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880817.793870:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880817.793873:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 02000000:00000001:0.0:1788880817.793877:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880817.793878:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788880817.793879:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880817.793882:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:1.0:1788880817.793882:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880817.793885:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880817.793886:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788880817.793887:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880817.793890:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880817.793891:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880817.793894:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880817.793896:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880817.793897:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880817.793900:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880817.793901:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880817.793903:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880817.793906:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880817.793913:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=35 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880817.793916:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=35 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880817.793919:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880817.793920:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880817.793922:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880817.793924:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880817.793927:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880817.793928:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880817.793930:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880817.793931:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880817.793935:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880817.793941:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880817.793943:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880817.793944:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880817.793947:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880817.793953:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780687360:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880817.793956:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880817.793958:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880817.793960:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d838cab4 x1875776780687360/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880817.793965:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880817.793966:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880817.793968:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000784e3b9. 02000000:00000001:1.0:1788880817.793970:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880817.793971:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880817.793972:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880817.793974:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880817.793975:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008eccda10. 02000000:00000001:1.0:1788880817.793983:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880817.793984:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880817.793985:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d838cab4. 00000100:00000001:1.0:1788880817.793988:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880817.793988:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880817.793990:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880817.793993:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880817.793994:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880817.793996:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880817.793998:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.798286:0:201285:0:(obd_class.h:1223:obd_import_event()) Process entered 00000002:00000001:3.0:1788880817.798292:0:201285:0:(mdc_request.c:2879:mdc_import_event()) Process entered 00000002:00000001:3.0:1788880817.798294:0:201285:0:(mdc_request.c:2953:mdc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.798296:0:201285:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00000001:3.0:1788880817.798298:0:201285:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:3.0:1788880817.798300:0:201285:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:3.0:1788880817.798304:0:201285:0:(import.c:485:import_select_connection()) Process entered 00000100:00080000:3.0:1788880817.798308:0:201285:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 11479 00000400:00000200:3.0:1788880817.798316:0:201285:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000100:00000040:3.0:1788880817.798319:0:201285:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:3.0:1788880817.798321:0:201285:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788880817.798322:0:201285:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000001:3.0:1788880817.798324:0:201285:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000001:3.0:1788880817.798326:0:201285:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:3.0:1788880817.798327:0:201285:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:3.0:1788880817.798329:0:201285:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880817.798332:0:201285:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 00000020:00000001:3.0:1788880817.798333:0:201285:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000020:00000001:3.0:1788880817.798335:0:201285:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000100:00000040:3.0:1788880817.798336:0:201285:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:3.0:1788880817.798337:0:201285:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788880817.798338:0:201285:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000001:3.0:1788880817.798339:0:201285:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000040:3.0:1788880817.798341:0:201285:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 00000100:00080000:3.0:1788880817.798343:0:201285:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: import 00000000b98b8a4e using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000100:00000001:3.0:1788880817.798344:0:201285:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880817.798346:0:201285:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:3.0:1788880817.798363:0:201285:0:(sec.c:1628:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880817.798365:0:201285:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.798366:0:201285:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:3.0:1788880817.798369:0:201285:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 2097152, lost: 0. 00000008:00000001:3.0:1788880817.798371:0:201285:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.798372:0:201285:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880817.798376:0:201285:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000020:00000040:3.0:1788880817.798378:0:201285:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=5 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:3.0:1788880817.798381:0:201285:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 02000000:00000001:3.0:1788880817.798382:0:201285:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.798384:0:201285:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880817.798386:0:201285:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880817.798389:0:201285:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880817.798393:0:201285:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 000000002979e919. 00000100:00000001:3.0:1788880817.798398:0:201285:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880817.798403:0:201285:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880817.798410:0:201285:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880817.798412:0:201285:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880817.798414:0:201285:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_rcv+-1] req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880817.798430:0:201285:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.798432:0:201285:0:(recover.c:221:ptlrpc_request_handle_notconn()) Process leaving 00000100:00000001:3.0:1788880817.798433:0:201285:0:(client.c:1701:after_reply()) Process leaving (rc=18446744073709551509 : -107 : ffffffffffffff95) 00000100:00000040:3.0:1788880817.798437:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Rpc:RNQU/200/0 rc -107/-107 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880817.798442:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551509 : -107 : 0xffffffffffffff95) 00000100:00000001:3.0:1788880817.798444:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880817.798445:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880817.798447:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Interpret:RNQU/200/0 rc -107/-107 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880817.798459:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000fd6b7ee1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780687488:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880817.798463:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880817.798465:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880817.798467:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880817.798467:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000fd6b7ee1 x1875776780687488/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880833 ref 1 fl Complete:RNQU/200/0 rc -107/-107 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.798472:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880817.798473:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880817.798474:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880817.798476:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880817.798478:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000051bfba0. 00000100:00000040:2.0:1788880817.798481:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880817.798484:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880817.798486:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880817.798488:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880817.798491:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880817.798494:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000010:3.0:1788880817.798495:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5d1e8ed. 00000100:00000001:2.0:1788880817.798497:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880817.798499:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880817.798500:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880817.798501:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880817.798503:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000fd6b7ee1. 02000000:00000001:2.0:1788880817.798504:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880817.798506:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:2.0:1788880817.798508:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.798508:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880817.798513:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:2.0:1788880817.798516:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780687616:192.168.204.111@tcp:38: 00000100:00000001:3.0:1788880817.798518:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880817.798521:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880817.798521:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880817.798525:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880817.798527:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880817.798527:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880817.798531:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880817.798535:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880817.798539:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000008d40d8dd. 02000000:00000001:2.0:1788880817.798542:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880817.798547:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001f454358. 00000400:00000010:2.0:1788880817.798553:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c56cc487. 00000100:00000200:2.0:1788880817.798558:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780687616, portal 10 00000100:00000001:2.0:1788880817.798560:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880817.798561:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000040:2.0:1788880817.798566:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880827 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.798571:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880817.798578:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880817.798580:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000793778c8. 00000100:00000200:2.0:1788880817.798581:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780687616, offset 0 00000400:00000200:2.0:1788880817.798585:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880817.798593:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880817.798597:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880817.798600:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880817.798602:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880817.798604:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880817.798606:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [446:446:256:4294967295] 192.168.204.111@tcp LPNI seq info [446:446:8:4294967295] 00000400:00000200:2.0:1788880817.798614:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880817.798618:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880817.798621:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c52725cf. 00000800:00000200:2.0:1788880817.798625:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880817.798631:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880817.798634:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c52725cf type 1, nob 616 niov 1 nkiov 1 00000100:00000001:2.0:1788880817.798646:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880817.798649:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13645 usecs (14 jiffies) 00000100:00000001:2.0:1788880817.798653:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306448918 00000100:00000001:2.0:1788880817.798655:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306453967 00000100:00000001:2.0:1788880817.798657:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306448918 00000100:00000001:2.0:1788880817.798658:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306453967 00000100:00000001:2.0:1788880817.798661:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306448918 00000100:00000001:2.0:1788880817.798662:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306453967 00000100:00000001:2.0:1788880817.798664:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306448918 00000100:00000001:2.0:1788880817.798666:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306453967 00000100:00000001:2.0:1788880817.798667:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880817.798670:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880817.798677:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880817.798679:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880817.798683:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880817.798684:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:2.0:1788880817.798686:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880817.798686:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880817.798687:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880817.798689:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880817.798764:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880817.798768:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c52725cf. 00000400:00000200:0.0:1788880817.798772:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880817.798776:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880817.798780:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000793778c8 00000400:00000010:0.0:1788880817.798782:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000793778c8. 00000100:00000001:0.0:1788880817.798786:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880817.798789:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880827 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880817.798796:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880817.798798:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880827 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880817.798814:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880817.798815:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880817.803208:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880817.803225:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880817.803229:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880817.803254:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880817.803265:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880817.803278:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0d900 00000400:00000200:2.0:1788880817.803285:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xec5 [1] + 0 00000800:00000001:2.0:1788880817.803291:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880817.803296:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880817.803297:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880817.803300:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880817.803305:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880817.803310:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880827 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880817.803318:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880827 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880817.803333:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880817.803338:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880817.803340:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880817.803369:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880817.803373:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880817.803380:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880817.803381:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880817.803383:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001f454358. 00000400:00000200:1.0:1788880817.803386:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c56cc487 00000400:00000010:1.0:1788880817.803388:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c56cc487. 00000100:00000001:1.0:1788880817.803390:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880817.803393:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880827 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880817.803399:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880827 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880817.803403:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880817.803404:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880817.803406:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880817.803408:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880817.803409:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880817.803411:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880817.803413:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880817.803420:0:201284:0:(import.c:2149:obd_at_measure()) add 0 to 0000000028cb2d55 time=45 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880817.803422:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=2 v=5 (1 1 1 0) 00000100:00000001:1.0:1788880817.803424:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880817.803425:0:201284:0:(client.c:1493:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:1.0:1788880817.803427:0:201284:0:(client.c:1698:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:1.0:1788880817.803429:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880827 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880817.803447:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:1.0:1788880817.803449:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880817.803451:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880817.803452:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:1.0:1788880817.803453:0:201284:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:1.0:1788880817.803455:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788880817.803458:0:201284:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.111@tcp failed (-19) 00000100:00000001:1.0:1788880817.803460:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:1.0:1788880817.803462:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880827 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880817.803466:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780687616:192.168.204.111@tcp:38: 00000100:00000001:1.0:1788880817.803470:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880817.803471:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880817.803473:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780687616/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880827 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880817.803476:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880817.803478:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880817.803480:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000008d40d8dd. 02000000:00000001:1.0:1788880817.803482:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880817.803484:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880817.803485:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880817.803487:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880817.803489:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 000000002979e919. 02000000:00000001:1.0:1788880817.803493:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880817.803494:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880817.803496:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:1.0:1788880817.803498:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880817.803499:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880817.803501:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880817.803507:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880817.803514:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880817.803517:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880817.803518:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880817.854353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880817.854359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880817.854365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880817.854368:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880817.854370:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880817.854373:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880817.854376:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880817.854378:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880817.854380:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880817.854382:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880817.854445:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880817.854448:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880817.854450:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880818.878341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880818.878349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880818.878355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880818.878356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880818.878413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880818.878417:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880818.878420:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880819.902336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880819.902344:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880819.902350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880819.902351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880819.902403:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880819.902406:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880819.902408:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880820.926317:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880820.926323:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880820.926328:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880820.926328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880820.926381:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880820.926386:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880820.926388:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880821.950349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880821.950356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880821.950362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880821.950363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880821.950432:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880821.950438:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880821.950442:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880822.910341:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306453967, at 4306454016 00000100:00000040:1.0:1788880822.910342:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306453967, at 4306454016 00000100:00000010:0.0:1788880822.910359:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000da3f50d1. 00000100:00000040:2.0:1788880822.910361:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306453967, at 4306454016 00000100:00000010:1.0:1788880822.910362:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000007d3948ef. 00000100:00000040:3.0:1788880822.910372:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306453967, at 4306454016 00000100:00000010:2.0:1788880822.910384:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000975d5882. 00000100:00000010:3.0:1788880822.910401:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001d590ced. 00000100:00000040:1.0:1788880822.910404:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880822.910410:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880822.910418:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002dcbd5d0. 00000020:00000040:1.0:1788880822.910424:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880822.910429:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880822.910438:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880822.910442:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880822.910449:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:1.0:1788880822.910455:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880822.910463:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880822.910470:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880822.910472:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880822.910476:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002dcbd5d0] to pc [ptlrpcd_00_01+1] req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880822.910495:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880822.910496:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880822.910498:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880822.910500:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880822.910501:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880822.910503:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880822.910504:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880822.910505:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880822.910508:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:1.0:1788880822.910510:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880822.910513:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880822.910518:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880822.910520:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880822.910524:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880822.910525:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000008b25f66] to pc [ptlrpcd_00_02+2] req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880822.910529:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880822.910532:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:2.0:1788880822.910534:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880822.910539:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880822.910553:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880822.910556:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880822.910559:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880822.910562:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880822.910565:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880822.910567:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:1.0:1788880822.910571:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880822.910576:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788880822.910577:0:200894:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00000001:1.0:1788880822.910583:0:200894:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00000001:2.0:1788880822.910583:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:2.0:1788880822.910591:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:0.0:1788880822.910593:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00080000:1.0:1788880822.910594:0:200894:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:0.0:1788880822.910604:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880822.910604:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880822.910608:0:200894:0:(import.c:485:import_select_connection()) Process entered 00000100:00080000:1.0:1788880822.910619:0:200894:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 11781 00000100:00000001:3.0:1788880822.910621:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880822.910624:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880822.910629:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880822.910629:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880822.910644:0:200894:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 02000000:00000001:3.0:1788880822.910647:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880822.910651:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880822.910652:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:1.0:1788880822.910655:0:200894:0:(import.c:2149:obd_at_measure()) add 10 to 0000000040f48a4f time=7 v=5 (1 1 1 0) 02000000:00000001:3.0:1788880822.910660:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880822.910661:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:1.0:1788880822.910663:0:200894:0:(import.c:2215:obd_at_measure()) AT 0000000040f48a4f change: old=5 new=10 delta=5 (val=10) hist 10 1 1 0 00000100:00080000:1.0:1788880822.910670:0:200894:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: tried all connections, increasing latency to 5s 00000100:00000001:2.0:1788880822.910670:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00100000:3.0:1788880822.910671:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780687744:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880822.910682:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880822.910683:0:200894:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880822.910689:0:200894:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788880822.910689:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880822.910692:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:1.0:1788880822.910697:0:200894:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 02000000:00000001:3.0:1788880822.910698:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880822.910710:0:200894:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 02000000:00000001:3.0:1788880822.910712:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880822.910714:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000020:00000001:1.0:1788880822.910720:0:200894:0:(genops.c:813:class_conn2export()) Process entered 00000100:00000001:2.0:1788880822.910721:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880822.910721:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000014eb60d4. 00000020:00000040:1.0:1788880822.910726:0:200894:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:1.0:1788880822.910730:0:200894:0:(lustre_handles.c:129:class_handle2object()) Process entered 02000000:00000001:3.0:1788880822.910732:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880822.910733:0:200894:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 02000000:00000001:2.0:1788880822.910733:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000020:00000001:1.0:1788880822.910741:0:200894:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 02000000:00000001:2.0:1788880822.910742:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788880822.910744:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000162a13e6. 00000020:00000001:1.0:1788880822.910747:0:200894:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 02000000:00000001:2.0:1788880822.910752:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880822.910753:0:200894:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000400:00000010:3.0:1788880822.910756:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000001:1.0:1788880822.910760:0:200894:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00100000:2.0:1788880822.910763:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780687872:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880822.910764:0:200894:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000200:3.0:1788880822.910766:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780687744, portal 25 00000100:00000001:1.0:1788880822.910769:0:200894:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000001:3.0:1788880822.910776:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000020:00000040:1.0:1788880822.910777:0:200894:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 00000100:00000001:2.0:1788880822.910778:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880822.910779:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938047872 : -126823771503744 : ffff8ca78a268380) 00000100:00080000:1.0:1788880822.910787:0:200894:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: import 00000000b98b8a4e using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000100:00000040:3.0:1788880822.910791:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880838 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880822.910795:0:200894:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880822.910801:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880822.910807:0:200894:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:2.0:1788880822.910807:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880822.910815:0:200894:0:(sec.c:1628:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880822.910821:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880822.910825:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880822.910826:0:200894:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880822.910831:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000087acc4de. 00000100:00000040:3.0:1788880822.910834:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880822.910836:0:200894:0:(obd_class.h:889:obd_reconnect()) Process entered 02000000:00000001:2.0:1788880822.910840:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880822.910840:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 00000008:00100000:1.0:1788880822.910842:0:200894:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 2097152, lost: 0. 00000100:00000200:3.0:1788880822.910846:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780687744, offset 0 00000008:00000001:1.0:1788880822.910850:0:200894:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880822.910852:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8697e0a. 00000100:00000001:1.0:1788880822.910862:0:200894:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880822.910864:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d208f26. 00000400:00000200:3.0:1788880822.910869:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880822.910870:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000070db75b2. 00000020:00000040:1.0:1788880822.910877:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000200:2.0:1788880822.910879:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780687872, portal 10 02000000:00000001:1.0:1788880822.910881:0:200894:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 00000100:00000001:2.0:1788880822.910883:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1788880822.910884:0:200894:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880822.910888:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880822.910890:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000001:1.0:1788880822.910893:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880822.910896:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000200:3.0:1788880822.910899:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880822.910901:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880822.910906:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880838 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880822.910906:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000010:1.0:1788880822.910908:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 00000000b278c5d9. 00000400:00000200:3.0:1788880822.910913:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880822.910916:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880822.910923:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00100000:1.0:1788880822.910934:0:200894:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:3.0:1788880822.910935:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [447:447:256:4294967295] 192.168.204.111@tcp LPNI seq info [447:447:8:4294967295] 00000100:00000001:2.0:1788880822.910938:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880822.910952:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880822.910962:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:2.0:1788880822.910967:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000170fdf47. 00000100:00000200:2.0:1788880822.910977:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780687872, offset 0 00000800:00000200:3.0:1788880822.910980:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880822.910988:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000800:00000010:3.0:1788880822.910989:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:1.0:1788880822.910990:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880822.910997:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880822.910999:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000070db75b2] to pc [ptlrpcd_rcv+-1] req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880822.911009:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880822.911015:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880822.911022:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880822.911024:0:200894:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880822.911029:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:1.0:1788880822.911030:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:2.0:1788880822.911035:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880822.911039:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000040:1.0:1788880822.911040:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:2.0:1788880822.911044:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880822.911045:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880822.911048:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11791) 00000100:00000001:0.0:1788880822.911052:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880822.911055:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880822.911058:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:0.0:1788880822.911061:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880822.911068:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880822.911069:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [448:448:255:4294967295] 192.168.204.111@tcp LPNI seq info [448:448:7:4294967295] 00000100:00000001:0.0:1788880822.911081:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880822.911082:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306459087 00000100:00000001:0.0:1788880822.911086:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880822.911090:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b0b8cfdf. 02000000:00000001:0.0:1788880822.911091:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:2.0:1788880822.911092:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880822.911096:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880822.911103:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880822.911105:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00100000:0.0:1788880822.911109:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780688000:192.168.204.111@tcp:38: 00000800:00000200:2.0:1788880822.911112:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880822.911114:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306459087 00000100:00000001:0.0:1788880822.911115:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000010:2.0:1788880822.911117:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000007cb951ea. 02000000:00000001:0.0:1788880822.911121:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000010:3.0:1788880822.911127:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c7c25dfc. 02000000:00000001:0.0:1788880822.911128:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880822.911138:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:2.0:1788880822.911138:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880822.911139:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788880822.911150:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306459087 02000000:00000010:0.0:1788880822.911151:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000d8aaf3e3. 02000000:00000001:0.0:1788880822.911160:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880822.911160:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:3.0:1788880822.911161:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002aba003e. 00000800:00000200:2.0:1788880822.911172:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000007cb951ea type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880822.911172:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000010:0.0:1788880822.911173:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006f1b1f6b. 00000100:00000001:3.0:1788880822.911184:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306459087 00000100:00000001:2.0:1788880822.911187:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880822.911190:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000100:00000010:3.0:1788880822.911190:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000026aaaaf2. 00000100:00000040:2.0:1788880822.911192:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 192936 usecs (193 jiffies) 00000100:00000001:3.0:1788880822.911194:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880822.911202:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880822.911203:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306454209 00000100:00000200:0.0:1788880822.911205:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780688000, portal 10 00000100:00000001:3.0:1788880822.911211:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880822.911213:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880822.911213:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306459087 00000100:00000001:0.0:1788880822.911223:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938046976 : -126823771504640 : ffff8ca78a268000) 00000100:00000001:2.0:1788880822.911226:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306454209 00000100:00000001:3.0:1788880822.911227:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880822.911230:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306459087 00000100:00000001:3.0:1788880822.911268:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880822.911272:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880837 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880822.911277:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306454210 00000100:00000001:0.0:1788880822.911282:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880822.911283:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306459087 00000100:00000040:0.0:1788880822.911287:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880822.911288:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306454210 00000400:00000010:0.0:1788880822.911292:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000dc289a80. 00000100:00000001:2.0:1788880822.911293:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306459087 00000100:00000001:2.0:1788880822.911297:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880822.911299:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780688000, offset 0 00000100:00000001:2.0:1788880822.911299:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880822.911305:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880822.911306:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880822.911312:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880822.911314:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880822.911315:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880822.911318:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880822.911321:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880822.911322:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880822.911324:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [449:449:254:4294967295] 192.168.204.111@tcp LPNI seq info [449:449:6:4294967295] 00000400:00000200:0.0:1788880822.911332:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880822.911339:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880822.911342:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880822.911343:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000001c4b7888. 00000800:00000010:3.0:1788880822.911347:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000800:00000200:0.0:1788880822.911353:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880822.911354:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:0.0:1788880822.911360:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880822.911367:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000200:0.0:1788880822.911368:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000001c4b7888 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:0.0:1788880822.911377:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880822.911379:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13588 usecs (14 jiffies) 00000400:00000200:3.0:1788880822.911379:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 00000400:00000010:3.0:1788880822.911382:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 00000100:00000001:0.0:1788880822.911383:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306454031 00000100:00000001:0.0:1788880822.911389:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306459087 00000100:00000001:3.0:1788880822.911391:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880822.911392:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306454031 00000100:00000200:3.0:1788880822.911396:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880838 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880822.911407:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306459087 00000100:00000001:3.0:1788880822.911413:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880822.911414:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306454031 00000100:00000040:3.0:1788880822.911417:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880838 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880822.911422:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306459087 00000100:00000001:0.0:1788880822.911434:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306454031 00000100:00000001:3.0:1788880822.911434:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880822.911438:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880822.911442:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306459087 00000100:00000001:0.0:1788880822.911445:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880822.911448:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880822.911450:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880822.911451:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880822.911454:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000007cb951ea. 00000400:00000200:3.0:1788880822.911458:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880822.911459:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880822.911464:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880822.911470:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000170fdf47 00000400:00000010:3.0:1788880822.911471:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000170fdf47. 00000100:00000001:0.0:1788880822.911473:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880822.911474:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880822.911478:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000200:3.0:1788880822.911479:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880838 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880822.911493:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880822.911495:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880822.911499:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880838 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880822.911500:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880822.911504:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880822.911516:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880822.911517:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880822.911525:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880822.911594:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880822.911596:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000001c4b7888. 00000400:00000200:3.0:1788880822.911598:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880822.911600:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880822.911607:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000dc289a80 00000400:00000010:3.0:1788880822.911608:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000dc289a80. 00000100:00000001:3.0:1788880822.911610:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880822.911612:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880837 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880822.911617:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880822.911618:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880837 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880822.911622:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880822.911623:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880822.913247:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880822.913262:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880822.913265:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880822.913267:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880822.913276:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880822.913293:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0da00 00000400:00000200:0.0:1788880822.913300:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xed5 [1] + 224 00000800:00000001:0.0:1788880822.913307:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880822.913311:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880822.913313:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880822.913317:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880822.913324:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880822.913330:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880822.913338:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880822.913371:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880822.913380:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880822.913385:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880822.913386:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880822.913388:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880822.913393:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880822.913400:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0d980 00000400:00000200:0.0:1788880822.913405:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xecd [1] + 224 00000800:00000001:0.0:1788880822.913409:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880822.913410:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880822.913414:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880822.913416:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880822.913419:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:2.0:1788880822.913420:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:0.0:1788880822.913425:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:2.0:1788880822.913425:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880822.913429:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8697e0a. 00000400:00000200:2.0:1788880822.913435:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d208f26 00000400:00000010:2.0:1788880822.913437:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d208f26. 00000100:00000001:2.0:1788880822.913440:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880822.913441:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880822.913446:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880822.913448:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880822.913479:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880822.913482:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880822.913499:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880822.913501:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880822.913502:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880822.913505:0:201288:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:0.0:1788880822.913506:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:2.0:1788880822.913508:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:0.0:1788880822.913510:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880822.913512:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880822.913514:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880822.913517:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880822.913525:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=40 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880822.913526:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880822.913529:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:2.0:1788880822.913531:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=40 v=5 (1 1 1 1) 00000400:00000001:3.0:1788880822.913534:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880822.913536:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000001:3.0:1788880822.913537:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880822.913539:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880822.913541:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000162a13e6. 00010000:00000001:2.0:1788880822.913543:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:3.0:1788880822.913546:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00010000:00000001:2.0:1788880822.913548:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880822.913550:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000001:3.0:1788880822.913555:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880822.913559:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880822.913560:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880822.913564:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880822.913567:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880822.913570:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880822.913578:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880822.913579:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880822.913595:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880822.913597:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880822.913599:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880822.913601:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880822.913601:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880822.913605:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880822.913605:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880822.913607:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880822.913608:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880822.913610:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880822.913615:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=50 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880822.913618:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=50 v=5 (1 1 1 1) 00000100:00000040:2.0:1788880822.913620:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880822.913621:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880822.913624:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880822.913629:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00100000:2.0:1788880822.913632:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780687872:192.168.204.111@tcp:400: 00010000:00000001:3.0:1788880822.913644:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880822.913647:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880822.913649:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880822.913651:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780687872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880822.913658:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880822.913659:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880822.913662:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880822.913664:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000087acc4de. 00000100:00000040:3.0:1788880822.913667:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880822.913668:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880822.913671:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880822.913673:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880822.913676:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880822.913679:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 00000100:00000001:3.0:1788880822.913683:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880822.913684:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880822.913686:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880822.913688:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:3.0:1788880822.913689:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880822.913692:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880822.913692:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880822.913694:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880822.913696:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880822.913699:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880822.913700:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880822.913702:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880822.913705:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880822.913707:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880822.913714:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780687744:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880822.913720:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880822.913722:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880822.913725:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002dcbd5d0 x1875776780687744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880838 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880822.913730:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880822.913732:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880822.913734:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000014eb60d4. 02000000:00000001:3.0:1788880822.913736:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880822.913737:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880822.913737:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880822.913739:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880822.913740:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 02000000:00000001:3.0:1788880822.913742:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880822.913742:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880822.913743:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002dcbd5d0. 00000100:00000001:3.0:1788880822.913745:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880822.913746:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880822.913747:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880822.913750:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880822.913750:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880822.913752:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880822.913753:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880822.917757:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880822.917772:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880822.917775:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880822.917777:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880822.917789:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880822.917804:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0da80 00000400:00000200:2.0:1788880822.917821:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xedd [1] + 0 00000800:00000001:2.0:1788880822.917830:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880822.917849:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880822.917852:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880822.917857:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880822.917863:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880822.917869:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880837 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880822.917880:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880837 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880822.917898:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880822.917905:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880822.917908:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880822.917942:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880822.917946:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880822.917952:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880822.917954:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880822.917956:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006f1b1f6b. 00000400:00000200:0.0:1788880822.917960:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000400:00000010:0.0:1788880822.917963:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00000001:0.0:1788880822.917966:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880822.917970:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880837 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880822.917979:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880837 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880822.917991:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880822.917993:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880822.917995:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880822.917998:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880822.918000:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880822.918002:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880822.918005:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880822.918013:0:201284:0:(import.c:2149:obd_at_measure()) add 0 to 0000000028cb2d55 time=50 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880822.918016:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=7 v=10 (10 1 1 0) 00000100:00000001:0.0:1788880822.918019:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880822.918020:0:201284:0:(client.c:1493:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:0.0:1788880822.918022:0:201284:0:(client.c:1698:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:0.0:1788880822.918026:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880837 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880822.918032:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:0.0:1788880822.918035:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880822.918036:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880822.918038:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:0.0:1788880822.918040:0:201284:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:0.0:1788880822.918043:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788880822.918048:0:201284:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.111@tcp failed (-19) 00000100:00000001:0.0:1788880822.918050:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:0.0:1788880822.918053:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880837 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880822.918060:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780688000:192.168.204.111@tcp:38: 00000100:00000001:0.0:1788880822.918064:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880822.918067:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880822.918069:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000070db75b2 x1875776780688000/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880837 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880822.918085:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880822.918088:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880822.918090:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000d8aaf3e3. 02000000:00000001:0.0:1788880822.918094:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880822.918096:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880822.918097:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880822.918100:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880822.918102:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 00000000b278c5d9. 02000000:00000001:0.0:1788880822.918105:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880822.918106:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880822.918107:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000070db75b2. 00000100:00000001:0.0:1788880822.918110:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880822.918111:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880822.918113:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880822.918120:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880822.918121:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880822.918124:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880822.918125:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880822.974329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880822.974334:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880822.974338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880822.974339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880822.974382:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880822.974385:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880822.974387:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880823.998331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880823.998337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880823.998341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880823.998341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880823.998396:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880823.998399:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880823.998401:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880825.022356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880825.022363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880825.022369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880825.022370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880825.022424:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880825.022428:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880825.022430:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880826.046323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880826.046348:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880826.046350:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880826.046353:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880826.046358:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880826.046360:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880826.046362:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880826.046364:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880826.046369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880826.046370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880826.046426:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880826.046430:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880826.046433:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880827.070343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880827.070349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880827.070354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880827.070354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880827.070406:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880827.070409:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880827.070411:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880828.030338:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306459087, at 4306459136 00000100:00000040:1.0:1788880828.030338:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306459087, at 4306459136 00000100:00000040:2.0:1788880828.030338:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306459087, at 4306459136 00000100:00000010:1.0:1788880828.030352:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c7c25dfc. 00000100:00000010:0.0:1788880828.030354:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b0b8cfdf. 00000100:00000010:2.0:1788880828.030355:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002aba003e. 00000100:00000040:1.0:1788880828.030375:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880828.030379:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880828.030384:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000070db75b2. 00000020:00000040:1.0:1788880828.030388:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000040:3.0:1788880828.030390:0:205461:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306459087, at 4306459136 00000100:00000001:1.0:1788880828.030391:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880828.030394:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880828.030399:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880828.030399:0:205461:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000026aaaaf2. 02000000:00000010:1.0:1788880828.030407:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:1.0:1788880828.030413:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880828.030422:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880828.030430:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880828.030432:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880828.030436:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000070db75b2] to pc [ptlrpcd_00_03+3] req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880828.030459:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880828.030461:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880828.030464:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880828.030479:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002dcbd5d0. 00000100:00000001:2.0:1788880828.030480:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:1.0:1788880828.030482:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880828.030484:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880828.030488:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880828.030488:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880828.030489:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880828.030490:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:2.0:1788880828.030493:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880828.030495:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880828.030495:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880828.030500:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:1.0:1788880828.030505:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880828.030507:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880828.030510:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880828.030514:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880828.030515:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880828.030517:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880828.030519:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788880828.030521:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780688128:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880828.030530:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880828.030531:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880828.030545:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002dcbd5d0] to pc [ptlrpcd_00_00+0] req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880828.030552:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880828.030559:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880828.030562:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880828.030565:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880828.030569:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:1.0:1788880828.030571:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:2.0:1788880828.030573:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000087acc4de. 00000100:00080000:1.0:1788880828.030576:0:200894:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 02000000:00000001:2.0:1788880828.030578:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.030579:0:200894:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:1.0:1788880828.030581:0:200894:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:1.0:1788880828.030586:0:200894:0:(import.c:485:import_select_connection()) Process entered 00000400:00000010:2.0:1788880828.030587:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8697e0a. 00000100:00080000:1.0:1788880828.030589:0:200894:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 11786 00000400:00000010:2.0:1788880828.030596:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d208f26. 00000100:00000001:0.0:1788880828.030598:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880828.030599:0:200894:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000100:00000001:0.0:1788880828.030602:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880828.030602:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:1.0:1788880828.030603:0:200894:0:(import.c:2149:obd_at_measure()) add 15 to 0000000040f48a4f time=13 v=10 (10 1 1 0) 00000100:00000200:2.0:1788880828.030604:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780688128, portal 25 00000100:00000001:0.0:1788880828.030606:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880828.030607:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880828.030609:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00001000:1.0:1788880828.030610:0:200894:0:(import.c:2215:obd_at_measure()) AT 0000000040f48a4f change: old=10 new=15 delta=5 (val=15) hist 15 1 1 0 00000100:00000001:2.0:1788880828.030612:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938046976 : -126823771504640 : ffff8ca78a268000) 00000100:00000040:0.0:1788880828.030614:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788880828.030617:0:200894:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: tried all connections, increasing latency to 10s 00000100:00000040:2.0:1788880828.030622:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880844 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880828.030624:0:200894:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880828.030629:0:200894:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:1.0:1788880828.030634:0:200894:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880828.030639:0:200894:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000001:0.0:1788880828.030640:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880828.030643:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880828.030644:0:200894:0:(genops.c:813:class_conn2export()) Process entered 02000000:00000001:0.0:1788880828.030647:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000020:00000040:1.0:1788880828.030647:0:200894:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 02000000:00000001:0.0:1788880828.030649:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880828.030652:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880828.030652:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000020:00000001:1.0:1788880828.030656:0:200894:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000100:00100000:0.0:1788880828.030659:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780688256:192.168.204.111@tcp:400: 00000020:00000040:1.0:1788880828.030660:0:200894:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 00000100:00000040:2.0:1788880828.030660:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000020:00000001:1.0:1788880828.030664:0:200894:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000400:00000010:2.0:1788880828.030665:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009c31f261. 00000100:00000001:0.0:1788880828.030667:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000020:00000001:1.0:1788880828.030668:0:200894:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000100:00000200:2.0:1788880828.030670:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780688128, offset 0 00000100:00000040:1.0:1788880828.030673:0:200894:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880828.030676:0:200894:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000400:00000200:2.0:1788880828.030679:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880828.030680:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:1.0:1788880828.030680:0:200894:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 02000000:00000001:0.0:1788880828.030684:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.030684:0:200894:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 02000000:00000001:0.0:1788880828.030689:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000020:00000040:1.0:1788880828.030690:0:200894:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 00000400:00000200:2.0:1788880828.030691:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000010:0.0:1788880828.030695:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a58a4768. 00000100:00080000:1.0:1788880828.030695:0:200894:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: import 00000000b98b8a4e using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000400:00000200:2.0:1788880828.030698:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788880828.030700:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.030701:0:200894:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880828.030704:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788880828.030705:0:200894:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 00000400:00000010:0.0:1788880828.030706:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006f1b1f6b. 00000400:00000200:2.0:1788880828.030709:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:1.0:1788880828.030711:0:200894:0:(sec.c:1628:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880828.030713:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000400:00000200:2.0:1788880828.030714:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:1.0:1788880828.030715:0:200894:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.030719:0:200894:0:(obd_class.h:889:obd_reconnect()) Process entered 00000100:00000200:0.0:1788880828.030720:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780688256, portal 10 00000400:00000200:2.0:1788880828.030721:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [450:450:256:4294967295] 192.168.204.111@tcp LPNI seq info [450:450:8:4294967295] 00000008:00100000:1.0:1788880828.030723:0:200894:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 2097152, lost: 0. 00000100:00000001:0.0:1788880828.030724:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880828.030727:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938047872 : -126823771503744 : ffff8ca78a268380) 00000008:00000001:1.0:1788880828.030729:0:200894:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.030733:0:200894:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880828.030737:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880844 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880828.030737:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880828.030742:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000200:2.0:1788880828.030742:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:1.0:1788880828.030751:0:200894:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 00000800:00000200:2.0:1788880828.030754:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880828.030755:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880828.030759:0:200894:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880828.030760:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000011e1ab42. 00000100:00000040:0.0:1788880828.030761:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880828.030766:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:0.0:1788880828.030767:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f4d84419. 02000000:00000001:1.0:1788880828.030771:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000200:0.0:1788880828.030772:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780688256, offset 0 00000800:00000200:2.0:1788880828.030772:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:1.0:1788880828.030776:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880828.030783:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000010:1.0:1788880828.030783:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 000000008eac1e29. 00000800:00000200:2.0:1788880828.030783:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880828.030789:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000011e1ab42 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880828.030791:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880828.030793:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880828.030798:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880828.030799:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880828.030800:0:200894:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:0.0:1788880828.030803:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880828.030809:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880828.030810:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306464207 00000400:00000200:0.0:1788880828.030815:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:2.0:1788880828.030815:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003dccfd54. 00000100:00000001:2.0:1788880828.030822:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880828.030823:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [451:451:255:4294967295] 192.168.204.111@tcp LPNI seq info [451:451:7:4294967295] 00000100:00000001:1.0:1788880828.030823:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880828.030827:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880828.030829:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306464207 00000100:00000040:1.0:1788880828.030835:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000008b25f66] to pc [ptlrpcd_rcv+-1] req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788880828.030835:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000668fd8a. 00000400:00000200:0.0:1788880828.030840:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880828.030840:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:2.0:1788880828.030848:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306464207 00000800:00000200:0.0:1788880828.030854:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880828.030856:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005fb7b615. 00000800:00000010:0.0:1788880828.030861:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000f9f1055. 00000100:00000001:2.0:1788880828.030862:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:0.0:1788880828.030868:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880828.030868:0:200894:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880828.030869:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306464207 00000100:00000040:1.0:1788880828.030874:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:2.0:1788880828.030875:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e91f0731. 00000800:00000200:0.0:1788880828.030876:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880828.030881:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:0.0:1788880828.030882:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000f9f1055 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:1.0:1788880828.030882:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000800:00000200:3.0:1788880828.030882:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880828.030887:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880828.030890:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880828.030890:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000011e1ab42. 00000100:00000001:2.0:1788880828.030891:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880828.030892:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11796) 00000100:00000040:0.0:1788880828.030896:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 191535 usecs (192 jiffies) 00000100:00000001:2.0:1788880828.030898:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880828.030898:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880828.030902:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306459328 00000100:00000001:2.0:1788880828.030904:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880828.030910:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880828.030912:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306464207 00000100:00000001:1.0:1788880828.030912:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880828.030913:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880828.030915:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880828.030918:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306459328 00000100:00000001:1.0:1788880828.030918:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880828.030918:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:1.0:1788880828.030924:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880828.030924:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880828.030925:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306464207 00000400:00000200:3.0:1788880828.030925:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009c31f261 00000100:00000001:2.0:1788880828.030929:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880828.030931:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306459328 00000400:00000010:3.0:1788880828.030931:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009c31f261. 00000100:00000001:2.0:1788880828.030932:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880828.030938:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306464207 00000100:00000001:3.0:1788880828.030939:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880828.030940:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880828.030940:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.030943:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880828.030947:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:3.0:1788880828.030947:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880844 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880828.030949:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880828.030951:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880828.030952:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306459328 00000100:00100000:1.0:1788880828.030956:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780688384:192.168.204.111@tcp:38: 00000100:00000001:0.0:1788880828.030957:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306464207 00000100:00000001:0.0:1788880828.030961:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880828.030964:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880828.030965:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880828.030967:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880828.030969:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880828.030970:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880828.030973:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880828.030974:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880828.030978:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000010:1.0:1788880828.030979:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000b278c5d9. 00000100:00000001:0.0:1788880828.030981:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000001:1.0:1788880828.030984:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880828.030985:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880828.030987:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880828.030990:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880828.030991:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000100:00000001:0.0:1788880828.030994:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880828.030998:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000200:1.0:1788880828.031003:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780688384, portal 10 00000100:00000001:1.0:1788880828.031006:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880828.031007:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000001:3.0:1788880828.031007:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880828.031014:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880844 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880828.031015:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880848 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880828.031041:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880828.031043:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880828.031047:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880828.031051:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880828.031058:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000ae22117. 00000100:00000200:1.0:1788880828.031060:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780688384, offset 0 00000400:00000200:1.0:1788880828.031063:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880828.031068:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880828.031070:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880828.031071:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880828.031073:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880828.031075:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880828.031076:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [452:452:255:4294967295] 192.168.204.111@tcp LPNI seq info [452:452:7:4294967295] 00000400:00000200:1.0:1788880828.031084:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880828.031084:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880828.031090:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000f9f1055. 00000800:00000200:1.0:1788880828.031091:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880828.031096:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000400:00000200:3.0:1788880828.031096:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:1.0:1788880828.031100:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880828.031105:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000200:1.0:1788880828.031109:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880828.031112:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 616 niov 1 nkiov 1 00000400:00000200:3.0:1788880828.031112:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f4d84419 00000100:00000001:1.0:1788880828.031117:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880828.031117:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f4d84419. 00000100:00000040:1.0:1788880828.031121:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13552 usecs (14 jiffies) 00000100:00000001:3.0:1788880828.031123:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880828.031124:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306459150 00000100:00000001:1.0:1788880828.031127:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306464207 00000100:00000200:3.0:1788880828.031129:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880844 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880828.031130:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306459150 00000100:00000001:1.0:1788880828.031132:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306464207 00000100:00000001:1.0:1788880828.031135:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306459150 00000100:00000001:1.0:1788880828.031136:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306464207 00000100:00000001:1.0:1788880828.031137:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306459150 00000100:00000001:1.0:1788880828.031138:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306464207 00000100:00000001:1.0:1788880828.031139:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880828.031140:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.031141:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.031145:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880828.031148:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880828.031149:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:1.0:1788880828.031150:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880828.031151:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880828.031152:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.031153:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880828.031177:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880828.031180:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880844 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880828.031198:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880828.031200:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880828.031250:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880828.031253:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000400:00000200:3.0:1788880828.031256:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880828.031260:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880828.031262:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000ae22117 00000400:00000010:3.0:1788880828.031264:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000ae22117. 00000100:00000001:3.0:1788880828.031266:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880828.031269:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880848 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880828.031311:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880828.031314:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880848 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880828.031320:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880828.031322:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880828.032661:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880828.032671:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880828.032674:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880828.032676:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880828.032683:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880828.032692:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0db00 00000400:00000200:0.0:1788880828.032699:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xee5 [1] + 224 00000800:00000001:0.0:1788880828.032704:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880828.032709:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880828.032711:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880828.032714:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880828.032719:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880828.032722:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880828.032744:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880828.032759:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880828.032765:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880828.032775:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880828.032777:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880828.032780:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880828.032785:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880828.032792:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0dc00 00000100:00000001:2.0:1788880828.032794:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880828.032798:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xef5 [1] + 0 00000100:00000001:2.0:1788880828.032799:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880828.032804:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880828.032808:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880828.032810:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880828.032814:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:2.0:1788880828.032814:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880828.032818:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880828.032820:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8697e0a. 00000400:00000200:2.0:1788880828.032824:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d208f26 00000400:00000010:2.0:1788880828.032826:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d208f26. 00000100:00000001:2.0:1788880828.032829:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880828.032830:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880828.032835:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880828.032836:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880848 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880828.032866:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880828.032867:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880848 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880828.032899:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880828.032903:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880828.032906:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880828.032916:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880828.032919:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880828.032922:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880828.032924:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880828.032926:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880828.032929:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880828.032931:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.032934:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:2.0:1788880828.032936:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=56 v=5 (5 5 5 5) 00000100:00000001:1.0:1788880828.032938:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:2.0:1788880828.032942:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=56 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880828.032944:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880828.032945:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880828.032947:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880828.032949:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880828.032951:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880828.032951:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880828.032955:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:2.0:1788880828.032957:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880828.032958:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:1.0:1788880828.032964:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000100:00000001:2.0:1788880828.032966:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788880828.032967:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:2.0:1788880828.032968:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880828.032971:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880828.032972:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:2.0:1788880828.032974:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880828.032979:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880848 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880828.032984:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780688128:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880828.032990:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880828.032994:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880828.032997:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000070db75b2 x1875776780688128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880828.033001:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880848 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880828.033006:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880828.033010:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880828.033013:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:2.0:1788880828.033014:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000087acc4de. 00000100:00000001:1.0:1788880828.033016:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880828.033019:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880828.033019:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:1.0:1788880828.033023:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:2.0:1788880828.033023:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880828.033027:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000040:2.0:1788880828.033027:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880828.033029:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880828.033031:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880828.033036:0:201284:0:(import.c:2149:obd_at_measure()) add 0 to 0000000028cb2d55 time=56 v=5 (5 5 5 5) 00000020:00000001:2.0:1788880828.033037:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880828.033040:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 00000100:00001000:1.0:1788880828.033041:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=13 v=15 (15 1 1 0) 00000100:00000001:1.0:1788880828.033046:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000001:2.0:1788880828.033047:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880828.033048:0:201284:0:(client.c:1493:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 02000000:00000001:2.0:1788880828.033050:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880828.033052:0:201284:0:(client.c:1698:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000010:2.0:1788880828.033053:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000070db75b2. 00000100:00000040:1.0:1788880828.033058:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880848 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880828.033059:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880828.033060:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880828.033063:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880828.033066:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880828.033067:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880828.033071:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880828.033074:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:2.0:1788880828.033074:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000001:3.0:1788880828.033075:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880828.033078:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.033080:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880828.033081:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880828.033083:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.033090:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880828.033093:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:1.0:1788880828.033095:0:201284:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:1.0:1788880828.033098:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788880828.033102:0:201284:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.111@tcp failed (-19) 00000100:00000001:1.0:1788880828.033105:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:1.0:1788880828.033112:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880848 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880828.033112:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880828.033122:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0db80 00000100:00100000:1.0:1788880828.033126:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780688384:192.168.204.111@tcp:38: 00000400:00000200:3.0:1788880828.033129:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xeed [1] + 224 00000800:00000001:3.0:1788880828.033136:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880828.033137:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880828.033140:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880828.033141:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:3.0:1788880828.033143:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:1.0:1788880828.033147:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780688384/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880848 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880828.033147:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880828.033152:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880828.033156:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880828.033158:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880828.033161:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880828.033163:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000b278c5d9. 02000000:00000001:1.0:1788880828.033169:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:3.0:1788880828.033169:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788880828.033171:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880828.033173:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880828.033174:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880828.033177:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 000000008eac1e29. 02000000:00000001:1.0:1788880828.033181:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880828.033183:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880828.033185:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000001:1.0:1788880828.033188:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880828.033191:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880828.033192:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880828.033195:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880828.033196:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880828.033197:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880828.033198:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880828.033199:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880828.033206:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880828.033209:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880828.033227:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880828.033230:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880828.033249:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880828.033250:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880828.033252:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006f1b1f6b. 00000400:00000200:0.0:1788880828.033255:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000400:00000010:0.0:1788880828.033256:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00000001:0.0:1788880828.033259:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880828.033262:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880828.033276:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880828.033282:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880828.033284:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880828.033286:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880828.033287:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880828.033288:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880828.033289:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880828.033291:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880828.033295:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=46 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880828.033298:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=46 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880828.033300:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880828.033301:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880828.033303:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880828.033305:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880828.033308:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880828.033310:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880828.033312:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880828.033313:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880828.033316:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880828.033322:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880828.033324:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880828.033326:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880828.033328:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880828.033335:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780688256:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880828.033339:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880828.033340:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880828.033342:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002dcbd5d0 x1875776780688256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880844 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880828.033358:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880828.033360:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880828.033362:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a58a4768. 02000000:00000001:0.0:1788880828.033365:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880828.033366:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880828.033367:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880828.033369:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880828.033370:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 02000000:00000001:0.0:1788880828.033373:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880828.033374:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880828.033375:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002dcbd5d0. 00000100:00000001:0.0:1788880828.033377:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880828.033378:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880828.033380:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880828.033383:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880828.033385:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880828.033386:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880828.033388:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880828.094326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880828.094333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880828.094355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880828.094358:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880828.094360:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880828.094363:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880828.094366:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880828.094368:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880828.094369:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880828.094372:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880828.094426:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880828.094430:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880828.094433:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880829.118411:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880829.118421:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880829.118429:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880829.118430:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880829.118468:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880829.118475:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880829.118481:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880830.144083:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880830.144088:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880830.144091:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880830.144092:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880830.144119:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880830.144123:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880830.144125:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880831.167357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880831.167361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880831.167365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880831.167366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880831.167399:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880831.167402:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880831.167403:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880832.190332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880832.190339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880832.190346:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880832.190347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880832.190405:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880832.190409:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880832.190413:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880833.150299:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306464207, at 4306464256 00000100:00000040:3.0:1788880833.150302:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306464207, at 4306464256 00000100:00000040:2.0:1788880833.150307:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306464207, at 4306464256 00000100:00000010:0.0:1788880833.150318:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003dccfd54. 00000100:00000010:3.0:1788880833.150320:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e91f0731. 00000100:00000040:1.0:1788880833.150321:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:2.0:1788880833.150324:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005fb7b615. 00000100:00000001:1.0:1788880833.150331:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880833.150349:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788880833.150354:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880833.150359:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880833.150363:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880833.150367:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880833.150374:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000252f4482. 00000100:00000001:1.0:1788880833.150381:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880833.150391:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880833.150401:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880833.150416:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880833.150422:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000008b25f66] to pc [ptlrpcd_00_01+1] req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880833.150455:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880833.150458:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880833.150462:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880833.150465:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001cf64abe. 00000020:00000040:1.0:1788880833.150468:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880833.150471:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880833.150473:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880833.150475:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.150477:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880833.150477:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880833.150479:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:0.0:1788880833.150483:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.150483:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880833.150486:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880833.150489:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880833.150493:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880833.150498:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880833.150508:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880833.150511:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.150517:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880833.150517:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880833.150520:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880833.150521:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880833.150524:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880833.150528:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001cf64abe] to pc [ptlrpcd_00_02+2] req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880833.150528:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780688512:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880833.150537:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880833.150545:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880833.150548:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880833.150552:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880833.150556:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000c9a81c5. 02000000:00000001:3.0:1788880833.150558:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.150564:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880833.150564:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000162a13e6. 00000100:00080000:1.0:1788880833.150568:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788880833.150573:0:147371:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000400:00000010:3.0:1788880833.150574:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000001:1.0:1788880833.150576:0:147371:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:1.0:1788880833.150578:0:147371:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:0.0:1788880833.150581:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880833.150582:0:147371:0:(import.c:485:import_select_connection()) Process entered 00000100:00000001:0.0:1788880833.150586:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880833.150589:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00080000:1.0:1788880833.150589:0:147371:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 11791 00000100:00000200:3.0:1788880833.150591:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780688512, portal 25 00000100:00000001:3.0:1788880833.150596:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880833.150597:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880833.150598:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000400:00000200:1.0:1788880833.150602:0:147371:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000100:00000040:3.0:1788880833.150607:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880833.150609:0:147371:0:(import.c:2149:obd_at_measure()) add 20 to 0000000040f48a4f time=18 v=15 (15 1 1 0) 00000100:00001000:1.0:1788880833.150617:0:147371:0:(import.c:2215:obd_at_measure()) AT 0000000040f48a4f change: old=15 new=20 delta=5 (val=20) hist 20 1 1 0 00000100:00080000:1.0:1788880833.150622:0:147371:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: tried all connections, increasing latency to 15s 00000100:00000040:1.0:1788880833.150635:0:147371:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:3.0:1788880833.150636:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880833.150638:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880833.150640:0:147371:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788880833.150643:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880833.150646:0:147371:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000400:00000010:3.0:1788880833.150647:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 00000100:00000001:0.0:1788880833.150648:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.150651:0:147371:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000200:3.0:1788880833.150652:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780688512, offset 0 02000000:00000001:0.0:1788880833.150654:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000020:00000001:1.0:1788880833.150656:0:147371:0:(genops.c:813:class_conn2export()) Process entered 02000000:00000001:0.0:1788880833.150658:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000020:00000040:1.0:1788880833.150659:0:147371:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 02000000:00000001:0.0:1788880833.150663:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880833.150664:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000020:00000001:1.0:1788880833.150665:0:147371:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000100:00100000:0.0:1788880833.150669:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780688640:192.168.204.111@tcp:400: 00000020:00000040:1.0:1788880833.150670:0:147371:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 00000100:00000001:0.0:1788880833.150674:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000020:00000001:1.0:1788880833.150676:0:147371:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 02000000:00000001:0.0:1788880833.150678:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:3.0:1788880833.150680:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000020:00000001:1.0:1788880833.150682:0:147371:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 02000000:00000001:0.0:1788880833.150683:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880833.150687:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788880833.150688:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788880833.150691:0:147371:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 02000000:00000010:0.0:1788880833.150692:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000c21a3ff8. 00000100:00000001:1.0:1788880833.150696:0:147371:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000400:00000200:3.0:1788880833.150696:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880833.150697:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880833.150702:0:147371:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000400:00000200:3.0:1788880833.150703:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:0.0:1788880833.150704:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006f1b1f6b. 00000100:00000001:1.0:1788880833.150707:0:147371:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000400:00000200:3.0:1788880833.150709:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:0.0:1788880833.150711:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000020:00000040:1.0:1788880833.150713:0:147371:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 00000100:00000200:0.0:1788880833.150717:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780688640, portal 10 00000400:00000200:3.0:1788880833.150717:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [453:453:256:4294967295] 192.168.204.111@tcp LPNI seq info [453:453:8:4294967295] 00000100:00080000:1.0:1788880833.150721:0:147371:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: import 00000000b98b8a4e using connection 192.168.204.111@tcp/192.168.204.111@tcp 00000100:00000001:0.0:1788880833.150724:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880833.150728:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938054144 : -126823771497472 : ffff8ca78a269c00) 00000100:00000001:1.0:1788880833.150729:0:147371:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880833.150734:0:147371:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 00000400:00000200:3.0:1788880833.150738:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880833.150740:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880833.150741:0:147371:0:(sec.c:1628:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880833.150748:0:147371:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880833.150748:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880833.150752:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880833.150754:0:147371:0:(obd_class.h:889:obd_reconnect()) Process entered 00000100:00000001:2.0:1788880833.150757:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880833.150757:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000008:00100000:1.0:1788880833.150760:0:147371:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 2097152, lost: 0. 00000100:00000001:0.0:1788880833.150762:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000008:00000001:1.0:1788880833.150766:0:147371:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880833.150770:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880833.150771:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880833.150772:0:147371:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880833.150774:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ee6cf6a3. 00000100:00000200:0.0:1788880833.150778:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780688640, offset 0 00000100:00000010:1.0:1788880833.150780:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000003b122924. 00000800:00000200:3.0:1788880833.150783:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000020:00000040:1.0:1788880833.150785:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000200:0.0:1788880833.150787:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880833.150791:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 02000000:00000001:1.0:1788880833.150793:0:147371:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 00000400:00000200:0.0:1788880833.150796:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788880833.150797:0:147371:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880833.150801:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880833.150801:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880833.150805:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000200:0.0:1788880833.150806:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788880833.150808:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.150810:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880833.150811:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000010:1.0:1788880833.150816:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 0000000096f76864. 00000100:00000001:3.0:1788880833.150816:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306469327 00000100:00000010:3.0:1788880833.150822:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d6fe56. 00000400:00000200:0.0:1788880833.150826:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880833.150827:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.150829:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880833.150834:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [454:454:255:4294967295] 192.168.204.111@tcp LPNI seq info [454:454:7:4294967295] 00000100:00000001:3.0:1788880833.150837:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306469327 00000100:00100000:1.0:1788880833.150839:0:147371:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880833.150845:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306464207 00000100:00000001:3.0:1788880833.150853:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306469327 00000400:00000200:0.0:1788880833.150856:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880833.150858:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000010:3.0:1788880833.150860:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d54626ec. 00000100:00000001:1.0:1788880833.150861:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880833.150864:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880833.150869:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003b122924] to pc [ptlrpcd_rcv+-1] req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:0.0:1788880833.150870:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b7399401. 00000100:00000001:3.0:1788880833.150872:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880833.150877:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306469327 00000100:00000010:3.0:1788880833.150884:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001bac11fc. 00000800:00000200:0.0:1788880833.150890:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880833.150890:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880833.150895:0:147371:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.150897:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880833.150900:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000200:0.0:1788880833.150901:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880833.150902:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880833.150906:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b7399401 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:1.0:1788880833.150909:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880833.150909:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.150912:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880833.150912:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880833.150914:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880833.150917:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11801) 00000800:00000010:2.0:1788880833.150918:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000040:0.0:1788880833.150920:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 190151 usecs (191 jiffies) 00000100:00000001:0.0:1788880833.150925:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306464447 00000400:00000200:2.0:1788880833.150925:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880833.150925:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880833.150929:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880833.150930:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880833.150932:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306469327 00000100:00000001:1.0:1788880833.150934:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880833.150935:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880833.150936:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880833.150938:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306464447 00000100:00000001:1.0:1788880833.150939:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880833.150939:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880833.150944:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306469327 00000100:00000001:3.0:1788880833.150944:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880833.150945:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 00000100:00000040:1.0:1788880833.150948:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.150950:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306464447 00000400:00000010:2.0:1788880833.150951:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 00000100:00000001:3.0:1788880833.150951:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.150953:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306469327 00000100:00000001:0.0:1788880833.150957:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306464447 00000100:00000001:0.0:1788880833.150961:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306469327 00000100:00000001:2.0:1788880833.150961:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880833.150966:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788880833.150970:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.150971:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.150975:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.150979:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880833.150980:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880833.150985:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.150989:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:1.0:1788880833.150992:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880833.150993:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000001:1.0:1788880833.150995:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880833.150995:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:1.0:1788880833.150999:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880833.151001:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880833.151006:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780688768:192.168.204.111@tcp:38: 00000100:00000001:0.0:1788880833.151010:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880833.151014:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880833.151016:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880833.151018:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880833.151019:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880833.151020:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880833.151024:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880833.151024:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880833.151025:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880833.151029:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880833.151033:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000b278c5d9. 02000000:00000001:1.0:1788880833.151036:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880833.151040:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000400:00000010:1.0:1788880833.151045:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000200:1.0:1788880833.151048:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780688768, portal 10 00000100:00000001:1.0:1788880833.151056:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880833.151057:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938062208 : -126823771489408 : ffff8ca78a26bb80) 00000100:00000040:1.0:1788880833.151063:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880858 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880833.151068:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880833.151070:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880833.151074:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880833.151077:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000065ad0e62. 00000800:00000010:2.0:1788880833.151077:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b7399401. 00000400:00000200:2.0:1788880833.151084:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:1.0:1788880833.151085:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780688768, offset 0 00000400:00000200:2.0:1788880833.151091:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880833.151095:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880833.151102:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880833.151102:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ee6cf6a3 00000400:00000200:1.0:1788880833.151105:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880833.151107:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880833.151111:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:2.0:1788880833.151113:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ee6cf6a3. 00000400:00000200:1.0:1788880833.151116:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880833.151120:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:1.0:1788880833.151121:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [455:455:255:4294967295] 192.168.204.111@tcp LPNI seq info [455:455:7:4294967295] 00000100:00000200:2.0:1788880833.151127:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880833.151140:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880833.151145:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000200:1.0:1788880833.151150:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880833.151152:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:1.0:1788880833.151155:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000800:00000200:1.0:1788880833.151165:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880833.151165:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880833.151169:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880833.151172:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880833.151176:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:1.0:1788880833.151188:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880833.151191:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13471 usecs (14 jiffies) 00000100:00000001:1.0:1788880833.151194:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306464270 00000100:00000001:1.0:1788880833.151196:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306469327 00000100:00000001:1.0:1788880833.151198:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306464270 00000100:00000001:1.0:1788880833.151199:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306469327 00000100:00000001:1.0:1788880833.151201:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306464270 00000100:00000001:1.0:1788880833.151203:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306469327 00000100:00000001:1.0:1788880833.151210:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306464270 00000100:00000001:1.0:1788880833.151212:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306469327 00000100:00000001:1.0:1788880833.151214:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880833.151216:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.151218:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.151220:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880833.151224:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880833.151225:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:1.0:1788880833.151227:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880833.151228:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880833.151229:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.151231:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880833.151273:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880833.151277:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000400:00000200:0.0:1788880833.151282:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880833.151287:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880833.151290:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000065ad0e62 00000400:00000010:0.0:1788880833.151292:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000065ad0e62. 00000100:00000001:0.0:1788880833.151295:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880833.151299:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880858 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.151313:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880833.151315:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880858 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.151321:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.151323:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880833.152245:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880833.152254:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880833.152256:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880833.152259:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880833.152289:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880833.152296:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0dc80 00000400:00000200:3.0:1788880833.152301:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xefd [1] + 224 00000800:00000001:3.0:1788880833.152306:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880833.152309:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880833.152310:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880833.152312:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880833.152315:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880833.152318:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880833.152323:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880833.152335:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880833.152339:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880833.152346:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880833.152352:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880833.152354:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880833.152357:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880833.152361:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0dd00 00000400:00000200:3.0:1788880833.152365:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xf05 [1] + 224 00000800:00000001:3.0:1788880833.152368:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880833.152377:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880833.152380:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880833.152381:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880833.152383:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880833.152386:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:2.0:1788880833.152387:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880833.152390:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880833.152392:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000162a13e6. 00000400:00000200:2.0:1788880833.152396:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000400:00000010:2.0:1788880833.152397:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000001:2.0:1788880833.152400:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880833.152401:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880833.152406:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880833.152406:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880833.152424:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880833.152435:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880833.152448:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880833.152453:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880833.152458:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880833.152458:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880833.152461:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880833.152463:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880833.152464:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880833.152468:0:201286:0:(client.c:1580:after_reply()) Process entered 00000400:00000001:0.0:1788880833.152469:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880833.152471:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:2.0:1788880833.152472:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000010:0.0:1788880833.152473:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006f1b1f6b. 00000100:00000001:2.0:1788880833.152475:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:0.0:1788880833.152477:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000100:00000001:2.0:1788880833.152478:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880833.152479:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00000001:0.0:1788880833.152482:0:201287:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:2.0:1788880833.152484:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880833.152486:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:2.0:1788880833.152492:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=61 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880833.152496:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=61 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880833.152501:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000200:0.0:1788880833.152504:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880833.152505:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880833.152510:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880833.152513:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:2.0:1788880833.152514:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.152515:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880833.152518:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880833.152519:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880833.152519:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.152521:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880833.152522:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880833.152523:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880833.152526:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=51 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880833.152530:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=51 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880833.152532:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:2.0:1788880833.152532:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.152534:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880833.152537:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880833.152540:0:201287:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.152544:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880833.152546:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880833.152547:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880833.152550:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880833.152552:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880833.152554:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880833.152557:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880833.152559:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880833.152568:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880833.152571:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780688512:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880833.152579:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880833.152583:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880833.152584:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880833.152587:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780688512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.152589:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880833.152592:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880833.152599:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880833.152600:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880833.152603:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880833.152606:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000c9a81c5. 00000100:00100000:0.0:1788880833.152611:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780688640:192.168.204.111@tcp:400: 02000000:00000001:2.0:1788880833.152612:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880833.152615:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880833.152617:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880833.152624:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880833.152628:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880833.152631:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000001cf64abe x1875776780688640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880849 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:2.0:1788880833.152631:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880833.152635:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000252f4482. 00000100:00000001:0.0:1788880833.152640:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880833.152643:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:2.0:1788880833.152643:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000010:0.0:1788880833.152647:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000c21a3ff8. 02000000:00000001:2.0:1788880833.152647:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880833.152649:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 02000000:00000001:0.0:1788880833.152653:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880833.152654:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000020:00000001:0.0:1788880833.152656:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880833.152657:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:0.0:1788880833.152658:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880833.152661:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:0.0:1788880833.152662:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880833.152665:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 00000100:00000001:2.0:1788880833.152667:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1788880833.152669:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880833.152670:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880833.152673:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788880833.152674:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:0.0:1788880833.152676:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000001cf64abe. 00000100:00000001:2.0:1788880833.152677:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.152680:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880833.152682:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880833.152684:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880833.152688:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880833.152689:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.152691:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880833.152693:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880833.158519:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.158536:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880833.158541:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.158543:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880833.158565:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880833.158579:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0dd80 00000400:00000200:2.0:1788880833.158588:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xf0d [1] + 0 00000800:00000001:2.0:1788880833.158594:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.158602:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880833.158605:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880833.158609:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880833.158615:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880833.158621:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880858 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880833.158632:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880858 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880833.158651:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880833.158658:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.158661:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880833.158736:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880833.158740:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880833.158745:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880833.158746:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880833.158748:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:1.0:1788880833.158752:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000400:00000010:1.0:1788880833.158753:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:1.0:1788880833.158756:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880833.158765:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880858 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880833.158771:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880858 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880833.158775:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880833.158776:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880833.158777:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880833.158780:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880833.158781:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880833.158782:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880833.158784:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880833.158792:0:201284:0:(import.c:2149:obd_at_measure()) add 0 to 0000000028cb2d55 time=61 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880833.158795:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=18 v=20 (20 1 1 0) 00000100:00000001:1.0:1788880833.158798:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880833.158800:0:201284:0:(client.c:1493:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:1.0:1788880833.158808:0:201284:0:(client.c:1698:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:1.0:1788880833.158812:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880858 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880833.158820:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:1.0:1788880833.158824:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880833.158825:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880833.158827:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00000001:1.0:1788880833.158829:0:201284:0:(import.c:1092:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:1.0:1788880833.158832:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788880833.158835:0:201284:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.111@tcp failed (-19) 00000100:00000001:1.0:1788880833.158837:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:1.0:1788880833.158840:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880858 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880833.158845:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780688768:192.168.204.111@tcp:38: 00000100:00000001:1.0:1788880833.158848:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880833.158849:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880833.158851:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000003b122924 x1875776780688768/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/224 e 0 to 0 dl 1788880858 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880833.158855:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880833.158856:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880833.158858:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000b278c5d9. 02000000:00000001:1.0:1788880833.158861:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880833.158862:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880833.158863:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880833.158864:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880833.158866:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 0000000096f76864. 02000000:00000001:1.0:1788880833.158867:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880833.158869:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880833.158870:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000003b122924. 00000100:00000001:1.0:1788880833.158872:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880833.158877:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880833.158878:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880833.158884:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880833.158885:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.158887:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880833.158888:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880833.214320:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880833.214326:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880833.214331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880833.214332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880833.214362:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880833.214365:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880833.214380:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000001:0.0:1788880833.375702:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.375718:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880833.375722:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.375725:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880833.375736:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880833.375751:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 328 into portal 15 MB=0x6aa0251236e80 00000400:00000200:0.0:1788880833.375759:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index f from 12345-192.168.204.111@tcp of length 328/328 into md 0xd [2] + 1968 00000400:00000010:0.0:1788880833.375763:0:201279:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dd218a96. 00000400:00000200:0.0:1788880833.375768:0:201279:0:(lib-md.c:42:lnet_md_unlink()) Queueing unlink of md 0000000033227dba 00000800:00000001:0.0:1788880833.375773:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.375776:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880833.375778:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880833.375783:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880833.375793:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000033227dba 00000400:00000010:0.0:1788880833.375795:0:201279:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 152 at 0000000033227dba. 00000100:00000001:0.0:1788880833.375801:0:201279:0:(events.c:286:request_in_callback()) Process entered 00000100:00000200:0.0:1788880833.375803:0:201279:0:(events.c:296:request_in_callback()) event type 2, status 0, service ldlm_cbd 00000100:00000040:0.0:1788880833.375806:0:201279:0:(events.c:337:request_in_callback()) incoming req@00000000fde439b4 x1875776788196992 msgsize 328 00000100:00100000:0.0:1788880833.375810:0:201279:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.111@tcp (source: 12345-192.168.204.111@tcp) 00000100:00000040:0.0:1788880833.375812:0:201279:0:(events.c:349:request_in_callback()) Buffer complete: 0 buffers still posted 00000100:00000001:0.0:1788880833.375822:0:201279:0:(events.c:373:request_in_callback()) Process leaving 00000800:00000001:0.0:1788880833.375829:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.375832:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880833.375833:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.375835:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880833.375838:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880833.375850:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 328 into portal 15 MB=0x6aa0251236f00 00000400:00000200:0.0:1788880833.375855:0:201279:0:(lib-ptl.c:640:lnet_ptl_match_md()) Delaying PUT from 12345-192.168.204.111@tcp ptl 15 MB 0x6aa0251236f00 off 0 len 328 00000100:00000010:3.0:1788880833.376000:0:203393:0:(service.c:67:ptlrpc_alloc_rqbd()) kmalloced '(rqbd)': 968 at 000000001e0ff966. 00000100:00000010:3.0:1788880833.376025:0:203393:0:(service.c:77:ptlrpc_alloc_rqbd()) kmalloced '(rqbd->rqbd_buffer)': 8192 at 00000000c7a7b593. 00000100:00100000:3.0:1788880833.376028:0:203393:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00000200:3.0:1788880833.376032:0:203393:0:(niobuf.c:1191:ptlrpc_register_rqbd()) ldlm_cbd: registering portal 15 00000400:00000010:3.0:1788880833.376036:0:203393:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000012059fd7. 00000400:00000010:3.0:1788880833.376041:0:203393:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 152 at 00000000a200a8d0. 00000400:00000200:3.0:1788880833.376047:0:203393:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index f from 12345-192.168.204.111@tcp of length 328/328 into md 0xf15 [2] + 0 00000400:00000200:3.0:1788880833.376050:0:203393:0:(lib-ptl.c:731:lnet_ptl_attach_md()) Resuming delayed PUT from 12345-192.168.204.111@tcp portal 15 match 1875776788197120 offset 0 length 328. 00000400:00000200:3.0:1788880833.376054:0:203393:0:(lib-move.c:4965:lnet_recv_delayed_msg_list()) Resuming delayed PUT from 12345-192.168.204.111@tcp portal 15 match 1875776788197120 offset 0 length 328. 00000100:00000001:3.0:1788880833.376074:0:203393:0:(service.c:2282:ptlrpc_server_handle_req_in()) Process entered 00000100:00100000:3.0:1788880833.376075:0:203393:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875776788196992 02000000:00000001:3.0:1788880833.376077:0:203393:0:(sec.c:2497:sptlrpc_svc_unwrap_request()) Process entered 00000100:00000001:3.0:1788880833.376079:0:203393:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880833.376081:0:203393:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880833.376091:0:203393:0:(sec.c:2546:sptlrpc_svc_unwrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880833.376095:0:203393:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875776788196992 00000800:00000001:0.0:1788880833.376097:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000001:3.0:1788880833.376097:0:203393:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:3.0:1788880833.376099:0:203393:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000001:3.0:1788880833.376102:0:203393:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880833.376105:0:203393:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=3 00000020:00000001:3.0:1788880833.376107:0:203393:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000800:00000001:0.0:1788880833.376109:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880833.376110:0:203393:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000400:00000200:0.0:1788880833.376114:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880833.376117:0:203393:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000400:00000200:0.0:1788880833.376122:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000020:00000010:3.0:1788880833.376122:0:203393:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 00000000d4dd0048. 00000100:00000001:3.0:1788880833.376127:0:203393:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880833.376129:0:201280:0:(events.c:286:request_in_callback()) Process entered 00000020:00000010:3.0:1788880833.376130:0:203393:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 00000000d4dd0048. 00000100:00000200:0.0:1788880833.376139:0:201280:0:(events.c:296:request_in_callback()) event type 2, status 0, service ldlm_cbd 00000020:00000010:3.0:1788880833.376142:0:203393:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000005842920. 00000100:00000010:0.0:1788880833.376144:0:201280:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006f591845. 00000100:00000040:0.0:1788880833.376149:0:201280:0:(events.c:337:request_in_callback()) incoming req@000000006f591845 x1875776788197120 msgsize 328 00000020:00000010:3.0:1788880833.376149:0:203393:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 00000000ba6100d9. 00000020:00000010:3.0:1788880833.376156:0:203393:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 000000000bcc11c1. 00000100:00100000:0.0:1788880833.376157:0:201280:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.111@tcp (source: 12345-192.168.204.111@tcp) 00000100:00000040:3.0:1788880833.376165:0:203393:0:(service.c:1432:ptlrpc_at_set_timer()) armed ldlm_cbd at +6s 00000100:00000001:3.0:1788880833.376168:0:203393:0:(service.c:2003:ptlrpc_server_request_add()) Process entered 00000100:00000001:3.0:1788880833.376169:0:203393:0:(service.c:1919:ptlrpc_server_hpreq_init()) Process entered 00000100:00000001:3.0:1788880833.376170:0:203393:0:(service.c:1949:ptlrpc_server_hpreq_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.376173:0:201280:0:(events.c:373:request_in_callback()) Process leaving 00000100:00000001:3.0:1788880833.376176:0:203393:0:(service.c:2101:ptlrpc_server_request_add()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.376179:0:203393:0:(service.c:2445:ptlrpc_server_handle_req_in()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880833.376185:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880833.376187:0:203393:0:(service.c:2282:ptlrpc_server_handle_req_in()) Process entered 00000800:00000001:0.0:1788880833.376190:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1788880833.376191:0:203393:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875776788197120 02000000:00000001:3.0:1788880833.376193:0:203393:0:(sec.c:2497:sptlrpc_svc_unwrap_request()) Process entered 00000100:00000001:3.0:1788880833.376196:0:203393:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880833.376198:0:203393:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880833.376201:0:203393:0:(sec.c:2546:sptlrpc_svc_unwrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880833.376203:0:203393:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875776788197120 00000020:00000001:3.0:1788880833.376205:0:203393:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:3.0:1788880833.376207:0:203393:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000001:3.0:1788880833.376209:0:203393:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880833.376210:0:203393:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=4 00000020:00000001:3.0:1788880833.376213:0:203393:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000020:00000001:3.0:1788880833.376215:0:203393:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000100:00000001:3.0:1788880833.376218:0:203393:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:3.0:1788880833.376226:0:203393:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 00000000d4dd0048. 00000020:00000010:1.0:1788880833.376228:0:203394:0:(lprocfs_status.c:1410:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 0000000073d230ef. 00000100:00000001:3.0:1788880833.376229:0:203393:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880833.376262:0:203394:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 00000100:00000001:1.0:1788880833.376264:0:203394:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 00000020:00000010:3.0:1788880833.376266:0:203393:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 00000000d4dd0048. 00000100:00100000:1.0:1788880833.376271:0:203394:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.111@tcp, seq: 6 00000020:00000010:3.0:1788880833.376273:0:203393:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000064b136ca. 00000100:00000040:1.0:1788880833.376276:0:203394:0:(service.c:2255:ptlrpc_server_request_get()) RPC GETting export 00000000ea058c72 : new rpc_count 1 00000020:00000010:3.0:1788880833.376277:0:203393:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 00000000ac4552f2. 00000100:00000001:1.0:1788880833.376280:0:203394:0:(service.c:2257:ptlrpc_server_request_get()) Process leaving (rc=18446617250102589520 : -126823606962096 : ffff8ca793f53850) 00000020:00000010:3.0:1788880833.376282:0:203393:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 000000002165b0aa. 00000100:00000001:3.0:1788880833.376288:0:203393:0:(service.c:2003:ptlrpc_server_request_add()) Process entered 00000100:00000040:1.0:1788880833.376289:0:203394:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Interpret req@00000000fde439b4 x1875776788196992/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:454/0 lens 328/0 e 0 to 0 dl 1788880844 ref 1 fl New:/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880833.376290:0:203393:0:(service.c:1919:ptlrpc_server_hpreq_init()) Process entered 00000100:00000001:3.0:1788880833.376293:0:203393:0:(service.c:1949:ptlrpc_server_hpreq_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.376297:0:203393:0:(service.c:2101:ptlrpc_server_request_add()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.376300:0:203393:0:(service.c:2445:ptlrpc_server_handle_req_in()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880833.376307:0:203394:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:1.0:1788880833.376312:0:203394:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 000000000a91cda9. 00000100:00000001:1.0:1788880833.376314:0:203394:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:1.0:1788880833.376317:0:203394:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 000000000a91cda9. 00000100:00100000:1.0:1788880833.376321:0:203394:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@00000000fde439b4 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:fa4856d5-51a8-499e-b3f8-b477743a3dec+4:358207:x1875776788196992:12345-192.168.204.111@tcp:104: 00000100:00000200:1.0:1788880833.376325:0:203394:0:(service.c:2563:ptlrpc_server_handle_request()) got req 1875776788196992 00010000:00000001:1.0:1788880833.376327:0:203394:0:(ldlm_lockd.c:2360:ldlm_callback_handler()) Process entered 00010000:00000001:1.0:1788880833.376334:0:203394:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880833.376335:0:203394:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880833.376337:0:203394:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000003520ed0c refcount=2 00000020:00000001:1.0:1788880833.376339:0:203394:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 00010000:00000001:1.0:1788880833.376341:0:203394:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 00010000:00000001:1.0:1788880833.376344:0:203394:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1788880833.376346:0:203394:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880833.376353:0:203394:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880833.376362:0:203394:0:(pack_generic.c:296:lustre_pack_reply_v2()) Process entered 02000000:00000001:1.0:1788880833.376364:0:203394:0:(sec.c:2568:sptlrpc_svc_alloc_rs()) Process entered 02000000:00000010:1.0:1788880833.376366:0:203394:0:(sec_null.c:305:null_alloc_rs()) kmalloced '(rs)': 520 at 00000000b278c5d9. 02000000:00000001:1.0:1788880833.376368:0:203394:0:(sec.c:2605:sptlrpc_svc_alloc_rs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.376370:0:203394:0:(pack_generic.c:331:lustre_pack_reply_v2()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880833.376373:0:203394:0:(ldlm_lib.c:3317:target_pack_pool_reply()) Process entered 00010000:00000001:1.0:1788880833.376374:0:203394:0:(ldlm_lib.c:3327:target_pack_pool_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880833.376377:0:203394:0:(import.c:2149:obd_at_measure()) add 1 to 0000000013afe31a time=338 v=5 (1 0 1 0) 00000100:00000001:1.0:1788880833.376380:0:203394:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:1.0:1788880833.376382:0:203394:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 13 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880833.376383:0:203394:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 02000000:00000001:1.0:1788880833.376385:0:203394:0:(sec.c:2619:sptlrpc_svc_wrap_reply()) Process entered 02000000:00000001:1.0:1788880833.376387:0:203394:0:(sec.c:2630:sptlrpc_svc_wrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.376388:0:203394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880833.376390:0:203394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880833.376392:0:203394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000200:1.0:1788880833.376395:0:203394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 16, xid 1875776788196992, offset 224 00000400:00000200:1.0:1788880833.376398:0:203394:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880833.376405:0:203394:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880833.376409:0:203394:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [456:456:256:4294967295] 192.168.204.111@tcp LPNI seq info [456:456:8:4294967295] 00000400:00000200:1.0:1788880833.376417:0:203394:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880833.376422:0:203394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880833.376427:0:203394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000800:00000200:1.0:1788880833.376444:0:203394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880833.376452:0:203394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880833.376455:0:203394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880833.376472:0:203394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880833.376475:0:203394:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00010000:00000001:1.0:1788880833.376478:0:203394:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:1.0:1788880833.376486:0:203394:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at 000000004c515e6c. 00010000:00000001:1.0:1788880833.376490:0:203394:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:1.0:1788880833.376500:0:203394:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00010000:00000001:1.0:1788880833.376503:0:203394:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880833.376505:0:203394:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880833.376507:0:203394:0:(ldlm_lockd.c:2533:ldlm_callback_handler()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880833.376512:0:203394:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fde439b4 x1875776788196992/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:454/0 lens 328/224 e 0 to 0 dl 1788880844 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788880833.376522:0:203394:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@00000000fde439b4 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:fa4856d5-51a8-499e-b3f8-b477743a3dec+4:358207:x1875776788196992:12345-192.168.204.111@tcp:104: Request processed in 214us (713us total) trans 0 rc 0/0 00000100:00100000:1.0:1788880833.376530:0:203394:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.111@tcp, seq: 6 00000100:00000040:1.0:1788880833.376533:0:203394:0:(service.c:1113:ptlrpc_server_finish_active_request()) RPC PUTting export 00000000ea058c72 : new rpc_count 0 00000100:00000001:1.0:1788880833.376536:0:203394:0:(service.c:1955:ptlrpc_server_hpreq_fini()) Process entered 00000100:00000001:1.0:1788880833.376539:0:203394:0:(service.c:1966:ptlrpc_server_hpreq_fini()) Process leaving 00000020:00000001:3.0:1788880833.376545:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880833.376548:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880833.376550:0:203399:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000020:00000010:1.0:1788880833.376552:0:203394:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 00000000ba6100d9. 00010000:00000001:3.0:1788880833.376553:0:203399:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00010000:00000001:3.0:1788880833.376556:0:203399:0:(ldlm_lockd.c:1905:ldlm_handle_bl_callback()) Process entered 00010000:00010000:3.0:1788880833.376558:0:203399:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00000010:1.0:1788880833.376562:0:203394:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 000000000bcc11c1. 00010000:00010000:3.0:1788880833.376564:0:203399:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock 000000003520ed0c already unused, calling callback (00000000387296d6) 10000000:00000001:3.0:1788880833.376568:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 00000020:00000010:1.0:1788880833.376569:0:203394:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000005842920. 10000000:00010000:3.0:1788880833.376570:0:203399:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:3.0:1788880833.376577:0:203399:0:(ldlm_request.c:1770:ldlm_cli_cancel()) Process entered 00000800:00000200:2.0:1788880833.376578:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000020:00000040:1.0:1788880833.376579:0:203394:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 3 00010000:00000001:3.0:1788880833.376579:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:3.0:1788880833.376581:0:203399:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880833.376584:0:203399:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000003520ed0c refcount=3 00000800:00000010:2.0:1788880833.376585:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000020:00000001:3.0:1788880833.376586:0:203399:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 02000000:00000001:1.0:1788880833.376589:0:203394:0:(sec.c:2642:sptlrpc_svc_free_rs()) Process entered 00010000:00000001:3.0:1788880833.376590:0:203399:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862491648 : -126823847059968 : ffff8ca785a59e00) 02000000:00000010:1.0:1788880833.376593:0:203394:0:(sec_null.c:330:null_free_rs()) kvfree 'rs': 520 at 00000000556dc547. 00000400:00000200:2.0:1788880833.376594:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00010000:00000001:3.0:1788880833.376594:0:203399:0:(ldlm_request.c:1386:ldlm_cli_cancel_local()) Process entered 00010000:00010000:3.0:1788880833.376597:0:203399:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 02000000:00000001:1.0:1788880833.376599:0:203394:0:(sec.c:2655:sptlrpc_svc_free_rs()) Process leaving 00000100:00000010:1.0:1788880833.376603:0:203394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000400:00000200:2.0:1788880833.376604:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:1.0:1788880833.376608:0:203394:0:(sec.c:2642:sptlrpc_svc_free_rs()) Process entered 00000400:00000200:2.0:1788880833.376609:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 02000000:00000010:1.0:1788880833.376611:0:203394:0:(sec_null.c:330:null_free_rs()) kvfree 'rs': 520 at 000000008ca60ac8. 00000400:00000010:2.0:1788880833.376612:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 02000000:00000001:1.0:1788880833.376615:0:203394:0:(sec.c:2655:sptlrpc_svc_free_rs()) Process leaving 00000100:00000010:1.0:1788880833.376618:0:203394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:2.0:1788880833.376619:0:201278:0:(events.c:386:reply_out_callback()) Process entered 02000000:00000001:2.0:1788880833.376622:0:201278:0:(sec.c:2642:sptlrpc_svc_free_rs()) Process entered 10000000:00000001:3.0:1788880833.376622:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 02000000:00000001:1.0:1788880833.376624:0:203394:0:(sec.c:2642:sptlrpc_svc_free_rs()) Process entered 02000000:00000010:2.0:1788880833.376625:0:201278:0:(sec_null.c:330:null_free_rs()) kvfree 'rs': 520 at 00000000b278c5d9. 10000000:00010000:3.0:1788880833.376625:0:203399:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 02000000:00000010:1.0:1788880833.376626:0:203394:0:(sec_null.c:330:null_free_rs()) kvfree 'rs': 520 at 00000000c2821725. 02000000:00000001:2.0:1788880833.376631:0:201278:0:(sec.c:2655:sptlrpc_svc_free_rs()) Process leaving 02000000:00000001:1.0:1788880833.376632:0:203394:0:(sec.c:2655:sptlrpc_svc_free_rs()) Process leaving 10000000:01000000:3.0:1788880833.376633:0:203399:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00000001:2.0:1788880833.376634:0:201278:0:(events.c:398:reply_out_callback()) Process leaving 00000100:00000010:1.0:1788880833.376635:0:203394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b5bd1bf6. 10000000:00000001:3.0:1788880833.376637:0:203399:0:(mgc_request.c:719:mgc_requeue_add()) Process entered 10000000:00000040:3.0:1788880833.376639:0:203399:0:(mgc_request.c:723:mgc_requeue_add()) log lustre-sptlrpc: requeue (r=2 sp=0 st=1) 02000000:00000001:1.0:1788880833.376640:0:203394:0:(sec.c:2642:sptlrpc_svc_free_rs()) Process entered 10000000:00000040:3.0:1788880833.376641:0:203399:0:(mgc_request.c:726:mgc_requeue_add()) lockh 0x0 02000000:00000010:1.0:1788880833.376642:0:203394:0:(sec_null.c:330:null_free_rs()) kvfree 'rs': 520 at 000000008e52983e. 00010000:00000001:3.0:1788880833.376643:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00010000:00000001:3.0:1788880833.376645:0:203399:0:(ldlm_lock.c:617:__ldlm_handle2lock()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880833.376646:0:203394:0:(sec.c:2655:sptlrpc_svc_free_rs()) Process leaving 00000100:00000010:1.0:1788880833.376648:0:203394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000dc99d6ba. 10000000:00000001:3.0:1788880833.376650:0:203399:0:(mgc_request.c:742:mgc_requeue_add()) Process leaving 10000000:00000001:3.0:1788880833.376652:0:203399:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:3.0:1788880833.376653:0:203399:0:(mgc_request.c:123:config_log_put()) log lustre-sptlrpc(000000007b8fd88a) refs 2 10000000:00000001:3.0:1788880833.376658:0:203399:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:3.0:1788880833.376659:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880833.376660:0:203394:0:(sec.c:2642:sptlrpc_svc_free_rs()) Process entered 02000000:00000010:1.0:1788880833.376663:0:203394:0:(sec_null.c:330:null_free_rs()) kvfree 'rs': 520 at 0000000023d951a9. 00010000:00000001:3.0:1788880833.376663:0:203399:0:(ldlm_lock.c:2502:ldlm_lock_cancel()) Process entered 02000000:00000001:1.0:1788880833.376668:0:203394:0:(sec.c:2655:sptlrpc_svc_free_rs()) Process leaving 00010000:00000001:3.0:1788880833.376668:0:203399:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00000100:00000010:1.0:1788880833.376670:0:203394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00010000:00000001:3.0:1788880833.376671:0:203399:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 02000000:00000001:1.0:1788880833.376675:0:203394:0:(sec.c:2642:sptlrpc_svc_free_rs()) Process entered 00010000:00000001:3.0:1788880833.376675:0:203399:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:3.0:1788880833.376677:0:203399:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880833.376678:0:203394:0:(sec_null.c:330:null_free_rs()) kvfree 'rs': 520 at 000000004ef71132. 02000000:00000001:1.0:1788880833.376680:0:203394:0:(sec.c:2655:sptlrpc_svc_free_rs()) Process leaving 00000020:00000040:3.0:1788880833.376680:0:203399:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object 000000003520ed0c with handle 0x805d548a36a886eb from hash 00010000:00000001:3.0:1788880833.376683:0:203399:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00000100:00000010:1.0:1788880833.376684:0:203394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006a9c2e02. 00010000:00000001:3.0:1788880833.376686:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00000100:00000010:1.0:1788880833.376687:0:203394:0:(service.c:994:ptlrpc_server_drop_request()) kvfree 'rqbd->rqbd_buffer': 8192 at 0000000060d8d6bb. 00010000:00000001:3.0:1788880833.376688:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:3.0:1788880833.376690:0:203399:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00000100:00000010:1.0:1788880833.376692:0:203394:0:(service.c:995:ptlrpc_server_drop_request()) kfreed 'rqbd': 968 at 00000000e885baec. 00010000:00000001:3.0:1788880833.376692:0:203399:0:(ldlm_lock.c:2538:ldlm_lock_cancel()) Process leaving 00010000:00000001:3.0:1788880833.376695:0:203399:0:(ldlm_request.c:1422:ldlm_cli_cancel_local()) Process leaving (rc=70368744177664 : 70368744177664 : 400000000000) 00000100:00000001:1.0:1788880833.376697:0:203394:0:(service.c:2635:ptlrpc_server_handle_request()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880833.376697:0:203399:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:3.0:1788880833.376701:0:203399:0:(ldlm_request.c:1527:ldlm_cli_cancel_req()) Process entered 00000100:00000001:1.0:1788880833.376707:0:203394:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 00000100:00000010:3.0:1788880833.376707:0:203399:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000100:00000001:1.0:1788880833.376710:0:203394:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 00000020:00000040:3.0:1788880833.376711:0:203399:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00100000:1.0:1788880833.376715:0:203394:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.111@tcp, seq: 7 00000100:00000001:3.0:1788880833.376716:0:203399:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:1.0:1788880833.376717:0:203394:0:(service.c:2255:ptlrpc_server_request_get()) RPC GETting export 00000000ea058c72 : new rpc_count 1 02000000:00000001:3.0:1788880833.376719:0:203399:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880833.376721:0:203394:0:(service.c:2257:ptlrpc_server_request_get()) Process leaving (rc=18446617250849363200 : -126822860188416 : ffff8ca7c0781500) 02000000:00000001:3.0:1788880833.376723:0:203399:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880833.376725:0:203394:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Interpret req@000000006f591845 x1875776788197120/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:454/0 lens 328/0 e 0 to 0 dl 1788880844 ref 1 fl New:/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880833.376733:0:203394:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 02000000:00000010:3.0:1788880833.376734:0:203399:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000000c9a81c5. 00000020:00000010:1.0:1788880833.376735:0:203394:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 000000000a91cda9. 00000100:00000001:1.0:1788880833.376738:0:203394:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880833.376739:0:203399:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:1.0:1788880833.376740:0:203394:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 000000000a91cda9. 00010000:00000001:3.0:1788880833.376743:0:203399:0:(ldlm_request.c:1461:ldlm_cancel_pack()) Process entered 00000100:00100000:1.0:1788880833.376745:0:203394:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@000000006f591845 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358207:x1875776788197120:12345-192.168.204.111@tcp:104: 00010000:00010000:3.0:1788880833.376746:0:203399:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000200:1.0:1788880833.376752:0:203394:0:(service.c:2563:ptlrpc_server_handle_request()) got req 1875776788197120 00010000:00000001:1.0:1788880833.376754:0:203394:0:(ldlm_lockd.c:2360:ldlm_callback_handler()) Process entered 00010000:00000001:1.0:1788880833.376757:0:203394:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880833.376758:0:203394:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880833.376759:0:203394:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000004fb52558 refcount=2 00000020:00000001:1.0:1788880833.376760:0:203394:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000001:1.0:1788880833.376762:0:203394:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000001:1.0:1788880833.376763:0:203394:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1788880833.376769:0:203394:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880833.376771:0:203394:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880833.376775:0:203394:0:(pack_generic.c:296:lustre_pack_reply_v2()) Process entered 02000000:00000001:1.0:1788880833.376776:0:203394:0:(sec.c:2568:sptlrpc_svc_alloc_rs()) Process entered 02000000:00000010:1.0:1788880833.376779:0:203394:0:(sec_null.c:305:null_alloc_rs()) kmalloced '(rs)': 520 at 000000008eac1e29. 02000000:00000001:1.0:1788880833.376780:0:203394:0:(sec.c:2605:sptlrpc_svc_alloc_rs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.376782:0:203394:0:(pack_generic.c:331:lustre_pack_reply_v2()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880833.376785:0:203394:0:(ldlm_lib.c:3317:target_pack_pool_reply()) Process entered 00010000:00000001:1.0:1788880833.376786:0:203394:0:(ldlm_lib.c:3327:target_pack_pool_reply()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880833.376789:0:203399:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00001000:1.0:1788880833.376791:0:203394:0:(import.c:2149:obd_at_measure()) add 1 to 0000000013afe31a time=38 v=5 (1 0 1 0) 00010000:00000001:3.0:1788880833.376792:0:203399:0:(ldlm_request.c:1500:ldlm_cancel_pack()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.376796:0:203399:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880833.376797:0:203394:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000001:3.0:1788880833.376798:0:203399:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880833.376803:0:203394:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 13 to 192.168.204.111@tcp 00000100:00000040:3.0:1788880833.376805:0:203399:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_03+3] req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880833.376808:0:203394:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 02000000:00000001:1.0:1788880833.376813:0:203394:0:(sec.c:2619:sptlrpc_svc_wrap_reply()) Process entered 02000000:00000001:1.0:1788880833.376816:0:203394:0:(sec.c:2630:sptlrpc_svc_wrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.376820:0:203394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880833.376825:0:203394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880833.376828:0:203394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000030b8c48c. 00000100:00000200:1.0:1788880833.376830:0:203394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 16, xid 1875776788197120, offset 224 00000400:00000200:1.0:1788880833.376834:0:203394:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00010000:00000001:3.0:1788880833.376836:0:203399:0:(ldlm_request.c:1595:ldlm_cli_cancel_req()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:3.0:1788880833.376840:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00000400:00000200:1.0:1788880833.376841:0:203394:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00010000:00000001:3.0:1788880833.376843:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:3.0:1788880833.376846:0:203399:0:(ldlm_request.c:2504:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880833.376849:0:203394:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [457:457:256:4294967295] 192.168.204.111@tcp LPNI seq info [457:457:8:4294967295] 00010000:00000001:3.0:1788880833.376850:0:203399:0:(ldlm_request.c:1834:ldlm_cli_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:3.0:1788880833.376854:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880833.376855:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00010000:3.0:1788880833.376859:0:203399:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:2.0:1788880833.376860:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880833.376865:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880833.376870:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880833.376873:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880833.376874:0:203394:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00010000:00000001:3.0:1788880833.376874:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00000100:00000001:0.0:1788880833.376875:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880833.376878:0:203399:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.111@tcp lock: 000000003520ed0c/0x805d548a36a886eb lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7be1 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000200:1.0:1788880833.376886:0:203394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880833.376893:0:203394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000100:00000001:2.0:1788880833.376900:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000200:1.0:1788880833.376901:0:203394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00010000:00000001:3.0:1788880833.376902:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000100:00000001:2.0:1788880833.376904:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.376905:0:203399:0:(ldlm_lockd.c:1935:ldlm_handle_bl_callback()) Process leaving 00010000:00000010:3.0:1788880833.376907:0:203399:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at 000000004c515e6c. 00000800:00000200:1.0:1788880833.376911:0:203394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00010000:00000001:3.0:1788880833.376911:0:203399:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880833.376916:0:203394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880833.376916:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000020:00000001:3.0:1788880833.376916:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880833.376919:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880833.376921:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00010000:00010000:3.0:1788880833.376922:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880833.376924:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880833.376926:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880833.376927:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880833.376929:0:203394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880833.376929:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788880833.376934:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780688896:192.168.204.111@tcp:103: 00000100:00000040:1.0:1788880833.376935:0:203394:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000020:00000001:3.0:1788880833.376937:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880833.376939:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880833.376943:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00010000:00010000:3.0:1788880833.376943:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880833.376946:0:203394:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:1.0:1788880833.376949:0:203394:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at 000000009c63ae8f. 02000000:00000001:2.0:1788880833.376950:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00010000:00000001:1.0:1788880833.376952:0:203394:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 02000000:00000001:2.0:1788880833.376954:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880833.376958:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880833.376962:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000bef544e8. 00010000:00010000:1.0:1788880833.376965:0:203394:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 02000000:00000001:2.0:1788880833.376965:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880833.376970:0:203394:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880833.376972:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8697e0a. 00010000:00000001:1.0:1788880833.376973:0:203394:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880833.376977:0:203394:0:(ldlm_lockd.c:2533:ldlm_callback_handler()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880833.376978:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d208f26. 00000100:00000040:1.0:1788880833.376982:0:203394:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006f591845 x1875776788197120/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:454/0 lens 328/224 e 0 to 0 dl 1788880844 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000020:00000001:3.0:1788880833.376983:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000100:00000200:2.0:1788880833.376984:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780688896, portal 18 00000800:00000200:0.0:1788880833.376985:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000020:00000001:3.0:1788880833.376987:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880833.376991:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000100:00000001:2.0:1788880833.376991:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00010000:00010000:3.0:1788880833.376991:0:203398:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00000001:2.0:1788880833.376995:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00010000:00000001:3.0:1788880833.376996:0:203398:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00000400:00000200:0.0:1788880833.376997:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00010000:00000001:3.0:1788880833.376999:0:203398:0:(ldlm_lockd.c:1905:ldlm_handle_bl_callback()) Process entered 00010000:00010000:3.0:1788880833.377002:0:203398:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000040:2.0:1788880833.377006:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880833.377007:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00100000:1.0:1788880833.377007:0:203394:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@000000006f591845 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358207:x1875776788197120:12345-192.168.204.111@tcp:104: Request processed in 269us (852us total) trans 0 rc 0/0 00000400:00000200:0.0:1788880833.377012:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000030b8c48c 00000400:00000010:0.0:1788880833.377016:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000030b8c48c. 00000100:00000001:0.0:1788880833.377020:0:201279:0:(events.c:386:reply_out_callback()) Process entered 00000100:00100000:1.0:1788880833.377023:0:203394:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.111@tcp, seq: 7 00000100:00000001:0.0:1788880833.377025:0:201279:0:(events.c:398:reply_out_callback()) Process leaving 00010000:00010000:3.0:1788880833.377025:0:203398:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock 000000004fb52558 already unused, calling callback (00000000387296d6) 00000100:00000040:1.0:1788880833.377028:0:203394:0:(service.c:1113:ptlrpc_server_finish_active_request()) RPC PUTting export 00000000ea058c72 : new rpc_count 0 10000000:00000001:3.0:1788880833.377031:0:203398:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 00000100:00000001:1.0:1788880833.377033:0:203394:0:(service.c:1955:ptlrpc_server_hpreq_fini()) Process entered 00000100:00000001:2.0:1788880833.377033:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 10000000:00010000:3.0:1788880833.377034:0:203398:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880833.377037:0:203394:0:(service.c:1966:ptlrpc_server_hpreq_fini()) Process leaving 00000100:00000040:2.0:1788880833.377041:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000020:00000010:1.0:1788880833.377042:0:203394:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 00000000ac4552f2. 00000400:00000010:2.0:1788880833.377046:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c013629f. 00000020:00000010:1.0:1788880833.377047:0:203394:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 000000002165b0aa. 00000100:00000200:2.0:1788880833.377051:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 17, xid 1875776780688896, offset 0 00000020:00000010:1.0:1788880833.377052:0:203394:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000064b136ca. 00000020:00000040:1.0:1788880833.377058:0:203394:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 2 00000400:00000200:2.0:1788880833.377060:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880833.377062:0:203394:0:(service.c:2635:ptlrpc_server_handle_request()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880833.377068:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880833.377073:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880833.377077:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880833.377081:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880833.377085:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00010000:00000001:3.0:1788880833.377086:0:203398:0:(ldlm_request.c:1770:ldlm_cli_cancel()) Process entered 00010000:00000001:3.0:1788880833.377089:0:203398:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:3.0:1788880833.377091:0:203398:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000400:00000200:2.0:1788880833.377093:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [458:458:256:4294967295] 192.168.204.111@tcp LPNI seq info [458:458:8:4294967295] 00000020:00000040:3.0:1788880833.377093:0:203398:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000004fb52558 refcount=3 00000020:00000001:3.0:1788880833.377097:0:203398:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000001:3.0:1788880833.377104:0:203398:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280349184 : -126823429202432 : ffff8ca79e8d9e00) 00010000:00000001:3.0:1788880833.377106:0:203398:0:(ldlm_request.c:1386:ldlm_cli_cancel_local()) Process entered 00010000:00010000:3.0:1788880833.377110:0:203398:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000400:00000200:2.0:1788880833.377115:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 10000000:00000001:3.0:1788880833.377124:0:203398:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 00000800:00000200:2.0:1788880833.377125:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 10000000:00010000:3.0:1788880833.377127:0:203398:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000010:2.0:1788880833.377130:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f9812869. 10000000:01000000:3.0:1788880833.377137:0:203398:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000800:00000200:2.0:1788880833.377138:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 10000000:00000001:3.0:1788880833.377142:0:203398:0:(mgc_request.c:719:mgc_requeue_add()) Process entered 10000000:00000040:3.0:1788880833.377143:0:203398:0:(mgc_request.c:723:mgc_requeue_add()) log lustre-client: requeue (r=2 sp=0 st=1) 10000000:00000040:3.0:1788880833.377147:0:203398:0:(mgc_request.c:726:mgc_requeue_add()) lockh 0x0 00000800:00000200:2.0:1788880833.377151:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00010000:00000001:3.0:1788880833.377155:0:203398:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000800:00000200:2.0:1788880833.377156:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f9812869 type 1, nob 424 niov 1 nkiov 1 00010000:00000001:3.0:1788880833.377157:0:203398:0:(ldlm_lock.c:617:__ldlm_handle2lock()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:3.0:1788880833.377168:0:203398:0:(mgc_request.c:742:mgc_requeue_add()) Process leaving 10000000:00000001:3.0:1788880833.377170:0:203398:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:3.0:1788880833.377172:0:203398:0:(mgc_request.c:123:config_log_put()) log lustre-client(00000000c2974351) refs 2 10000000:00000001:3.0:1788880833.377174:0:203398:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:3.0:1788880833.377174:0:203398:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.377176:0:203398:0:(ldlm_lock.c:2502:ldlm_lock_cancel()) Process entered 00010000:00000001:3.0:1788880833.377178:0:203398:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:3.0:1788880833.377179:0:203398:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:3.0:1788880833.377180:0:203398:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00000100:00000001:2.0:1788880833.377181:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.377181:0:203398:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880833.377187:0:203398:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object 000000004fb52558 with handle 0x805d548a36a886e4 from hash 00000100:00000001:2.0:1788880833.377189:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306469553 00010000:00000001:3.0:1788880833.377190:0:203398:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:3.0:1788880833.377192:0:203398:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:3.0:1788880833.377195:0:203398:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:3.0:1788880833.377196:0:203398:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:3.0:1788880833.377197:0:203398:0:(ldlm_lock.c:2538:ldlm_lock_cancel()) Process leaving 00010000:00000001:3.0:1788880833.377199:0:203398:0:(ldlm_request.c:1422:ldlm_cli_cancel_local()) Process leaving (rc=70368744177664 : 70368744177664 : 400000000000) 00000100:00000001:2.0:1788880833.377200:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306469327 00010000:00000001:3.0:1788880833.377201:0:203398:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:3.0:1788880833.377204:0:203398:0:(ldlm_request.c:1527:ldlm_cli_cancel_req()) Process entered 00000100:00000001:2.0:1788880833.377207:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306469553 00000100:00000010:3.0:1788880833.377208:0:203398:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880833.377212:0:203398:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880833.377215:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306469327 00000100:00000001:3.0:1788880833.377215:0:203398:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880833.377218:0:203398:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880833.377220:0:203398:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880833.377221:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306469553 02000000:00000010:3.0:1788880833.377224:0:203398:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 0000000064b136ca. 00000100:00000001:2.0:1788880833.377226:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306469327 00000100:00000001:3.0:1788880833.377228:0:203398:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.377230:0:203398:0:(ldlm_request.c:1461:ldlm_cancel_pack()) Process entered 00000100:00000001:2.0:1788880833.377231:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306469553 00000800:00000200:0.0:1788880833.377252:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880833.377256:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f9812869. 00000400:00000200:0.0:1788880833.377258:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880833.377264:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306469327 00010000:00010000:3.0:1788880833.377264:0:203398:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000400:00000200:0.0:1788880833.377266:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880833.377271:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880833.377274:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c013629f 00010000:00010000:3.0:1788880833.377277:0:203398:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000400:00000010:0.0:1788880833.377279:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c013629f. 00000100:00000001:2.0:1788880833.377279:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.377279:0:203398:0:(ldlm_request.c:1500:ldlm_cancel_pack()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.377285:0:203398:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880833.377286:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.377287:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880833.377290:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880833.377293:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880833.377298:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880833.377301:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880833.377305:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880833.377306:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880833.377307:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880833.377310:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880833.377311:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880833.377313:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.377318:0:203398:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.377324:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880833.377326:0:203398:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_00+0] req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.377328:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880833.377344:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:3.0:1788880833.377345:0:203398:0:(ldlm_request.c:1595:ldlm_cli_cancel_req()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880833.377346:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:3.0:1788880833.377348:0:203398:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00000100:00000001:0.0:1788880833.377349:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00010000:00000001:3.0:1788880833.377350:0:203398:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:3.0:1788880833.377352:0:203398:0:(ldlm_request.c:2504:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.377354:0:203398:0:(ldlm_request.c:1834:ldlm_cli_cancel()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880833.377355:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 10000000:00000001:3.0:1788880833.377357:0:203398:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880833.377358:0:203398:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:2.0:1788880833.377363:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:3.0:1788880833.377365:0:203398:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00000100:00000001:2.0:1788880833.377368:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880833.377368:0:203398:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.111@tcp lock: 000000004fb52558/0x805d548a36a886e4 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7bda expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:0.0:1788880833.377375:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880833.377378:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00010000:00000040:3.0:1788880833.377378:0:203398:0:(ldlm_resource.c:1704:ldlm_resource_putref()) putref res: 00000000b425dd0a count: 0 02000000:00000001:0.0:1788880833.377382:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880833.377384:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880833.377386:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.377387:0:203398:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000100:00100000:0.0:1788880833.377388:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780689024:192.168.204.111@tcp:103: 00010000:00000001:3.0:1788880833.377389:0:203398:0:(ldlm_lockd.c:1935:ldlm_handle_bl_callback()) Process leaving 00010000:00000010:3.0:1788880833.377391:0:203398:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at 000000009c63ae8f. 00000100:00000001:0.0:1788880833.377396:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00010000:00000001:3.0:1788880833.377396:0:203398:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880833.377400:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000020:00000001:3.0:1788880833.377400:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880833.377401:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880833.377402:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880833.377404:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 02000000:00000001:0.0:1788880833.377405:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000020:00000001:3.0:1788880833.377406:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880833.377408:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880833.377409:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000c79839a2. 00010000:00010000:3.0:1788880833.377409:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:3.0:1788880833.377410:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880833.377412:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880833.377414:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880833.377414:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000010:0.0:1788880833.377418:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dd218a96. 00000400:00000010:0.0:1788880833.377421:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000100:00000200:0.0:1788880833.377423:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780689024, portal 18 00000100:00000001:0.0:1788880833.377424:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880833.377425:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000040:0.0:1788880833.377428:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.377446:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880833.377449:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880833.377451:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ee6cf6a3. 00000100:00000200:0.0:1788880833.377454:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 17, xid 1875776780689024, offset 0 00000400:00000200:0.0:1788880833.377458:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880833.377462:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880833.377465:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880833.377468:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880833.377470:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880833.377472:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880833.377475:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [459:459:256:4294967295] 192.168.204.111@tcp LPNI seq info [459:459:8:4294967295] 00000400:00000200:0.0:1788880833.377483:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880833.377492:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880833.377494:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000050c5542b. 00000800:00000200:0.0:1788880833.377497:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880833.377500:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880833.377501:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000050c5542b type 1, nob 424 niov 1 nkiov 1 00000100:00000001:0.0:1788880833.377509:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.377511:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306469554 00000100:00000001:0.0:1788880833.377514:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306469553 00000100:00000001:0.0:1788880833.377516:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306469554 00000100:00000001:0.0:1788880833.377518:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306469553 00000100:00000001:0.0:1788880833.377519:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306469554 00000100:00000001:0.0:1788880833.377521:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306469553 00000100:00000001:0.0:1788880833.377522:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306469554 00000100:00000001:0.0:1788880833.377523:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306469553 00000100:00000001:0.0:1788880833.377524:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880833.377525:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.377527:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.377528:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880833.377531:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880833.377532:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880833.377534:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880833.377534:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880833.377536:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.377537:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880833.377572:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880833.377575:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000050c5542b. 00000400:00000200:2.0:1788880833.377579:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880833.377583:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880833.377586:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ee6cf6a3 00000400:00000010:2.0:1788880833.377587:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ee6cf6a3. 00000100:00000001:2.0:1788880833.377590:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880833.377593:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880833.377611:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880833.377613:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880833.377618:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880833.377619:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880833.379229:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.379246:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880833.379248:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.379250:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880833.379255:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880833.379262:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 18 MB=0x6aa0250b0de80 00000400:00000200:0.0:1788880833.379266:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 12 from 12345-192.168.204.111@tcp of length 224/224 into md 0xf29 [1] + 224 00000800:00000001:0.0:1788880833.379273:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.379277:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880833.379278:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880833.379280:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880833.379283:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880833.379286:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880833.379292:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.379306:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880833.379310:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.379312:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.379327:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880833.379330:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880833.379335:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880833.379337:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880833.379339:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dd218a96. 00000400:00000200:3.0:1788880833.379342:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000400:00000010:3.0:1788880833.379344:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00000001:3.0:1788880833.379352:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880833.379357:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880833.379363:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880833.379367:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880833.379368:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.379374:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880833.379376:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880833.379377:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880833.379378:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880833.379379:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880833.379383:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000081111466 time=338 v=5 (5 0 5 0) 00000100:00001000:3.0:1788880833.379385:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=61 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880833.379386:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880833.379387:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.379389:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880833.379391:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.379393:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880833.379397:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880833.379403:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880833.379405:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880833.379407:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880833.379410:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880833.379417:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780689024:192.168.204.111@tcp:103: 00000100:00000001:3.0:1788880833.379421:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.379423:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880833.379426:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780689024/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880833.379432:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880833.379450:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880833.379452:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000c79839a2. 02000000:00000001:3.0:1788880833.379455:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880833.379457:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880833.379458:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880833.379460:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880833.379462:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 0000000064b136ca. 02000000:00000001:3.0:1788880833.379465:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880833.379467:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880833.379468:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:3.0:1788880833.379471:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880833.379472:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.379474:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.379529:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880833.379531:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.379533:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880833.379535:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880833.379548:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.379553:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880833.379554:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.379555:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880833.379558:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880833.379562:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 18 MB=0x6aa0250b0de00 00000400:00000200:0.0:1788880833.379565:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 12 from 12345-192.168.204.111@tcp of length 224/224 into md 0xf21 [1] + 224 00000800:00000001:0.0:1788880833.379567:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.379588:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880833.379589:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880833.379591:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880833.379594:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880833.379596:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880833.379600:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.379610:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880833.379613:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880833.379614:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880833.379640:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880833.379643:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880833.379645:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880833.379646:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880833.379647:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8697e0a. 00000400:00000200:2.0:1788880833.379650:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d208f26 00000400:00000010:2.0:1788880833.379651:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d208f26. 00000100:00000001:2.0:1788880833.379653:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880833.379656:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880833.379661:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880833.379665:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880833.379666:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880833.379667:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880833.379668:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880833.379669:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880833.379670:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880833.379671:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880833.379673:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000081111466 time=38 v=5 (5 0 5 0) 00000100:00001000:2.0:1788880833.379675:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=61 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880833.379676:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880833.379677:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880833.379678:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880833.379679:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880833.379680:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880833.379681:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880833.379685:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880833.379691:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880833.379693:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880833.379695:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880833.379702:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780688896:192.168.204.111@tcp:103: 00000100:00000001:2.0:1788880833.379706:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880833.379708:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880833.379710:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780688896/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880833.379715:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880833.379717:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880833.379719:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000bef544e8. 02000000:00000001:2.0:1788880833.379722:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880833.379723:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880833.379724:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880833.379726:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880833.379728:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000000c9a81c5. 02000000:00000001:2.0:1788880833.379730:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880833.379731:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880833.379732:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:2.0:1788880833.379735:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880833.379736:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880833.379737:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880833.379741:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880833.379742:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880833.379743:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880833.379744:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000010:3.1:1788880833.382265:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 000000003520ed0c. 00010000:00000010:3.1:1788880833.386274:0:0:0:(ldlm_resource.c:1555:__ldlm_resource_free()) slab-freed '(res)': 256 at 00000000b425dd0a. 00010000:00000010:3.1:1788880833.386287:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 000000004fb52558. 00000800:00000001:2.0:1788880833.485858:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.485875:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880833.485879:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.485881:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880833.485892:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880833.485907:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 328 into portal 15 MB=0x6aa0251238100 00000400:00000200:2.0:1788880833.485915:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index f from 12345-192.168.204.111@tcp of length 328/328 into md 0xf15 [2] + 328 00000800:00000001:2.0:1788880833.485941:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.485951:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880833.485953:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880833.485958:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880833.485965:0:201278:0:(events.c:286:request_in_callback()) Process entered 00000100:00000200:2.0:1788880833.485967:0:201278:0:(events.c:296:request_in_callback()) event type 2, status 0, service ldlm_cbd 00000100:00000010:2.0:1788880833.485973:0:201278:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000fd6b7ee1. 00000100:00000040:2.0:1788880833.485977:0:201278:0:(events.c:337:request_in_callback()) incoming req@00000000fd6b7ee1 x1875776788201728 msgsize 328 00000100:00100000:2.0:1788880833.485981:0:201278:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.111@tcp (source: 12345-192.168.204.111@tcp) 00000100:00000001:2.0:1788880833.485993:0:201278:0:(events.c:373:request_in_callback()) Process leaving 00000800:00000001:2.0:1788880833.486000:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.486003:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.486023:0:203393:0:(service.c:2282:ptlrpc_server_handle_req_in()) Process entered 00000100:00100000:3.0:1788880833.486026:0:203393:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875776788201728 02000000:00000001:3.0:1788880833.486028:0:203393:0:(sec.c:2497:sptlrpc_svc_unwrap_request()) Process entered 00000100:00000001:3.0:1788880833.486030:0:203393:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880833.486032:0:203393:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880833.486035:0:203393:0:(sec.c:2546:sptlrpc_svc_unwrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880833.486039:0:203393:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875776788201728 00000020:00000001:3.0:1788880833.486040:0:203393:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:3.0:1788880833.486042:0:203393:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a88651 00000020:00000001:3.0:1788880833.486043:0:203393:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880833.486045:0:203393:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000ea058c72 refcount=3 00000020:00000001:3.0:1788880833.486047:0:203393:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000020:00000001:3.0:1788880833.486049:0:203393:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617250083375104 : -126823626176512 : ffff8ca792d00800) 00000100:00000001:3.0:1788880833.486052:0:203393:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:3.0:1788880833.486055:0:203393:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 00000000d4dd0048. 00000100:00000001:3.0:1788880833.486057:0:203393:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880833.486059:0:203393:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 00000000d4dd0048. 00000020:00000010:3.0:1788880833.486066:0:203393:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000064b136ca. 00000020:00000010:3.0:1788880833.486070:0:203393:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at 000000003a5795a6. 00000020:00000010:3.0:1788880833.486079:0:203393:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at 00000000c88a3ea9. 00000100:00000040:3.0:1788880833.486087:0:203393:0:(service.c:1432:ptlrpc_at_set_timer()) armed ldlm_cbd at +6s 00000100:00000001:3.0:1788880833.486090:0:203393:0:(service.c:2003:ptlrpc_server_request_add()) Process entered 00000100:00000001:3.0:1788880833.486091:0:203393:0:(service.c:1919:ptlrpc_server_hpreq_init()) Process entered 00000100:00000001:3.0:1788880833.486093:0:203393:0:(service.c:1949:ptlrpc_server_hpreq_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.486101:0:203393:0:(service.c:2101:ptlrpc_server_request_add()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.486111:0:203393:0:(service.c:2445:ptlrpc_server_handle_req_in()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.486122:0:203393:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 00000100:00000001:3.0:1788880833.486124:0:203393:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 00000100:00100000:3.0:1788880833.486128:0:203393:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.111@tcp, seq: 8 00000100:00000040:3.0:1788880833.486131:0:203393:0:(service.c:2255:ptlrpc_server_request_get()) RPC GETting export 00000000ea058c72 : new rpc_count 1 00000100:00000001:3.0:1788880833.486133:0:203393:0:(service.c:2257:ptlrpc_server_request_get()) Process leaving (rc=18446617249938438144 : -126823771113472 : ffff8ca78a2c7800) 00000100:00000040:3.0:1788880833.486139:0:203393:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Interpret req@00000000fd6b7ee1 x1875776788201728/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:454/0 lens 328/0 e 0 to 0 dl 1788880844 ref 1 fl New:/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880833.486146:0:203393:0:(service.c:1267:ptlrpc_update_export_timer()) Process entered 00000020:00000010:3.0:1788880833.486148:0:203393:0:(genops.c:1076:obd_export_timed_init()) kmalloced '(*data)': 48 at 00000000d4dd0048. 00000100:00000001:3.0:1788880833.486149:0:203393:0:(service.c:1289:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788880833.486151:0:203393:0:(genops.c:1084:obd_export_timed_fini()) kfreed '*data': 48 at 00000000d4dd0048. 00000100:00100000:3.0:1788880833.486155:0:203393:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@00000000fd6b7ee1 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358231:x1875776788201728:12345-192.168.204.111@tcp:104: 00000100:00000200:3.0:1788880833.486158:0:203393:0:(service.c:2563:ptlrpc_server_handle_request()) got req 1875776788201728 00010000:00000001:3.0:1788880833.486160:0:203393:0:(ldlm_lockd.c:2360:ldlm_callback_handler()) Process entered 00010000:00000001:3.0:1788880833.486165:0:203393:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:3.0:1788880833.486166:0:203393:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880833.486167:0:203393:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=2 00000020:00000001:3.0:1788880833.486168:0:203393:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:3.0:1788880833.486170:0:203393:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:3.0:1788880833.486173:0:203393:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:3.0:1788880833.486175:0:203393:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880833.486178:0:203393:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:3.0:1788880833.486192:0:203393:0:(pack_generic.c:296:lustre_pack_reply_v2()) Process entered 02000000:00000001:3.0:1788880833.486193:0:203393:0:(sec.c:2568:sptlrpc_svc_alloc_rs()) Process entered 02000000:00000010:3.0:1788880833.486197:0:203393:0:(sec_null.c:305:null_alloc_rs()) kmalloced '(rs)': 520 at 00000000d7b2c6b7. 02000000:00000001:3.0:1788880833.486198:0:203393:0:(sec.c:2605:sptlrpc_svc_alloc_rs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.486200:0:203393:0:(pack_generic.c:331:lustre_pack_reply_v2()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.486204:0:203393:0:(ldlm_lib.c:3317:target_pack_pool_reply()) Process entered 00010000:00000001:3.0:1788880833.486205:0:203393:0:(ldlm_lib.c:3327:target_pack_pool_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880833.486208:0:203393:0:(import.c:2149:obd_at_measure()) add 1 to 0000000013afe31a time=38 v=5 (1 0 1 0) 00000100:00000001:3.0:1788880833.486211:0:203393:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788880833.486213:0:203393:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 13 to 192.168.204.111@tcp 00000100:00000001:3.0:1788880833.486214:0:203393:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 02000000:00000001:3.0:1788880833.486216:0:203393:0:(sec.c:2619:sptlrpc_svc_wrap_reply()) Process entered 02000000:00000001:3.0:1788880833.486217:0:203393:0:(sec.c:2630:sptlrpc_svc_wrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.486220:0:203393:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880833.486221:0:203393:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880833.486224:0:203393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000003539bb5a. 00000100:00000200:3.0:1788880833.486227:0:203393:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 16, xid 1875776788201728, offset 224 00000400:00000200:3.0:1788880833.486231:0:203393:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880833.486261:0:203393:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880833.486266:0:203393:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [460:460:256:4294967295] 192.168.204.111@tcp LPNI seq info [460:460:8:4294967295] 00000400:00000200:3.0:1788880833.486273:0:203393:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880833.486277:0:203393:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880833.486280:0:203393:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000800:00000200:3.0:1788880833.486284:0:203393:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880833.486289:0:203393:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880833.486292:0:203393:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880833.486301:0:203393:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880833.486303:0:203393:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00010000:00000001:3.0:1788880833.486306:0:203393:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:3.0:1788880833.486308:0:203393:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at 00000000edd37b97. 00010000:00000001:3.0:1788880833.486313:0:203393:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:3.0:1788880833.486323:0:203393:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00010000:00000001:3.0:1788880833.486325:0:203393:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.486326:0:203393:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.486328:0:203393:0:(ldlm_lockd.c:2533:ldlm_callback_handler()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880833.486332:0:203393:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000fd6b7ee1 x1875776788201728/t0(0) o104->fa4856d5-51a8-499e-b3f8-b477743a3dec@192.168.204.111@tcp:454/0 lens 328/224 e 0 to 0 dl 1788880844 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788880833.486338:0:203393:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@00000000fd6b7ee1 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:fa4856d5-51a8-499e-b3f8-b477743a3dec+3:358231:x1875776788201728:12345-192.168.204.111@tcp:104: Request processed in 192us (360us total) trans 0 rc 0/0 00000100:00100000:3.0:1788880833.486343:0:203393:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.111@tcp, seq: 8 00000100:00000040:3.0:1788880833.486346:0:203393:0:(service.c:1113:ptlrpc_server_finish_active_request()) RPC PUTting export 00000000ea058c72 : new rpc_count 0 00000100:00000001:3.0:1788880833.486347:0:203393:0:(service.c:1955:ptlrpc_server_hpreq_fini()) Process entered 00000100:00000001:3.0:1788880833.486350:0:203393:0:(service.c:1966:ptlrpc_server_hpreq_fini()) Process leaving 00000020:00000010:3.0:1788880833.486353:0:203393:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at 000000003a5795a6. 00000020:00000001:2.0:1788880833.486354:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000010:3.0:1788880833.486356:0:203393:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at 00000000c88a3ea9. 00000020:00000001:2.0:1788880833.486357:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880833.486361:0:203399:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000020:00000010:3.0:1788880833.486361:0:203393:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000064b136ca. 00010000:00000001:2.0:1788880833.486366:0:203399:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00010000:00000001:2.0:1788880833.486368:0:203399:0:(ldlm_lockd.c:1905:ldlm_handle_bl_callback()) Process entered 00000020:00000040:3.0:1788880833.486368:0:203393:0:(genops.c:900:class_export_put()) PUTting export 00000000ea058c72 : new refcount 2 00010000:00010000:2.0:1788880833.486372:0:203399:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:3.0:1788880833.486372:0:203393:0:(service.c:2635:ptlrpc_server_handle_request()) Process leaving (rc=1 : 1 : 1) 00010000:00010000:2.0:1788880833.486380:0:203399:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock 00000000408d8a8f already unused, calling callback (00000000387296d6) 10000000:00000001:2.0:1788880833.486382:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:2.0:1788880833.486383:0:203399:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880833.486388:0:203399:0:(ldlm_request.c:1770:ldlm_cli_cancel()) Process entered 00010000:00000001:2.0:1788880833.486389:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:2.0:1788880833.486390:0:203399:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880833.486391:0:203399:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=3 00000020:00000001:2.0:1788880833.486393:0:203399:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00000800:00000200:0.0:1788880833.486394:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00010000:00000001:2.0:1788880833.486394:0:203399:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:2.0:1788880833.486397:0:203399:0:(ldlm_request.c:1386:ldlm_cli_cancel_local()) Process entered 00010000:00010000:2.0:1788880833.486400:0:203399:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000010:0.0:1788880833.486401:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:0.0:1788880833.486406:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880833.486414:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 10000000:00000001:2.0:1788880833.486415:0:203399:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:2.0:1788880833.486417:0:203399:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788880833.486420:0:203399:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 10000000:00000001:2.0:1788880833.486422:0:203399:0:(mgc_request.c:719:mgc_requeue_add()) Process entered 10000000:00000040:2.0:1788880833.486423:0:203399:0:(mgc_request.c:723:mgc_requeue_add()) log lustre-cliir: requeue (r=2 sp=0 st=3) 10000000:00000040:2.0:1788880833.486425:0:203399:0:(mgc_request.c:726:mgc_requeue_add()) lockh 0x0 00010000:00000001:2.0:1788880833.486427:0:203399:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000400:00000200:0.0:1788880833.486428:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000003539bb5a 00010000:00000001:2.0:1788880833.486428:0:203399:0:(ldlm_lock.c:617:__ldlm_handle2lock()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880833.486440:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000003539bb5a. 00000100:00000001:0.0:1788880833.486443:0:201279:0:(events.c:386:reply_out_callback()) Process entered 00000100:00000001:0.0:1788880833.486444:0:201279:0:(events.c:398:reply_out_callback()) Process leaving 10000000:00000001:2.0:1788880833.486447:0:203399:0:(mgc_request.c:742:mgc_requeue_add()) Process leaving 10000000:00000001:2.0:1788880833.486450:0:203399:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:2.0:1788880833.486451:0:203399:0:(mgc_request.c:123:config_log_put()) log lustre-cliir(0000000008e74578) refs 2 10000000:00000001:2.0:1788880833.486452:0:203399:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:2.0:1788880833.486453:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880833.486455:0:203399:0:(ldlm_lock.c:2502:ldlm_lock_cancel()) Process entered 00010000:00000001:2.0:1788880833.486458:0:203399:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:2.0:1788880833.486459:0:203399:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:2.0:1788880833.486461:0:203399:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:2.0:1788880833.486462:0:203399:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880833.486464:0:203399:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object 00000000408d8a8f with handle 0x805d548a36a886dd from hash 00010000:00000001:2.0:1788880833.486466:0:203399:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:2.0:1788880833.486468:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880833.486469:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788880833.486470:0:203399:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:2.0:1788880833.486471:0:203399:0:(ldlm_lock.c:2538:ldlm_lock_cancel()) Process leaving 00010000:00000001:2.0:1788880833.486472:0:203399:0:(ldlm_request.c:1422:ldlm_cli_cancel_local()) Process leaving (rc=70368744177664 : 70368744177664 : 400000000000) 00010000:00000001:2.0:1788880833.486474:0:203399:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:2.0:1788880833.486476:0:203399:0:(ldlm_request.c:1527:ldlm_cli_cancel_req()) Process entered 00000100:00000010:2.0:1788880833.486480:0:203399:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000020:00000040:2.0:1788880833.486482:0:203399:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880833.486486:0:203399:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880833.486487:0:203399:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880833.486491:0:203399:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880833.486502:0:203399:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000bef544e8. 00000100:00000001:2.0:1788880833.486507:0:203399:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880833.486510:0:203399:0:(ldlm_request.c:1461:ldlm_cancel_pack()) Process entered 00010000:00010000:2.0:1788880833.486513:0:203399:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788880833.486519:0:203399:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00000001:2.0:1788880833.486521:0:203399:0:(ldlm_request.c:1500:ldlm_cancel_pack()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880833.486524:0:203399:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880833.486526:0:203399:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880833.486532:0:203399:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d838cab4] to pc [ptlrpcd_00_01+1] req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880833.486554:0:203399:0:(ldlm_request.c:1595:ldlm_cli_cancel_req()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:2.0:1788880833.486557:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880833.486558:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788880833.486559:0:203399:0:(ldlm_request.c:2504:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880833.486562:0:203399:0:(ldlm_request.c:1834:ldlm_cli_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880833.486564:0:203399:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880833.486566:0:203399:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:3.0:1788880833.486570:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:2.0:1788880833.486572:0:203399:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00000100:00000001:3.0:1788880833.486575:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880833.486576:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00010000:2.0:1788880833.486576:0:203399:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886dd lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x92f11317f60a7bd3 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:3.0:1788880833.486580:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880833.486581:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000040:2.0:1788880833.486588:0:203399:0:(ldlm_resource.c:1704:ldlm_resource_putref()) putref res: 00000000b9a8b8b1 count: 0 00000100:00000040:3.0:1788880833.486589:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880833.486600:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00010000:00000001:2.0:1788880833.486601:0:203399:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000100:00000001:3.0:1788880833.486603:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880833.486604:0:203399:0:(ldlm_lockd.c:1935:ldlm_handle_bl_callback()) Process leaving 00010000:00000010:2.0:1788880833.486607:0:203399:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at 00000000edd37b97. 02000000:00000001:3.0:1788880833.486610:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00010000:00000001:2.0:1788880833.486612:0:203399:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880833.486614:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000020:00000001:2.0:1788880833.486616:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 02000000:00000001:3.0:1788880833.486618:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880833.486619:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880833.486624:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788880833.486624:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780689152:192.168.204.111@tcp:103: 00000020:00000001:2.0:1788880833.486627:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000100:00000001:3.0:1788880833.486630:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000020:00000001:2.0:1788880833.486635:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880833.486636:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00010000:00010000:2.0:1788880833.486638:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 02000000:00000001:3.0:1788880833.486640:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880833.486642:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880833.486643:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880833.486645:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 02000000:00000001:3.0:1788880833.486651:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880833.486655:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000064b136ca. 02000000:00000001:3.0:1788880833.486656:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880833.486661:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000623a1d39. 00000400:00000010:3.0:1788880833.486665:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000100:00000200:3.0:1788880833.486670:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780689152, portal 18 00000100:00000001:3.0:1788880833.486672:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880833.486673:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000100:00000040:3.0:1788880833.486678:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880833.486685:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880833.486687:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880833.486689:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e78d7264. 00000100:00000200:3.0:1788880833.486691:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 17, xid 1875776780689152, offset 0 00000400:00000200:3.0:1788880833.486696:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880833.486701:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880833.486705:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880833.486709:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880833.486712:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880833.486715:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880833.486718:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [461:461:256:4294967295] 192.168.204.111@tcp LPNI seq info [461:461:8:4294967295] 00000400:00000200:3.0:1788880833.486725:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880833.486729:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880833.486732:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000800:00000200:3.0:1788880833.486735:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880833.486739:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880833.486742:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:3.0:1788880833.486753:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.486756:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306469663 00000100:00000001:3.0:1788880833.486762:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306469554 00000100:00000001:3.0:1788880833.486766:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306469663 00000100:00000001:3.0:1788880833.486775:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306469554 00000100:00000001:3.0:1788880833.486778:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306469663 00000100:00000001:3.0:1788880833.486781:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306469554 00000100:00000001:3.0:1788880833.486784:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306469663 00000100:00000001:3.0:1788880833.486787:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306469554 00000100:00000001:3.0:1788880833.486789:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880833.486791:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.486794:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.486796:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880833.486800:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880833.486801:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880833.486803:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:0.0:1788880833.486805:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000100:00000001:3.0:1788880833.486807:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880833.486810:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880833.486810:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880833.486812:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.486816:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880833.486817:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880833.486823:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e78d7264 00000400:00000010:0.0:1788880833.486825:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e78d7264. 00000100:00000001:0.0:1788880833.486829:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880833.486833:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.486853:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880833.486857:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880833.486865:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880833.486866:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880833.490872:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.490886:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880833.490890:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.490892:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880833.490901:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880833.490914:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 18 MB=0x6aa0250b0df00 00000400:00000200:2.0:1788880833.490922:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 12 from 12345-192.168.204.111@tcp of length 224/224 into md 0xf35 [1] + 224 00000800:00000001:2.0:1788880833.490927:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.490934:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880833.490937:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880833.490940:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880833.490945:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880833.490950:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880833.490961:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880833.490978:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880833.490986:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880833.490989:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.490999:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880833.491003:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880833.491007:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880833.491009:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880833.491010:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000623a1d39. 00000400:00000200:3.0:1788880833.491014:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000400:00000010:3.0:1788880833.491015:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000001:3.0:1788880833.491017:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880833.491020:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880833.491025:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880833.491030:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880833.491031:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.491032:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880833.491034:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880833.491036:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880833.491037:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880833.491038:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880833.491052:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000081111466 time=38 v=5 (5 0 5 0) 00000100:00001000:3.0:1788880833.491056:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=61 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880833.491059:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880833.491060:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880833.491062:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880833.491064:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.491067:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880833.491071:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880833.491078:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880833.491080:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880833.491082:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880833.491085:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880833.491090:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780689152:192.168.204.111@tcp:103: 00000100:00000001:3.0:1788880833.491093:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.491095:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880833.491097:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d838cab4 x1875776780689152/t0(0) o103->MGC192.168.204.111@tcp@192.168.204.111@tcp:17/18 lens 328/224 e 0 to 0 dl 1788880849 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880833.491101:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880833.491103:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880833.491104:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000064b136ca. 02000000:00000001:3.0:1788880833.491107:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880833.491109:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880833.491110:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880833.491111:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880833.491113:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000bef544e8. 02000000:00000001:3.0:1788880833.491115:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880833.491117:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880833.491118:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d838cab4. 00000100:00000001:3.0:1788880833.491120:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880833.491121:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.491122:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880833.491133:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880833.491134:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880833.491136:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880833.491138:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000010:2.1:1788880833.492300:0:0:0:(ldlm_resource.c:1555:__ldlm_resource_free()) slab-freed '(res)': 256 at 00000000b9a8b8b1. 00010000:00000010:2.1:1788880833.492306:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at 00000000408d8a8f. 00010000:00000001:1.0:1788880834.238377:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880834.238387:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880834.238396:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880834.238398:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880834.238466:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880834.238472:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880834.238477:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880835.262362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880835.262371:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880835.262373:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880835.262377:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880835.262382:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880835.262392:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880835.262394:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880835.262407:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880835.262412:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880835.262430:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880835.262497:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880835.262504:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880835.262509:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880836.286353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880836.286360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880836.286366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880836.286368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880836.286423:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880836.286427:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880836.286429:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880837.310318:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880837.310323:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880837.310327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880837.310329:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880837.310331:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880837.310333:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880837.310334:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880837.310336:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880837.310337:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880837.310338:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880837.310404:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880837.310408:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880837.310410:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880838.270328:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880838.270336:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880838.270354:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006a9c2e02. 00000020:00000040:1.0:1788880838.270360:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880838.270364:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880838.270366:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880838.270369:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880838.270374:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:1.0:1788880838.270380:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880838.270388:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880838.270396:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880838.270398:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880838.270407:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006a9c2e02] to pc [ptlrpcd_00_02+2] req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880838.270430:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880838.270431:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880838.270433:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880838.270435:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000020:00000040:1.0:1788880838.270446:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880838.270448:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880838.270449:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880838.270450:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880838.270452:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:1.0:1788880838.270454:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880838.270457:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880838.270462:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880838.270464:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880838.270466:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000077a1dd13] to pc [ptlrpcd_00_03+3] req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880838.270473:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880838.270473:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:1.0:1788880838.270475:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788880838.270479:0:200894:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00000001:0.0:1788880838.270480:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880838.270481:0:200894:0:(import.c:701:ptlrpc_connect_import_locked()) Process entered 00000100:00000001:2.0:1788880838.270481:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00080000:1.0:1788880838.270484:0:200894:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:0.0:1788880838.270485:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880838.270486:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880838.270490:0:200894:0:(import.c:485:import_select_connection()) Process entered 00000100:00000001:2.0:1788880838.270490:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00080000:1.0:1788880838.270494:0:200894:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: connect to NID 192.168.204.111@tcp last attempt 11796 00000100:00000040:0.0:1788880838.270495:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880838.270499:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880838.270502:0:200894:0:(peer.c:1600:LNetPeerDiscovered()) Peer NID 192.168.204.111@tcp is discovered: rc = 1 00000100:00080000:1.0:1788880838.270506:0:200894:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: tried all connections, increasing latency to 20s 00000100:00000040:1.0:1788880838.270511:0:200894:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880838.270516:0:200894:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000001:0.0:1788880838.270521:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880838.270521:0:200894:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880838.270526:0:200894:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000100:00000001:0.0:1788880838.270527:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880838.270528:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000020:00000001:1.0:1788880838.270530:0:200894:0:(genops.c:813:class_conn2export()) Process entered 00000100:00000001:2.0:1788880838.270532:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880838.270533:0:200894:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 02000000:00000001:0.0:1788880838.270537:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000020:00000001:1.0:1788880838.270537:0:200894:0:(lustre_handles.c:129:class_handle2object()) Process entered 02000000:00000001:2.0:1788880838.270539:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880838.270542:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000020:00000040:1.0:1788880838.270542:0:200894:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 02000000:00000001:2.0:1788880838.270543:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000020:00000001:1.0:1788880838.270547:0:200894:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 02000000:00000001:0.0:1788880838.270548:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880838.270549:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880838.270555:0:200894:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000100:00100000:0.0:1788880838.270556:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006a9c2e02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780689280:192.168.204.111@tcp:400: 00000100:00100000:2.0:1788880838.270557:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780689408:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880838.270560:0:200894:0:(lustre_net.h:1975:ptlrpc_connection_put()) PUT conn=000000008a5be7d5 refcount 11 to 192.168.204.111@tcp 00000100:00000001:1.0:1788880838.270564:0:200894:0:(connection.c:156:ptlrpc_connection_addref()) Process entered 00000100:00000001:2.0:1788880838.270567:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880838.270569:0:200894:0:(connection.c:161:ptlrpc_connection_addref()) conn=000000008a5be7d5 refcount 12 to 192.168.204.111@tcp 00000100:00000001:0.0:1788880838.270573:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880838.270573:0:200894:0:(connection.c:163:ptlrpc_connection_addref()) Process leaving (rc=18446617250405033856 : -126823304517760 : ffff8ca7a5fc2780) 00000020:00000040:1.0:1788880838.270575:0:200894:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 02000000:00000001:2.0:1788880838.270575:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880838.270579:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00080000:1.0:1788880838.270580:0:200894:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff8ca787527000: import 00000000b98b8a4e using connection 192.168.204.111@tcp/192.168.204.111@tcp 02000000:00000001:2.0:1788880838.270581:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880838.270582:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880838.270584:0:200894:0:(import.c:628:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880838.270587:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:0.0:1788880838.270589:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:1.0:1788880838.270589:0:200894:0:(sec.c:1588:sptlrpc_import_sec_adapt()) Process entered 02000000:00000010:2.0:1788880838.270593:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000744e90e2. 02000000:00000010:0.0:1788880838.270595:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b476d059. 02000000:00000001:1.0:1788880838.270595:0:200894:0:(sec.c:1628:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880838.270597:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880838.270599:0:200894:0:(sec.c:1656:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880838.270600:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880838.270603:0:200894:0:(obd_class.h:889:obd_reconnect()) Process entered 00000008:00100000:1.0:1788880838.270606:0:200894:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682648 ocd_grant: 2097152, lost: 0. 00000400:00000010:2.0:1788880838.270606:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b8697e0a. 00000400:00000010:0.0:1788880838.270607:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006f1b1f6b. 00000008:00000001:1.0:1788880838.270612:0:200894:0:(osc_request.c:3832:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880838.270615:0:200894:0:(obd_class.h:903:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880838.270617:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f4d84419. 00000400:00000010:2.0:1788880838.270617:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d208f26. 00000100:00000010:1.0:1788880838.270618:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000dc99d6ba. 00000100:00000200:0.0:1788880838.270626:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780689280, portal 25 00000100:00000200:2.0:1788880838.270626:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780689408, portal 10 00000020:00000040:1.0:1788880838.270627:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880838.270632:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880838.270633:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1788880838.270635:0:200894:0:(sec.c:2066:sptlrpc_sepol_get()) Process entered 00000100:00000001:0.0:1788880838.270636:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938058624 : -126823771492992 : ffff8ca78a26ad80) 02000000:00000001:1.0:1788880838.270637:0:200894:0:(sec.c:2076:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880838.270637:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000001:1.0:1788880838.270643:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880838.270645:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:0.0:1788880838.270646:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880854 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880838.270647:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880838.270649:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880854 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880838.270652:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at 00000000216a9979. 00000100:00000001:1.0:1788880838.270657:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880838.270666:0:200894:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880838.270676:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880838.270678:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880838.270682:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:0.0:1788880838.270684:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880838.270685:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880838.270686:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880838.270690:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000dc289a80. 00000100:00000040:1.0:1788880838.270690:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000dc99d6ba] to pc [ptlrpcd_rcv+-1] req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880838.270692:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000032b7316b. 00000100:00000200:0.0:1788880838.270695:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780689280, offset 0 00000100:00000200:2.0:1788880838.270698:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780689408, offset 0 00000400:00000200:0.0:1788880838.270709:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880838.270718:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880838.270721:0:200894:0:(import.c:833:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880838.270723:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:1.0:1788880838.270728:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:0.0:1788880838.270729:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788880838.270733:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:0.0:1788880838.270737:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:1.0:1788880838.270738:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11807) 00000400:00000200:0.0:1788880838.270745:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880838.270749:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880838.270753:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [462:462:256:4294967295] 192.168.204.111@tcp LPNI seq info [462:462:8:4294967295] 00000400:00000200:0.0:1788880838.270766:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880838.270767:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880838.270771:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880838.270774:0:201284:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:0.0:1788880838.270777:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880838.270778:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:3.0:1788880838.270783:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880838.270784:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:0.0:1788880838.270786:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000005f53e16f. 00000400:00000200:2.0:1788880838.270789:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880838.270794:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788880838.270799:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880838.270801:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880838.270807:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [463:463:255:4294967295] 192.168.204.111@tcp LPNI seq info [463:463:7:4294967295] 00000100:00000001:3.0:1788880838.270808:0:201284:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880838.270811:0:201284:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880838.270813:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:3.0:1788880838.270816:0:201284:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880838.270819:0:201284:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880838.270821:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000005f53e16f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880838.270824:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:3.0:1788880838.270825:0:201284:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880838.270829:0:201284:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000dc99d6ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780689536:192.168.204.111@tcp:38: 00000100:00000001:3.0:1788880838.270836:0:201284:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880838.270840:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880838.270840:0:201284:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880838.270843:0:201284:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880838.270844:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880838.270847:0:201284:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000010:2.0:1788880838.270851:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000100:00000001:0.0:1788880838.270852:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306474447 02000000:00000010:3.0:1788880838.270853:0:201284:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000f8da91be. 02000000:00000001:3.0:1788880838.270858:0:201284:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880838.270860:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:3.0:1788880838.270862:0:201284:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000623a1d39. 00000100:00000001:0.0:1788880838.270863:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306469663 00000800:00000200:2.0:1788880838.270869:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:3.0:1788880838.270870:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000100:00000001:0.0:1788880838.270871:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306474447 00000800:00000200:2.0:1788880838.270873:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000200:3.0:1788880838.270875:0:201284:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780689536, portal 10 00000100:00000001:0.0:1788880838.270877:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306469663 00000100:00000001:2.0:1788880838.270879:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.270879:0:201284:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880838.270883:0:201284:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938052352 : -126823771499264 : ffff8ca78a269500) 00000100:00000001:0.0:1788880838.270884:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306474447 00000100:00000040:2.0:1788880838.270886:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 188781 usecs (189 jiffies) 00000100:00000040:3.0:1788880838.270891:0:201284:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880863 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880838.270892:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306469663 00000100:00000001:2.0:1788880838.270892:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306469565 00000800:00000200:1.0:1788880838.270895:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880838.270899:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306474447 00000100:00000001:0.0:1788880838.270900:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306474447 00000800:00000010:1.0:1788880838.270900:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000005f53e16f. 00000100:00000001:2.0:1788880838.270905:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306469565 00000100:00000001:0.0:1788880838.270908:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306469663 00000400:00000200:1.0:1788880838.270908:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880838.270912:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306474447 00000100:00000001:0.0:1788880838.270915:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880838.270916:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306469565 00000400:00000200:1.0:1788880838.270919:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880838.270921:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880838.270923:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306474447 00000100:00000001:0.0:1788880838.270927:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880838.270927:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000dc289a80 00000100:00000001:2.0:1788880838.270930:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306469565 00000400:00000010:1.0:1788880838.270931:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000dc289a80. 00000100:00000001:0.0:1788880838.270933:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880838.270937:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306474447 00000100:00000001:1.0:1788880838.270938:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880838.270938:0:201284:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880838.270943:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880838.270943:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788880838.270945:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880854 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880838.270948:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:3.0:1788880838.270948:0:201284:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880838.270950:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880838.270954:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880838.270954:0:201284:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a165c7f9. 00000100:00000001:2.0:1788880838.270955:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880838.270959:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:3.0:1788880838.270960:0:201284:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 12, xid 1875776780689536, offset 0 00000100:00000001:2.0:1788880838.270961:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880838.270962:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880838.270964:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880838.270969:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788880838.270970:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880854 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880838.270973:0:201284:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880838.270974:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880838.270977:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880838.270980:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880838.270983:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880838.270984:0:201284:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880838.270986:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880838.270992:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880838.270993:0:201284:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880838.270997:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880838.270999:0:201284:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880838.271002:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:3.0:1788880838.271004:0:201284:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880838.271007:0:201284:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880838.271011:0:201284:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [464:464:254:4294967295] 192.168.204.111@tcp LPNI seq info [464:464:6:4294967295] 00000400:00000200:3.0:1788880838.271018:0:201284:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880838.271023:0:201284:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880838.271026:0:201284:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000800:00000200:3.0:1788880838.271030:0:201284:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880838.271034:0:201284:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880838.271036:0:201284:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 616 niov 1 nkiov 1 00000100:00000001:3.0:1788880838.271039:0:201284:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880838.271041:0:201284:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13445 usecs (14 jiffies) 00000100:00000001:3.0:1788880838.271044:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306469390 00000100:00000001:3.0:1788880838.271046:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306474447 00000100:00000001:3.0:1788880838.271048:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306469390 00000100:00000001:3.0:1788880838.271049:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306474447 00000100:00000001:3.0:1788880838.271051:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306469390 00000100:00000001:3.0:1788880838.271052:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306474447 00000100:00000001:3.0:1788880838.271054:0:201284:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306469390 00000100:00000001:3.0:1788880838.271055:0:201284:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306474447 00000100:00000001:3.0:1788880838.271062:0:201284:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880838.271063:0:201284:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.271065:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.271067:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880838.271070:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880838.271072:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:3.0:1788880838.271074:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:1.0:1788880838.271075:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880838.271075:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880838.271077:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880838.271079:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000100:00000001:3.0:1788880838.271080:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880838.271082:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880838.271087:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880838.271089:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000032b7316b 00000400:00000010:1.0:1788880838.271090:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000032b7316b. 00000100:00000001:1.0:1788880838.271093:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880838.271095:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880854 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880838.271101:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880838.271102:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880854 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880838.271107:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880838.271109:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880838.271125:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880838.271127:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:1.0:1788880838.271130:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880838.271133:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880838.271135:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a165c7f9 00000400:00000010:1.0:1788880838.271137:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a165c7f9. 00000100:00000001:1.0:1788880838.271139:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880838.271142:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880863 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880838.271149:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880838.271151:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880863 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880838.271157:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880838.271158:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880838.272496:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880838.272504:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880838.272506:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880838.272508:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880838.272512:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880838.272520:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0df80 00000400:00000200:0.0:1788880838.272524:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xf41 [1] + 224 00000800:00000001:0.0:1788880838.272529:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880838.272534:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880838.272535:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880838.272537:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880838.272539:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880838.272542:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880838.272557:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880838.272568:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880838.272574:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880838.272577:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880838.272586:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880838.272590:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880838.272594:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880838.272596:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880838.272598:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006f1b1f6b. 00000400:00000200:3.0:1788880838.272601:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f4d84419 00000400:00000010:3.0:1788880838.272603:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f4d84419. 00000100:00000001:3.0:1788880838.272605:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880838.272664:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880838.272671:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880838.272676:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880838.272678:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880838.272680:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880838.272682:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880838.272684:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880838.272685:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880838.272686:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880838.272690:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=66 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880838.272693:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=66 v=5 (1 1 1 1) 00000800:00000001:2.0:1788880838.272695:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880838.272696:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880838.272698:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880838.272700:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880838.272702:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880838.272704:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880838.272704:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880838.272708:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880838.272711:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880838.272716:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880838.272729:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0e000 00000400:00000200:2.0:1788880838.272734:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xf3d [1] + 224 00000800:00000001:2.0:1788880838.272738:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880838.272742:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880838.272743:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880838.272745:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880838.272747:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880838.272749:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880838.272755:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880838.272766:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880838.272769:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880838.272771:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880838.272789:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880838.272796:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880838.272800:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880838.272800:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880838.272802:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880838.272804:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880838.272808:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880838.272810:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880838.272812:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880838.272814:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b8697e0a. 00000400:00000200:1.0:1788880838.272820:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d208f26 00000100:00100000:3.0:1788880838.272822:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006a9c2e02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780689280:192.168.204.111@tcp:400: 00000400:00000010:1.0:1788880838.272823:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d208f26. 00000100:00000001:1.0:1788880838.272829:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880838.272831:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880838.272834:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:1.0:1788880838.272835:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880838.272839:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006a9c2e02 x1875776780689280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880838.272858:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880838.272858:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880838.272862:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880838.272866:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b476d059. 00000100:00000001:1.0:1788880838.272867:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880838.272870:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880838.272872:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880838.272872:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:1.0:1788880838.272875:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:3.0:1788880838.272876:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880838.272877:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880838.272880:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880838.272880:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 02000000:00000001:1.0:1788880838.272883:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880838.272886:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00001000:1.0:1788880838.272888:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=56 v=5 (5 5 5 5) 02000000:00000010:3.0:1788880838.272890:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 00000100:00001000:1.0:1788880838.272892:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=56 v=5 (1 1 1 1) 02000000:00000001:3.0:1788880838.272896:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880838.272898:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880838.272900:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880838.272902:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880838.272902:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006a9c2e02. 00010000:00000001:1.0:1788880838.272906:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880838.272909:0:201288:0:(ldlm_request.c:1723:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.272909:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880838.272913:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880838.272914:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880838.272916:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880838.272917:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880838.272919:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880838.272921:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880838.272925:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880838.272925:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880838.272929:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.272934:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880838.272935:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880838.272938:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880838.272938:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880838.272939:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880838.272944:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880838.272949:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780689408:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880838.272952:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880838.272953:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880838.272954:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780689408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880854 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880838.272958:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880838.272959:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880838.272960:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000744e90e2. 02000000:00000001:1.0:1788880838.272962:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880838.272963:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880838.272963:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880838.272965:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880838.272965:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 02000000:00000001:1.0:1788880838.272967:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880838.272968:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880838.272968:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:1.0:1788880838.272970:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880838.272971:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880838.272981:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880838.272983:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880838.272984:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880838.272985:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880838.272986:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880838.275254:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880838.275261:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880838.275264:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880838.275266:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880838.275273:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880838.275282:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 416 into portal 10 MB=0x6aa0250b0e080 00000400:00000200:1.0:1788880838.275287:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 416/416 into md 0xf4d [1] + 0 00000800:00000001:1.0:1788880838.275291:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880838.275296:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880838.275299:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880838.275302:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880838.275306:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880838.275311:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880863 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880838.275321:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=416 offset=0 replen=544 req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880863 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880838.275335:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880838.275352:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880838.275356:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880838.275367:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880838.275370:0:201284:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880838.275373:0:201284:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880838.275374:0:201284:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880838.275381:0:201284:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000623a1d39. 00000400:00000200:3.0:1788880838.275384:0:201284:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000400:00000010:3.0:1788880838.275385:0:201284:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000001:3.0:1788880838.275387:0:201284:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880838.275390:0:201284:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880863 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880838.275396:0:201284:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/544 e 0 to 0 dl 1788880863 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880838.275401:0:201284:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880838.275402:0:201284:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880838.275403:0:201284:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880838.275405:0:201284:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880838.275406:0:201284:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880838.275408:0:201284:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880838.275409:0:201284:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880838.275413:0:201284:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=66 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880838.275416:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=23 v=20 (20 1 1 0) 00000100:00000001:3.0:1788880838.275419:0:201284:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880838.275420:0:201284:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880838.275421:0:201284:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880838.275424:0:201284:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880863 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880838.275431:0:201284:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.275433:0:201284:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880838.275435:0:201284:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880838.275436:0:201284:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880838.275437:0:201284:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880838.275440:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880863 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880838.275447:0:201284:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880838.275449:0:201284:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880838.275450:0:201284:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880838.275452:0:201284:0:(import.c:1057:ptlrpc_connect_interpret()) Process entered 00000100:00080000:3.0:1788880838.275462:0:201284:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0001-mdc-ffff8ca787527000: connect to target with instance 6 00000020:00000001:3.0:1788880838.275465:0:201284:0:(genops.c:813:class_conn2export()) Process entered 00000020:00000040:3.0:1788880838.275466:0:201284:0:(genops.c:825:class_conn2export()) looking for export cookie 0x805d548a36a886a5 00000020:00000001:3.0:1788880838.275467:0:201284:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:3.0:1788880838.275469:0:201284:0:(lustre_handles.c:147:class_handle2object()) GET export 00000000673f59bf refcount=5 00000020:00000001:3.0:1788880838.275470:0:201284:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000020:00000001:3.0:1788880838.275472:0:201284:0:(genops.c:827:class_conn2export()) Process leaving (rc=18446617249948774400 : -126823760777216 : ffff8ca78aca3000) 00000100:00001000:3.0:1788880838.275474:0:201284:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=1788880838 v=0 (0 0 0 0) 00000100:00001000:3.0:1788880838.275476:0:201284:0:(import.c:2215:obd_at_measure()) AT 0000000040f48a4f change: old=0 new=5 delta=5 (val=1) hist 1 0 0 0 00000100:00000040:3.0:1788880838.275478:0:201284:0:(obd_cksum.h:77:obd_cksum_types_supported_client()) Crypto hash speed: crc 1651, crc32c 9963, adler 1371 00000020:00000040:3.0:1788880838.275482:0:201284:0:(genops.c:900:class_export_put()) PUTting export 00000000673f59bf : new refcount 4 00000100:00000001:3.0:1788880838.275483:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 00000002:00000001:3.0:1788880838.275485:0:201284:0:(mdc_request.c:2879:mdc_import_event()) Process entered 00000008:00000020:3.0:1788880838.275488:0:201284:0:(osc_request.c:1062:osc_init_grant()) lustre-MDT0001-mdc-ffff8ca787527000, setting cl_avail_grant: 2097152 cl_lost_grant: 0. chunk bits: 12 cl_max_extent_pages: 16384 00000002:00000001:3.0:1788880838.275490:0:201284:0:(mdc_request.c:3107:mdc_init_ea_size()) Process entered 00000002:00000001:3.0:1788880838.275491:0:201284:0:(mdc_request.c:3114:mdc_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880838.275493:0:201284:0:(obd_class.h:1243:obd_notify()) Process entered 00800000:00000001:3.0:1788880838.275495:0:201284:0:(lmv_obd.c:188:lmv_notify()) Process entered 00000080:00000001:3.0:1788880838.275497:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:3.0:1788880838.275499:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:3.0:1788880838.275500:0:201284:0:(lmv_obd.c:223:lmv_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880838.275501:0:201284:0:(obd_class.h:1256:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880838.275502:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:3.0:1788880838.275502:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880838.275504:0:201284:0:(mdc_request.c:2953:mdc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.275505:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00080000:3.0:1788880838.275507:0:201284:0:(import.c:65:import_set_state_nolock()) 00000000b98b8a4e lustre-MDT0001_UUID: changing import state from CONNECTING to FULL 00000100:00000001:3.0:1788880838.275509:0:201284:0:(obd_class.h:1223:obd_import_event()) Process entered 00000002:00000001:3.0:1788880838.275509:0:201284:0:(mdc_request.c:2879:mdc_import_event()) Process entered 00000002:00000001:3.0:1788880838.275510:0:201284:0:(obd_class.h:1243:obd_notify()) Process entered 00800000:00000001:3.0:1788880838.275511:0:201284:0:(lmv_obd.c:188:lmv_notify()) Process entered 00800000:00000001:3.0:1788880838.275511:0:201284:0:(lmv_obd.c:82:lmv_set_mdc_active()) Process entered 00800000:00000040:3.0:1788880838.275512:0:201284:0:(lmv_obd.c:85:lmv_set_mdc_active()) Searching in lmv 00000000b999beb9 for uuid lustre-MDT0001_UUID (activate=1) 00800000:00000040:3.0:1788880838.275514:0:201284:0:(lmv_obd.c:91:lmv_set_mdc_active()) Target idx 0 is lustre-MDT0000_UUID conn 0x805d548a36a8869e 00800000:00000040:3.0:1788880838.275515:0:201284:0:(lmv_obd.c:91:lmv_set_mdc_active()) Target idx 1 is lustre-MDT0001_UUID conn 0x805d548a36a886a5 00800000:00000040:3.0:1788880838.275520:0:201284:0:(lmv_obd.c:106:lmv_set_mdc_active()) Found OBD lustre-MDT0001-mdc-ffff8ca787527000=ea27006f-52a8-4007-b69a-dcac8279445a device 4 (000000001b5d8280) type mdc at LMV idx 1 00800000:00000040:3.0:1788880838.275522:0:201284:0:(lmv_obd.c:111:lmv_set_mdc_active()) OBD 000000001b5d8280 already active! 00800000:00000001:3.0:1788880838.275523:0:201284:0:(lmv_obd.c:112:lmv_set_mdc_active()) Process leaving via out_lmv_lock (rc=0 : 0 : 0x0) 00000080:00000001:3.0:1788880838.275524:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:3.0:1788880838.275525:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00800000:00000001:3.0:1788880838.275526:0:201284:0:(lmv_obd.c:223:lmv_notify()) Process leaving (rc=0 : 0 : 0) 00000002:00000001:3.0:1788880838.275527:0:201284:0:(obd_class.h:1256:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:3.0:1788880838.275527:0:201284:0:(llite_lib.c:393:ll_upcall_notify()) Process entered 00000080:00000001:3.0:1788880838.275528:0:201284:0:(llite_lib.c:418:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880838.275529:0:201284:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=5 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000002:00000001:3.0:1788880838.275680:0:201284:0:(mdc_request.c:2953:mdc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.275681:0:201284:0:(obd_class.h:1234:obd_import_event()) Process leaving 00000100:00000001:3.0:1788880838.275683:0:201284:0:(import.c:1648:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00000001:3.0:1788880838.275685:0:201284:0:(import.c:1754:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.275687:0:201284:0:(import.c:1517:ptlrpc_connect_interpret()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880838.275690:0:201284:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880863 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000002:00000001:1.0:1788880838.275698:0:207982:0:(mdc_request.c:2638:mdc_kuc_reregister_thread()) Process entered 00000100:00100000:3.0:1788880838.275699:0:201284:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000dc99d6ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:ea27006f-52a8-4007-b69a-dcac8279445a:201284:1875776780689536:192.168.204.111@tcp:38: 00000020:00000001:1.0:1788880838.275702:0:207982:0:(kernelcomm.c:1325:libcfs_kkuc_group_foreach()) Process entered 00000100:00000001:3.0:1788880838.275705:0:201284:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880838.275707:0:207982:0:(kernelcomm.c:1339:libcfs_kkuc_group_foreach()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.275709:0:201284:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000020:00000001:1.0:1788880838.275711:0:207982:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:3.0:1788880838.275714:0:201284:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000dc99d6ba x1875776780689536/t0(0) o38->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 520/416 e 0 to 0 dl 1788880863 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:1.0:1788880838.275715:0:207982:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880838.275720:0:207982:0:(genops.c:1251:class_import_put()) Process leaving 00000002:00000001:1.0:1788880838.275723:0:207982:0:(mdc_request.c:2648:mdc_kuc_reregister_thread()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.275730:0:201284:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880838.275732:0:201284:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880838.275734:0:201284:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000f8da91be. 02000000:00000001:3.0:1788880838.275737:0:201284:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880838.275739:0:201284:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880838.275740:0:201284:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880838.275748:0:201284:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880838.275749:0:201284:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at 00000000216a9979. 02000000:00000001:3.0:1788880838.275751:0:201284:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880838.275752:0:201284:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880838.275758:0:201284:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000dc99d6ba. 00000100:00000001:3.0:1788880838.275760:0:201284:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880838.275761:0:201284:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880838.275762:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880838.275766:0:201284:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880838.275767:0:201284:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880838.275768:0:201284:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880838.275769:0:201284:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880838.334335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880838.334349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880838.334356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880838.334373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880838.334428:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880838.334432:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880838.334435:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880839.358338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880839.358343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880839.358348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880839.358349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880839.358401:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880839.358405:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880839.358407:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:1.0:1788880839.806321:0:203394:0:(service.c:1710:ptlrpc_at_check_timed()) Process entered 00000100:00000001:1.0:1788880839.806328:0:203394:0:(service.c:1721:ptlrpc_at_check_timed()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.318370:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880840.318378:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-cliir (0000000008e74578) refs 2 10000000:00000001:1.0:1788880840.318383:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.318385:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000001:1.0:1788880840.318387:0:203401:0:(mgc_request.c:120:config_log_put()) Process leaving 10000000:00000001:1.0:1788880840.318388:0:203401:0:(mgc_request.c:601:do_requeue()) Process entered 10000000:01000000:1.0:1788880840.318390:0:203401:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:00000001:1.0:1788880840.318392:0:203401:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:1.0:1788880840.318393:0:203401:0:(mgc_request.c:1801:mgc_process_log()) Process log lustre-cliir-ffff8ca787527000 from 14 10000000:00000001:1.0:1788880840.318396:0:203401:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:1.0:1788880840.318397:0:203401:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00000100:00000010:1.0:1788880840.318409:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000020:00000040:1.0:1788880840.318413:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880840.318417:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880840.318421:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880840.318426:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880840.318433:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000f5272de5. 00000100:00000001:1.0:1788880840.318440:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.318444:0:203401:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:1.0:1788880840.318446:0:203401:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:1.0:1788880840.318453:0:203401:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at 000000005636a4b0. 00010000:00000001:1.0:1788880840.318456:0:203401:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:1.0:1788880840.318465:0:203401:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 00000000408d8a8f. 00000020:00000001:1.0:1788880840.318468:0:203401:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:1.0:1788880840.318472:0:203401:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000408d8a8f with handle 0x805d548a36a886f2 to hash 00000020:00000001:1.0:1788880840.318473:0:203401:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:1.0:1788880840.318474:0:203401:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880840.318477:0:203401:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880840.318481:0:203401:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1788880840.318497:0:203401:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880840.318500:0:203401:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886f2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880840.318508:0:203401:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886f2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880840.318517:0:203401:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886f2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880840.318521:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880840.318522:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880840.318528:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000081730d7. 00000100:00000001:1.0:1788880840.318530:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250052433024 : -126823657118592 : ffff8ca790f7e480) 00000100:00000001:1.0:1788880840.318532:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.318533:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000001:1.0:1788880840.318537:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880840.318539:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.318541:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880840.318542:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880840.318551:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.318558:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880840.318560:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.318564:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880840.318566:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880840.318568:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880840.318570:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780689664:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880840.318575:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880840.318582:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880840.318585:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.318587:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880840.318593:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000002a02a4fa. 02000000:00000001:1.0:1788880840.318595:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880840.318607:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000400:00000010:1.0:1788880840.318615:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000200:1.0:1788880840.318622:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780689664, portal 25 00000100:00000001:1.0:1788880840.318624:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.318625:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000040:1.0:1788880840.318630:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.318637:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880840.318640:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880840.318643:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000100:00000200:1.0:1788880840.318646:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780689664, offset 0 00000400:00000200:1.0:1788880840.318652:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880840.318668:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880840.318674:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880840.318679:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880840.318683:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880840.318686:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880840.318689:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [465:465:256:4294967295] 192.168.204.111@tcp LPNI seq info [465:465:8:4294967295] 00000400:00000200:1.0:1788880840.318701:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880840.318707:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880840.318713:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000749ed09e. 00000800:00000200:1.0:1788880840.318721:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880840.318729:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880840.318734:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000749ed09e type 1, nob 424 niov 1 nkiov 1 00000100:00000001:1.0:1788880840.318754:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.318759:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306476495 00000100:00000001:1.0:1788880840.318767:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306474447 00000100:00000001:1.0:1788880840.318774:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306476495 00000100:00000001:1.0:1788880840.318777:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306474447 00000100:00000001:1.0:1788880840.318780:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306476495 00000100:00000001:1.0:1788880840.318788:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306474447 00000100:00000001:1.0:1788880840.318791:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306476495 00000100:00000001:1.0:1788880840.318795:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306474447 00000100:00000001:1.0:1788880840.318798:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.318801:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.318803:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880840.318806:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880840.318808:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 00000000081730d7 going to sleep for 11 seconds 00000100:00000001:1.0:1788880840.318812:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880840.318815:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880840.318918:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880840.318922:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000749ed09e. 00000400:00000200:0.0:1788880840.318928:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.318933:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880840.318938:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000400:00000010:0.0:1788880840.318939:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00000100:00000001:0.0:1788880840.318943:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880840.318946:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.318954:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880840.318956:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.318961:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880840.318963:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880840.320401:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.320415:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880840.320418:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.320419:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880840.320426:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880840.320437:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b0e100 00000400:00000200:2.0:1788880840.320443:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0xf55 [1] + 224 00000800:00000001:2.0:1788880840.320449:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.320454:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880840.320456:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880840.320458:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880840.320462:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880840.320465:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880840.320472:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880840.320497:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880840.320506:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.320510:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.320531:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880840.320539:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880840.320541:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880840.320543:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:1.0:1788880840.320547:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:1.0:1788880840.320549:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:1.0:1788880840.320552:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880840.320555:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880840.320562:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.320567:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880840.320568:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.320570:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880840.320572:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880840.320574:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880840.320576:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.320578:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880840.320583:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=68 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880840.320586:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=68 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880840.320588:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880840.320589:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.320591:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880840.320593:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.320600:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880840.320603:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.320608:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.320610:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880840.320612:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880840.320615:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880840.320622:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780689664:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880840.320627:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.320630:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.320633:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880840.320634:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.320637:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.320642:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880840.320645:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000081730d7. 00000100:00000001:1.0:1788880840.320648:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880840.320649:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.320652:0:203401:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:1.0:1788880840.320655:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880840.320657:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880840.320660:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=4 00000020:00000001:1.0:1788880840.320662:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880840.320665:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000040:1.0:1788880840.320671:0:203401:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 00000000408d8a8f, remote cookie: 0x92f11317f60a7d54, flags: 0x0 00010000:00000001:1.0:1788880840.320674:0:203401:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:1.0:1788880840.320677:0:203401:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:1.0:1788880840.320679:0:203401:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:1.0:1788880840.320680:0:203401:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:1.0:1788880840.320681:0:203401:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:1.0:1788880840.320683:0:203401:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x2:0x0].0x0 (000000005636a4b0) refcount = 1 00010000:00010000:1.0:1788880840.320691:0:203401:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886f2 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7d54 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880840.320696:0:203401:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:1.0:1788880840.320698:0:203401:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:1.0:1788880840.320699:0:203401:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788880840.320701:0:203401:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:1.0:1788880840.320703:0:203401:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880840.320705:0:203401:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886f2 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7d54 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880840.320709:0:203401:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:1.0:1788880840.320710:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.320711:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880840.320712:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.320712:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880840.320713:0:203401:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.320715:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.320717:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780689664/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.320720:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880840.320722:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880840.320724:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000002a02a4fa. 02000000:00000001:1.0:1788880840.320726:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880840.320728:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.320729:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.320731:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880840.320733:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000f5272de5. 02000000:00000001:1.0:1788880840.320736:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880840.320737:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880840.320738:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:1.0:1788880840.320741:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880840.320742:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:1.0:1788880840.320743:0:203401:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.320745:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880840.320746:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-cliir (0000000008e74578) refs 3 10000000:00000001:1.0:1788880840.320750:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.320752:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880840.320753:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880840.320754:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=3 00000020:00000001:1.0:1788880840.320755:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880840.320756:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880840.320757:0:203401:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:1.0:1788880840.320758:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.320759:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880840.320759:0:203401:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.320761:0:203401:0:(mgc_request.c:1519:mgc_process_recover_log()) Process entered 10000000:00000010:1.0:1788880840.320764:0:203401:0:(mgc_request.c:1531:mgc_process_recover_log()) kmalloced '(folios)': 32 at 000000008e2a7097. 00000100:00000010:1.0:1788880840.320774:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000020:00000040:1.0:1788880840.320776:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880840.320778:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880840.320780:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880840.320783:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880840.320785:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000f5272de5. 00000100:00000001:1.0:1788880840.320790:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.320794:0:203401:0:(client.c:227:ptlrpc_prep_bulk_imp()) Process entered 00000100:00000010:1.0:1788880840.320797:0:203401:0:(client.c:169:ptlrpc_new_bulk()) kmalloced '(desc)': 984 at 000000002a02a4fa. 00000100:00000010:1.0:1788880840.320800:0:203401:0:(client.c:174:ptlrpc_new_bulk()) kmalloced '(desc->bd_vec)': 64 at 000000000a91cda9. 00000020:00000040:1.0:1788880840.320802:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880840.320806:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880840.320807:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880840.320811:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000081730d7. 00000100:00000001:1.0:1788880840.320813:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250052433024 : -126823657118592 : ffff8ca790f7e480) 00000100:00000001:1.0:1788880840.320815:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.320816:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000001:1.0:1788880840.320819:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880840.320820:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.320822:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880840.320823:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880840.320826:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.320844:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880840.320846:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.320850:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880840.320852:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880840.320854:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880840.320856:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780689792:192.168.204.111@tcp:256: 00000100:00000001:1.0:1788880840.320859:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880840.320864:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880840.320866:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.320868:0:203401:0:(niobuf.c:346:ptlrpc_register_bulk()) Process entered 00000400:00000010:1.0:1788880840.320874:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000033e76d7c. 00000400:00000010:1.0:1788880840.320879:0:203401:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 00000000fc2ea56f. 00000100:00000200:1.0:1788880840.320884:0:203401:0:(niobuf.c:453:ptlrpc_register_bulk()) Setup 1 bulk put-sink buffers: 4 pages 16384 bytes, mbits x0x6aa0250b0e180-0x6aa0250b0e180, portal 33 00000100:00000001:1.0:1788880840.320887:0:203401:0:(niobuf.c:455:ptlrpc_register_bulk()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.320889:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880840.320892:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005309c375. 02000000:00000001:1.0:1788880840.320893:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880840.320896:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dc9bac80. 00000400:00000010:1.0:1788880840.320898:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000200:1.0:1788880840.320901:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780689792, portal 25 00000100:00000001:1.0:1788880840.320903:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.320904:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000040:1.0:1788880840.320908:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.320914:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880840.320917:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880840.320919:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000002fa6ff6. 00000100:00000200:1.0:1788880840.320921:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 304 bytes to portal 26, xid 1875776780689792, offset 0 00000400:00000200:1.0:1788880840.320926:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880840.320934:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880840.320939:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880840.320946:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880840.320949:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880840.320951:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880840.320953:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [466:466:256:4294967295] 192.168.204.111@tcp LPNI seq info [466:466:8:4294967295] 00000400:00000200:1.0:1788880840.320959:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880840.320963:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 304 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880840.320966:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000749ed09e. 00000800:00000200:1.0:1788880840.320970:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880840.320974:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880840.320976:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000749ed09e type 1, nob 400 niov 1 nkiov 1 00000100:00000001:1.0:1788880840.320991:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.320994:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306476497 00000100:00000001:1.0:1788880840.320999:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306476495 00000100:00000001:1.0:1788880840.321003:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306476497 00000100:00000001:1.0:1788880840.321005:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306476495 00000100:00000001:1.0:1788880840.321006:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306476497 00000100:00000001:1.0:1788880840.321008:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306476495 00000100:00000001:1.0:1788880840.321009:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306476497 00000100:00000001:1.0:1788880840.321011:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306476495 00000100:00000001:1.0:1788880840.321012:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.321014:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.321016:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880840.321017:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880840.321018:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 00000000081730d7 going to sleep for 11 seconds 00000100:00000001:1.0:1788880840.321020:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880840.321021:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880840.321080:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880840.321085:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000749ed09e. 00000400:00000200:0.0:1788880840.321089:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.321094:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880840.321097:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000002fa6ff6 00000400:00000010:0.0:1788880840.321099:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000002fa6ff6. 00000100:00000001:0.0:1788880840.321102:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880840.321106:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.321115:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880840.321118:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.321137:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880840.321139:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880840.322094:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.322102:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880840.322105:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.322107:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.322113:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880840.322122:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 44 into portal 33 MB=0x6aa0250b0e180 00000400:00000200:0.0:1788880840.322128:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 21 from 12345-192.168.204.111@tcp of length 44/44 into md 0xf5d [4] + 0 00000400:00000010:0.0:1788880840.322131:0:201280:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000033e76d7c. 00000400:00000200:0.0:1788880840.322134:0:201280:0:(lib-md.c:42:lnet_md_unlink()) Queueing unlink of md 00000000fc2ea56f 00000800:00000001:0.0:1788880840.322137:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.322142:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.322144:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.322147:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880840.322150:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000fc2ea56f 00000400:00000010:0.0:1788880840.322151:0:201280:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 00000000fc2ea56f. 00000100:00000001:0.0:1788880840.322154:0:201280:0:(events.c:174:client_bulk_callback()) Process entered 00000100:00000200:0.0:1788880840.322156:0:201280:0:(events.c:192:client_bulk_callback()) event type 2, status 0, req 0000000077a1dd13 desc 000000002a02a4fa mbits 1875776780689792 00000100:00000001:0.0:1788880840.322165:0:201280:0:(events.c:217:client_bulk_callback()) Process leaving 00000400:00000200:0.0:1788880840.322169:0:201280:0:(lib-md.c:96:lnet_cpt_of_md()) Couldn't resolve first page of md 0000000000000000 with offset 0 00000400:00000200:0.0:1788880840.322174:0:201280:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source Specified: 192.168.204.11@tcp to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880840.322181:0:201280:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [467:467:256:4294967295] 192.168.204.111@tcp LPNI seq info [467:467:8:4294967295] 00000100:00000001:1.0:1788880840.322189:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880840.322193:0:201280:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:192.168.204.11@tcp) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) 192.168.204.111@tcp : ACK try# 0 00000100:00000001:1.0:1788880840.322193:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880840.322201:0:201280:0:(socklnd_cb.c:931:ksocknal_send()) sending 0 bytes in 4 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880840.322205:0:201280:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 296 at 00000000a2bc2248. 00000800:00000200:0.0:1788880840.322209:0:201280:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880840.322212:0:201280:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880840.322214:0:201280:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000a2bc2248 type 0, nob 96 niov 1 nkiov 0 00000800:00000001:0.0:1788880840.322274:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000200:2.0:1788880840.322319:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880840.322322:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 296 at 00000000a2bc2248. 00000400:00000200:2.0:1788880840.322325:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880840.322330:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: ACK: OK 00000800:00000001:0.0:1788880840.322345:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880840.322992:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.322998:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880840.322999:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.323001:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.323004:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880840.323012:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 240 into portal 25 MB=0x6aa0250b0e180 00000400:00000200:0.0:1788880840.323026:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 240/240 into md 0xf61 [1] + 224 00000800:00000001:0.0:1788880840.323031:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.323036:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.323038:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.323041:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880840.323044:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880840.323049:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880840.323057:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=240 offset=224 replen=240 req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.323073:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880840.323078:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.323084:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.323111:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880840.323117:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880840.323126:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880840.323128:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dc9bac80. 00000400:00000200:1.0:1788880840.323131:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:1.0:1788880840.323133:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:1.0:1788880840.323136:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880840.323140:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880840.323149:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.323156:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880840.323157:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.323160:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880840.323162:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880840.323165:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880840.323167:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.323170:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880840.323175:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=68 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880840.323179:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=68 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880840.323181:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880840.323183:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.323185:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880840.323187:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.323189:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880840.323193:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Bulk req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880840.323203:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Bulk to Interpret req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 2 fl Bulk:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.323210:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880840.323212:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880840.323215:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880840.323222:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780689792:192.168.204.111@tcp:256: 00000100:00000001:1.0:1788880840.323257:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.323259:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.323262:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880840.323263:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.323267:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.323274:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880840.323276:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000081730d7. 00000100:00000001:1.0:1788880840.323279:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880840.323281:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:1.0:1788880840.323286:0:203401:0:(mgc_request.c:1599:mgc_process_recover_log()) Latest version 14, more 0. 10000000:00000001:1.0:1788880840.323290:0:203401:0:(mgc_request.c:1308:mgc_apply_recover_logs()) Process entered 10000000:00000010:1.0:1788880840.323295:0:203401:0:(mgc_request.c:1331:mgc_apply_recover_logs()) kmalloced '(nidlist)': 640 at 00000000216a9979. 10000000:00000010:1.0:1788880840.323299:0:203401:0:(mgc_request.c:1343:mgc_apply_recover_logs()) kmalloced '(obdname)': 128 at 00000000bac254ae. 10000000:00000200:1.0:1788880840.323315:0:203401:0:(mgc_request.c:1241:mgc_build_nidlist()) use 1 from 1 NIDs 10000000:00000040:1.0:1788880840.323319:0:203401:0:(mgc_request.c:1440:mgc_apply_recover_logs()) Looking for OBD lustre-MDT0001-mdc-ffff8ca787527000 00010000:00000001:1.0:1788880840.323327:0:203401:0:(ldlm_lib.c:159:client_import_add_nids_to_conn()) Process entered 00000100:00000001:1.0:1788880840.323330:0:203401:0:(lustre_peer.c:229:class_check_uuid()) Process entered 00000100:00000040:1.0:1788880840.323334:0:203401:0:(lustre_peer.c:232:class_check_uuid()) check if uuid 192.168.204.111@tcp has 192.168.204.111@tcp. 00000100:00000001:1.0:1788880840.323336:0:203401:0:(lustre_peer.c:251:class_check_uuid()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.323338:0:203401:0:(lustre_peer.c:189:class_add_nids_to_uuid()) Process entered 00000100:00000200:1.0:1788880840.323340:0:203401:0:(lustre_peer.c:194:class_add_nids_to_uuid()) Comparing 192.168.204.111@tcp with 192.168.204.111@tcp 00000100:00000200:1.0:1788880840.323342:0:203401:0:(lustre_peer.c:201:class_add_nids_to_uuid()) Updating UUID '192.168.204.111@tcp' 00000400:00000200:1.0:1788880840.323344:0:203401:0:(peer.c:1363:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:1.0:1788880840.323347:0:203401:0:(api-ni.c:3274:LNetNIInit()) refs 1 00000400:00000200:1.0:1788880840.323351:0:203401:0:(peer.c:1377:LNetAddPeer()) Adding NID[0]: 192.168.204.111@tcp 00000400:00000200:1.0:1788880840.323355:0:203401:0:(peer.c:1822:lnet_peer_add()) peer 192.168.204.111@tcp NID flags 0x100001: -17 00000400:00000200:1.0:1788880840.323362:0:203401:0:(peer.c:2890:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:1.0:1788880840.323365:0:203401:0:(peer.c:2899:lnet_discover_peer_locked()) peer 192.168.204.111@tcp NID 192.168.204.111@tcp: 0. pending discovery 00000400:00000200:1.0:1788880840.323367:0:203401:0:(peer.c:1431:LNetAddPeer()) A peer exists for NID 192.168.204.111@tcp 00000100:00000001:1.0:1788880840.323370:0:203401:0:(lustre_peer.c:219:class_add_nids_to_uuid()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.323372:0:203401:0:(ldlm_lib.c:174:client_import_add_nids_to_conn()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:1.0:1788880840.323375:0:203401:0:(mgc_request.c:1486:mgc_apply_recover_logs()) ir apply logs 14/14 for lustre-MDT0001-mdc-ffff8ca787527000 10000000:00000010:1.0:1788880840.323378:0:203401:0:(mgc_request.c:1491:mgc_apply_recover_logs()) kfreed 'obdname': 128 at 00000000bac254ae. 10000000:00000010:1.0:1788880840.323386:0:203401:0:(mgc_request.c:1493:mgc_apply_recover_logs()) kfreed 'nidlist': 640 at 00000000216a9979. 10000000:00000001:1.0:1788880840.323389:0:203401:0:(mgc_request.c:1495:mgc_apply_recover_logs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.323391:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.323394:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780689792/t0(0) o256->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 304/240 e 0 to 0 dl 1788880856 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.323401:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880840.323403:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880840.323405:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005309c375. 02000000:00000001:1.0:1788880840.323408:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880840.323409:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.323411:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.323414:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880840.323415:0:203401:0:(client.c:312:ptlrpc_free_bulk()) Process entered 00000020:00000001:1.0:1788880840.323417:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.323418:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.323420:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000010:1.0:1788880840.323423:0:203401:0:(client.c:333:ptlrpc_free_bulk()) kvfree 'desc->bd_vec': 64 at 000000000a91cda9. 00000100:00000010:1.0:1788880840.323425:0:203401:0:(client.c:334:ptlrpc_free_bulk()) kfreed 'desc': 984 at 000000002a02a4fa. 00000100:00000001:1.0:1788880840.323427:0:203401:0:(client.c:335:ptlrpc_free_bulk()) Process leaving 02000000:00000010:1.0:1788880840.323429:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000f5272de5. 02000000:00000001:1.0:1788880840.323432:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880840.323434:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880840.323436:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:1.0:1788880840.323438:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880840.323440:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000010:1.0:1788880840.323455:0:203401:0:(mgc_request.c:1651:mgc_process_recover_log()) kvfree 'folios': 32 at 000000008e2a7097. 10000000:01000000:1.0:1788880840.323467:0:203401:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'lustre-cliir' succeeded (0). 10000000:00000001:1.0:1788880840.323471:0:203401:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:1.0:1788880840.323474:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880840.323476:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880840.323479:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000408d8a8f refcount=3 00000020:00000001:1.0:1788880840.323481:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880840.323484:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280344064 : -126823429207552 : ffff8ca79e8d8a00) 00010000:00000001:1.0:1788880840.323488:0:203401:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:1.0:1788880840.323492:0:203401:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886f2 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7d54 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880840.323504:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.323505:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:1.0:1788880840.323506:0:203401:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 00000000408d8a8f/0x805d548a36a886f2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7d54 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880840.323513:0:203401:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:1.0:1788880840.323514:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.323515:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:1.0:1788880840.323517:0:203401:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.323519:0:203401:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.323522:0:203401:0:(mgc_request.c:621:do_requeue()) Process leaving 10000000:00000001:1.0:1788880840.323524:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880840.323526:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-client (00000000c2974351) refs 2 10000000:00000001:1.0:1788880840.323527:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.323537:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:1.0:1788880840.323538:0:203401:0:(mgc_request.c:123:config_log_put()) log lustre-cliir(0000000008e74578) refs 3 10000000:00000001:1.0:1788880840.323541:0:203401:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:1.0:1788880840.323542:0:203401:0:(mgc_request.c:601:do_requeue()) Process entered 10000000:01000000:1.0:1788880840.323544:0:203401:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:00000001:1.0:1788880840.323546:0:203401:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:1.0:1788880840.323549:0:203401:0:(mgc_request.c:1801:mgc_process_log()) Process log lustre-client-ffff8ca787527000 from 43 10000000:00000001:1.0:1788880840.323552:0:203401:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:1.0:1788880840.323553:0:203401:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00000100:00000010:1.0:1788880840.323558:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000020:00000040:1.0:1788880840.323562:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880840.323565:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880840.323568:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880840.323572:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880840.323577:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000f5272de5. 00000100:00000001:1.0:1788880840.323582:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.323586:0:203401:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:1.0:1788880840.323589:0:203401:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:1.0:1788880840.323595:0:203401:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at 000000008fce9a29. 00010000:00000001:1.0:1788880840.323600:0:203401:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:1.0:1788880840.323602:0:203401:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 00000000c2b00415. 00000020:00000001:1.0:1788880840.323611:0:203401:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:1.0:1788880840.323613:0:203401:0:(lustre_handles.c:93:class_handle_hash()) added object 00000000c2b00415 with handle 0x805d548a36a886f9 to hash 00000020:00000001:1.0:1788880840.323615:0:203401:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:1.0:1788880840.323616:0:203401:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617250280345600 : -126823429206016 : ffff8ca79e8d9000) 00010000:00000001:1.0:1788880840.323619:0:203401:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617250280345600 : -126823429206016 : ffff8ca79e8d9000) 00010000:00000001:1.0:1788880840.323622:0:203401:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1788880840.323626:0:203401:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880840.323628:0:203401:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 00000000c2b00415/0x805d548a36a886f9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880840.323635:0:203401:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 00000000c2b00415/0x805d548a36a886f9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880840.323649:0:203401:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 00000000c2b00415/0x805d548a36a886f9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788880840.323656:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880840.323658:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880840.323662:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000081730d7. 00000100:00000001:1.0:1788880840.323664:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250052433024 : -126823657118592 : ffff8ca790f7e480) 00000100:00000001:1.0:1788880840.323666:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.323667:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000001:1.0:1788880840.323671:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880840.323673:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.323675:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880840.323677:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880840.323680:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.323687:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880840.323689:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.323693:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880840.323695:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880840.323698:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880840.323706:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780689920:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880840.323710:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880840.323715:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880840.323717:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.323719:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880840.323723:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 000000002a02a4fa. 02000000:00000001:1.0:1788880840.323724:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880840.323728:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dc9bac80. 00000400:00000010:1.0:1788880840.323732:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000200:1.0:1788880840.323737:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780689920, portal 25 00000100:00000001:1.0:1788880840.323739:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.323740:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000040:1.0:1788880840.323744:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.323750:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880840.323753:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880840.323755:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 00000100:00000200:1.0:1788880840.323758:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780689920, offset 0 00000400:00000200:1.0:1788880840.323763:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880840.323770:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880840.323775:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880840.323779:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880840.323782:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880840.323791:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880840.323794:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [468:468:256:4294967295] 192.168.204.111@tcp LPNI seq info [468:468:8:4294967295] 00000400:00000200:1.0:1788880840.323803:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880840.323808:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880840.323812:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000800:00000200:1.0:1788880840.323817:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880840.323822:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880840.323824:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 424 niov 1 nkiov 1 00000100:00000001:1.0:1788880840.323844:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.323848:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306476500 00000100:00000001:1.0:1788880840.323854:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306476497 00000100:00000001:1.0:1788880840.323859:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306476500 00000100:00000001:1.0:1788880840.323869:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306476497 00000100:00000001:1.0:1788880840.323873:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306476500 00000100:00000001:1.0:1788880840.323877:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306476497 00000100:00000001:1.0:1788880840.323880:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306476500 00000100:00000001:1.0:1788880840.323883:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306476497 00000100:00000001:1.0:1788880840.323886:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.323889:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.323891:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880840.323893:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880840.323896:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 00000000081730d7 going to sleep for 11 seconds 00000100:00000001:1.0:1788880840.323898:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880840.323900:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880840.323926:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880840.323930:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000400:00000200:2.0:1788880840.323934:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880840.323939:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880840.323942:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000400:00000010:2.0:1788880840.323943:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000001:2.0:1788880840.323947:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880840.323950:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880840.323959:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880840.323961:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880840.323968:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880840.323969:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880840.324961:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.324977:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880840.324980:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.324983:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.324993:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880840.325003:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b0e200 00000400:00000200:0.0:1788880840.325009:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0xf69 [1] + 224 00000800:00000001:0.0:1788880840.325014:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.325020:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.325022:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.325025:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880840.325035:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880840.325040:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880840.325049:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.325065:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880840.325072:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.325075:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.325098:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880840.325104:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880840.325105:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880840.325107:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dc9bac80. 00000400:00000200:1.0:1788880840.325110:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:1.0:1788880840.325111:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:1.0:1788880840.325114:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880840.325117:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880840.325124:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.325130:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880840.325132:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.325134:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880840.325136:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880840.325138:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880840.325146:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.325148:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880840.325153:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=68 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880840.325156:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=68 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880840.325158:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880840.325160:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.325162:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880840.325164:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.325166:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880840.325169:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.325175:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.325178:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880840.325179:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880840.325182:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880840.325188:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780689920:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880840.325192:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.325194:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.325195:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880840.325197:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.325199:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.325204:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880840.325206:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000081730d7. 00000100:00000001:1.0:1788880840.325209:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880840.325210:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.325212:0:203401:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:1.0:1788880840.325214:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880840.325216:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880840.325218:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000c2b00415 refcount=4 00000020:00000001:1.0:1788880840.325220:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280345600 : -126823429206016 : ffff8ca79e8d9000) 00010000:00000001:1.0:1788880840.325222:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280345600 : -126823429206016 : ffff8ca79e8d9000) 00010000:00000040:1.0:1788880840.325230:0:203401:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 00000000c2b00415, remote cookie: 0x92f11317f60a7d5b, flags: 0x0 00010000:00000001:1.0:1788880840.325255:0:203401:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:1.0:1788880840.325260:0:203401:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:1.0:1788880840.325261:0:203401:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:1.0:1788880840.325263:0:203401:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:1.0:1788880840.325264:0:203401:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:1.0:1788880840.325266:0:203401:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x0:0x0].0x0 (000000008fce9a29) refcount = 1 00010000:00010000:1.0:1788880840.325270:0:203401:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 00000000c2b00415/0x805d548a36a886f9 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7d5b expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880840.325276:0:203401:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:1.0:1788880840.325278:0:203401:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:1.0:1788880840.325279:0:203401:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788880840.325283:0:203401:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:1.0:1788880840.325285:0:203401:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880840.325287:0:203401:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 00000000c2b00415/0x805d548a36a886f9 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7d5b expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880840.325292:0:203401:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:1.0:1788880840.325294:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.325295:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880840.325296:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.325297:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880840.325298:0:203401:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.325312:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.325316:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780689920/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.325322:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880840.325323:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880840.325325:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 000000002a02a4fa. 02000000:00000001:1.0:1788880840.325328:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880840.325329:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.325331:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.325333:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880840.325335:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000f5272de5. 02000000:00000001:1.0:1788880840.325342:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880840.325344:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880840.325345:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:1.0:1788880840.325349:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880840.325350:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:1.0:1788880840.325351:0:203401:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.325353:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880840.325355:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-client (00000000c2974351) refs 3 10000000:00000001:1.0:1788880840.325356:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.325358:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880840.325359:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880840.325361:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000c2b00415 refcount=3 00000020:00000001:1.0:1788880840.325362:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280345600 : -126823429206016 : ffff8ca79e8d9000) 00010000:00000001:1.0:1788880840.325364:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280345600 : -126823429206016 : ffff8ca79e8d9000) 00010000:00000001:1.0:1788880840.325366:0:203401:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:1.0:1788880840.325368:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.325369:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880840.325370:0:203401:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.325372:0:203401:0:(mgc_request.c:1668:mgc_process_cfg_log()) Process entered 10000000:00000010:1.0:1788880840.325375:0:203401:0:(mgc_request.c:1682:mgc_process_cfg_log()) kmalloced '(env)': 56 at 000000000a91cda9. 00000020:00000010:1.0:1788880840.325379:0:203401:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000f5272de5. 00000020:00000010:1.0:1788880840.325384:0:203401:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000005309c375. 00000040:00000010:1.0:1788880840.325388:0:203401:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 0000000099f3ac40. 00000020:00000010:1.0:1788880840.325393:0:203401:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000a71f1a29. 00000020:00000010:1.0:1788880840.325397:0:203401:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000852b7c70. 80000000:00000010:1.0:1788880840.325402:0:203401:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000b245c5fd. 00040000:00000010:1.0:1788880840.325408:0:203401:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 0000000056df9207. 20000000:00000010:1.0:1788880840.325416:0:203401:0:(mgs_handler.c:1787:mgs_key_init()) kmalloced '(value)': 392 at 000000009c91be43. 10000000:00000040:1.0:1788880840.325421:0:203401:0:(lustre_log.h:387:llog_ctxt_get()) GETting ctxt 000000006d5d0799 : new refcount 2 00000020:00000001:1.0:1788880840.325425:0:203401:0:(obd_config.c:2120:class_config_parse_llog()) Process entered 00000020:00000040:1.0:1788880840.325427:0:203401:0:(obd_config.c:2122:class_config_parse_llog()) looking up llog lustre-client 00000040:00000001:1.0:1788880840.325429:0:203401:0:(llog.c:1538:llog_open()) Process entered 00000040:00000010:1.0:1788880840.325433:0:203401:0:(llog.c:42:llog_alloc_handle()) kmalloced '(loghandle)': 352 at 000000002b77e71a. 00000040:00000001:1.0:1788880840.325437:0:203401:0:(llog_client.c:70:llog_client_open()) Process entered 00000020:00000040:1.0:1788880840.325439:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788880840.325442:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000020:00000040:1.0:1788880840.325450:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880840.325454:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880840.325456:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880840.325459:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880840.325463:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000659bd78d. 00000100:00000001:1.0:1788880840.325466:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.325471:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880840.325472:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880840.325475:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000081730d7. 00000100:00000001:1.0:1788880840.325477:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250052433024 : -126823657118592 : ffff8ca790f7e480) 00000100:00000001:1.0:1788880840.325479:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.325479:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000001:1.0:1788880840.325483:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880840.325484:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.325486:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880840.325487:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880840.325490:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.325497:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880840.325499:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.325501:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880840.325503:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880840.325505:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880840.325507:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780690048:192.168.204.111@tcp:501: 00000100:00000001:1.0:1788880840.325511:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880840.325515:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880840.325516:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.325518:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880840.325521:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005bd099ee. 02000000:00000001:1.0:1788880840.325523:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880840.325526:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dc9bac80. 00000400:00000010:1.0:1788880840.325530:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000200:1.0:1788880840.325534:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780690048, portal 25 00000100:00000001:1.0:1788880840.325536:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.325541:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000040:1.0:1788880840.325545:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.325553:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880840.325555:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880840.325557:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e249bdf9. 00000100:00000200:1.0:1788880840.325559:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 512 bytes to portal 26, xid 1875776780690048, offset 0 00000400:00000200:1.0:1788880840.325564:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880840.325572:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880840.325576:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880840.325579:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880840.325581:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880840.325583:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880840.325586:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [469:469:256:4294967295] 192.168.204.111@tcp LPNI seq info [469:469:8:4294967295] 00000400:00000200:1.0:1788880840.325595:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880840.325600:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 512 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880840.325604:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000800:00000200:1.0:1788880840.325609:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880840.325614:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880840.325617:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 608 niov 1 nkiov 1 00000100:00000001:1.0:1788880840.325630:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.325633:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306476502 00000100:00000001:1.0:1788880840.325639:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306476500 00000100:00000001:1.0:1788880840.325643:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306476502 00000100:00000001:1.0:1788880840.325647:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306476500 00000100:00000001:1.0:1788880840.325650:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306476502 00000100:00000001:1.0:1788880840.325653:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306476500 00000100:00000001:1.0:1788880840.325656:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306476502 00000100:00000001:1.0:1788880840.325659:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306476500 00000100:00000001:1.0:1788880840.325667:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.325670:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.325672:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880840.325674:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880840.325676:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 00000000081730d7 going to sleep for 11 seconds 00000100:00000001:1.0:1788880840.325678:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880840.325680:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880840.325718:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880840.325722:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000400:00000200:0.0:1788880840.325726:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.325731:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880840.325734:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e249bdf9 00000400:00000010:0.0:1788880840.325735:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e249bdf9. 00000100:00000001:0.0:1788880840.325739:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880840.325742:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.325750:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880840.325753:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.325759:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880840.325761:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880840.326814:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.326822:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880840.326823:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.326825:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880840.326830:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880840.326838:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 272 into portal 25 MB=0x6aa0250b0e280 00000400:00000200:2.0:1788880840.326842:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 272/272 into md 0xf71 [1] + 224 00000800:00000001:2.0:1788880840.326846:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.326849:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880840.326851:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880840.326853:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880840.326861:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880840.326864:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880840.326872:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=272 offset=224 replen=272 req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880840.326887:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880840.326894:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.326897:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.326922:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880840.326927:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880840.326928:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880840.326930:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dc9bac80. 00000400:00000200:1.0:1788880840.326933:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:1.0:1788880840.326934:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:1.0:1788880840.326936:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880840.326939:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880840.326944:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.326949:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880840.326950:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.326952:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880840.326953:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880840.326954:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880840.326955:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.326957:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880840.326960:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=68 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880840.326962:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=68 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880840.326964:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880840.326965:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.326966:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880840.326967:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.326969:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880840.326971:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.326975:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.326983:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880840.326984:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880840.326986:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880840.326991:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780690048:192.168.204.111@tcp:501: 00000100:00000001:1.0:1788880840.326993:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.326995:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.326997:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880840.326998:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.327000:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.327004:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880840.327005:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000081730d7. 00000100:00000001:1.0:1788880840.327006:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880840.327007:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880840.327010:0:203401:0:(llog_client.c:122:llog_client_open()) Process leaving 00000020:00000001:1.0:1788880840.327012:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.327013:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.327014:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880840.327015:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.327016:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780690048/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.327020:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880840.327021:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880840.327022:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005bd099ee. 02000000:00000001:1.0:1788880840.327024:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880840.327024:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.327025:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.327026:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880840.327027:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000659bd78d. 02000000:00000001:1.0:1788880840.327029:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880840.327030:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880840.327031:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:1.0:1788880840.327033:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880840.327036:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000040:00000001:1.0:1788880840.327038:0:203401:0:(llog.c:1563:llog_open()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880840.327041:0:203401:0:(llog.c:415:llog_init_handle()) Process entered 00000040:00000010:1.0:1788880840.327046:0:203401:0:(llog.c:420:llog_init_handle()) kmalloced '(llh)': 8192 at 00000000c3c95438. 00000040:00000001:1.0:1788880840.327048:0:203401:0:(llog.c:363:llog_read_header()) Process entered 00000040:00000001:1.0:1788880840.327049:0:203401:0:(llog_client.c:269:llog_client_read_header()) Process entered 00000020:00000040:1.0:1788880840.327051:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788880840.327053:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000020:00000040:1.0:1788880840.327054:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880840.327057:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880840.327058:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880840.327060:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880840.327063:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 00000000659bd78d. 00000100:00000001:1.0:1788880840.327066:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.327067:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880840.327068:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880840.327071:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000081730d7. 00000100:00000001:1.0:1788880840.327072:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250052433024 : -126823657118592 : ffff8ca790f7e480) 00000100:00000001:1.0:1788880840.327074:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.327075:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000001:1.0:1788880840.327077:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880840.327078:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.327079:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880840.327080:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880840.327082:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.327085:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880840.327086:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.327089:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880840.327090:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880840.327091:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880840.327093:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780690176:192.168.204.111@tcp:503: 00000100:00000001:1.0:1788880840.327096:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880840.327098:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880840.327099:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.327103:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880840.327116:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 16384 at 00000000664a6a04. 02000000:00000001:1.0:1788880840.327118:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880840.327120:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dc9bac80. 00000400:00000010:1.0:1788880840.327123:0:203401:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 00000000cd8de53b. 00000100:00000200:1.0:1788880840.327126:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 16384 bytes, xid 1875776780690176, portal 25 00000100:00000001:1.0:1788880840.327128:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.327128:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000040:1.0:1788880840.327135:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.327140:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880840.327141:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880840.327143:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000200:1.0:1788880840.327145:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 272 bytes to portal 26, xid 1875776780690176, offset 0 00000400:00000200:1.0:1788880840.327148:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880840.327154:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880840.327156:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880840.327158:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880840.327160:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880840.327164:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880840.327166:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [470:470:256:4294967295] 192.168.204.111@tcp LPNI seq info [470:470:8:4294967295] 00000400:00000200:1.0:1788880840.327172:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880840.327175:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 272 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880840.327178:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000800:00000200:1.0:1788880840.327181:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880840.327185:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880840.327186:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 368 niov 1 nkiov 1 00000100:00000001:1.0:1788880840.327195:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.327198:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306476503 00000100:00000001:1.0:1788880840.327207:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306476502 00000100:00000001:1.0:1788880840.327211:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306476503 00000100:00000001:1.0:1788880840.327220:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306476502 00000100:00000001:1.0:1788880840.327223:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306476503 00000100:00000001:1.0:1788880840.327225:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306476502 00000100:00000001:1.0:1788880840.327228:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306476503 00000100:00000001:1.0:1788880840.327230:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306476502 00000100:00000001:1.0:1788880840.327252:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.327255:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.327258:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880840.327259:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880840.327261:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 00000000081730d7 going to sleep for 11 seconds 00000100:00000001:1.0:1788880840.327263:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880840.327265:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880840.327309:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880840.327312:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:0.0:1788880840.327314:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.327318:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880840.327322:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:0.0:1788880840.327323:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:0.0:1788880840.327326:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880840.327329:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.327337:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880840.327345:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880856 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.327351:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880840.327352:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880840.328456:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.328464:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880840.328466:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.328468:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.328473:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880840.328486:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 8416 into portal 25 MB=0x6aa0250b0e300 00000400:00000200:0.0:1788880840.328493:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 8416/8416 into md 0xf79 [4] + 224 00000800:00000001:0.0:1788880840.328497:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.328503:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.328505:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.328507:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880840.328509:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880840.328512:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880856 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880840.328518:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=8416 offset=224 replen=8416 req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.328529:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880840.328537:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.328540:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.328567:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880840.328571:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880840.328573:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880840.328574:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dc9bac80. 00000400:00000200:1.0:1788880840.328576:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cd8de53b 00000400:00000010:1.0:1788880840.328577:0:203401:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 00000000cd8de53b. 00000100:00000001:1.0:1788880840.328579:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880840.328582:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880840.328586:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.328590:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880840.328591:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.328593:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880840.328594:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880840.328596:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880840.328597:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.328599:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880840.328602:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=68 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880840.328604:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=68 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880840.328605:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880840.328611:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.328613:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880840.328614:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.328615:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880840.328618:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.328622:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.328625:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880840.328626:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880840.328627:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880856 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880840.328631:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780690176:192.168.204.111@tcp:503: 00000100:00000001:1.0:1788880840.328634:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.328635:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.328637:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880840.328638:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.328639:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880856 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.328642:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880840.328643:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000081730d7. 00000100:00000001:1.0:1788880840.328645:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880840.328646:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880840.328658:0:203401:0:(llog_client.c:319:llog_client_read_header()) Process leaving 00000100:00000001:1.0:1788880840.328659:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.328661:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780690176/t0(0) o503->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8416 e 0 to 0 dl 1788880856 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.328667:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880840.328668:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880840.328669:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 16384 at 00000000664a6a04. 02000000:00000001:1.0:1788880840.328679:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880840.328680:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.328681:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.328683:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880840.328687:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 00000000659bd78d. 02000000:00000001:1.0:1788880840.328689:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880840.328689:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880840.328691:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:1.0:1788880840.328694:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880840.328696:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880840.328698:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.328699:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.328700:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000040:00000001:1.0:1788880840.328702:0:203401:0:(llog.c:403:llog_read_header()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880840.328704:0:203401:0:(llog.c:481:llog_init_handle()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880840.328708:0:203401:0:(llog.c:951:llog_process_or_fork()) Process entered 00000040:00000010:1.0:1788880840.328711:0:203401:0:(llog.c:953:llog_process_or_fork()) kmalloced '(lpi)': 96 at 00000000d9f2f285. 00000040:00001000:1.0:1788880840.328713:0:203401:0:(llog.c:968:llog_process_or_fork()) Processing [0xa:0x3:0x0] flags 0x004 startcat -1 startidx -1 first_idx 42 last_idx 0 read_mode 0 00000040:00000001:1.0:1788880840.328951:0:207999:0:(llog.c:552:llog_process_thread()) Process entered 00000020:00000010:1.0:1788880840.328956:0:207999:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 00000000659bd78d. 00000020:00000010:1.0:1788880840.328961:0:207999:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 000000005bd099ee. 00000040:00000010:1.0:1788880840.328964:0:207999:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000123d40c9. 00000020:00000010:1.0:1788880840.328968:0:207999:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000a3e174ee. 00000020:00000010:1.0:1788880840.328973:0:207999:0:(tgt_main.c:721:tgt_key_init()) slab-alloced 'thread': 1288 at 000000008f8293fd. 00000020:00000010:1.0:1788880840.328976:0:207999:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 0000000025bc9394. 80000000:00000010:1.0:1788880840.328979:0:207999:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000cf4d1652. 40000000:00000010:1.0:1788880840.328982:0:207999:0:(fid_handler.c:479:seq_key_init()) kmalloced '(value)': 48 at 00000000b96c0748. 00100000:00000010:1.0:1788880840.328986:0:207999:0:(lfsck_lib.c:33:lfsck_key_init()) kmalloced '(value)': 1800 at 000000005257fdb2. 00040000:00000010:1.0:1788880840.328990:0:207999:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 00000000d7d1e0d5. 00040000:00000010:1.0:1788880840.328994:0:207999:0:(qmt_dev.c:418:qmt_key_init()) kmalloced '(value)': 1256 at 00000000453588fe. 00040000:00000010:1.0:1788880840.328997:0:207999:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 00000000e7b82f46. 00000004:00000010:1.0:1788880840.329002:0:207999:0:(mdd_device.c:2408:mdd_key_init()) kmalloced '(value)': 1872 at 000000007f7d1e46. 00000004:00000010:1.0:1788880840.329007:0:207999:0:(mdt_handler.c:8470:mdt_key_init()) kmalloced '(value)': 2568 at 0000000013189f47. 00000004:00000010:1.0:1788880840.329012:0:207999:0:(lod_dev.c:2352:lod_key_init()) kmalloced '(value)': 1224 at 000000006f45b8ae. 00002000:00000010:1.0:1788880840.329015:0:207999:0:(ofd_dev.c:3002:ofd_key_init()) kmalloced '(value)': 816 at 000000002a02a4fa. 00000004:00000010:1.0:1788880840.329019:0:207999:0:(osp_dev.c:1888:osp_key_init()) kmalloced '(value)': 616 at 00000000216a9979. 00000020:00000010:1.0:1788880840.329022:0:207999:0:(lu_object.c:2013:lu_env_add_task()) kmalloced '(lei)': 40 at 000000000cca38e9. 00000040:00000010:1.0:1788880840.329030:0:207999:0:(llog.c:578:llog_process_thread()) kmalloced '(buf)': 8192 at 000000000d09cd28. 00000040:00001000:1.0:1788880840.329033:0:207999:0:(llog.c:615:llog_process_thread()) index: 43 last_index 64767 00000040:00000001:1.0:1788880840.329039:0:207999:0:(lustre_log.h:475:llog_next_block()) Process entered 00000040:00000001:1.0:1788880840.329041:0:207999:0:(llog_client.c:140:llog_client_next_block()) Process entered 00000020:00000040:1.0:1788880840.329042:0:207999:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788880840.329045:0:207999:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000020:00000040:1.0:1788880840.329047:0:207999:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880840.329049:0:207999:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880840.329051:0:207999:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880840.329053:0:207999:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880840.329056:0:207999:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000000fc6edce. 00000100:00000001:1.0:1788880840.329059:0:207999:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.329062:0:207999:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880840.329063:0:207999:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880840.329065:0:207999:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000cd8de53b. 00000100:00000001:1.0:1788880840.329067:0:207999:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250052434560 : -126823657117056 : ffff8ca790f7ea80) 00000100:00000001:1.0:1788880840.329068:0:207999:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.329069:0:207999:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000001:1.0:1788880840.329072:0:207999:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880840.329073:0:207999:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.329074:0:207999:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880840.329075:0:207999:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880840.329078:0:207999:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.329083:0:207999:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880840.329084:0:207999:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.329086:0:207999:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880840.329088:0:207999:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880840.329089:0:207999:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880840.329091:0:207999:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job llog_process_th:fa4856d5-51a8-499e-b3f8-b477743a3dec:207999:1875776780690304:192.168.204.111@tcp:502: 00000100:00000001:1.0:1788880840.329093:0:207999:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880840.329096:0:207999:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880840.329097:0:207999:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.329099:0:207999:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880840.329107:0:207999:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 16384 at 00000000664a6a04. 02000000:00000001:1.0:1788880840.329108:0:207999:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880840.329113:0:207999:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dc9bac80. 00000400:00000010:1.0:1788880840.329116:0:207999:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at 00000000e64ce9e0. 00000100:00000200:1.0:1788880840.329120:0:207999:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 16384 bytes, xid 1875776780690304, portal 25 00000100:00000001:1.0:1788880840.329121:0:207999:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.329122:0:207999:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000040:1.0:1788880840.329125:0:207999:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.329130:0:207999:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880840.329131:0:207999:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880840.329132:0:207999:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000200:1.0:1788880840.329134:0:207999:0:(niobuf.c:94:ptl_send_buf()) Sending 272 bytes to portal 26, xid 1875776780690304, offset 0 00000400:00000200:1.0:1788880840.329137:0:207999:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880840.329144:0:207999:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880840.329148:0:207999:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880840.329151:0:207999:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880840.329153:0:207999:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880840.329155:0:207999:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880840.329157:0:207999:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [471:471:256:4294967295] 192.168.204.111@tcp LPNI seq info [471:471:8:4294967295] 00000400:00000200:1.0:1788880840.329162:0:207999:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880840.329167:0:207999:0:(socklnd_cb.c:931:ksocknal_send()) sending 272 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880840.329169:0:207999:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000800:00000200:1.0:1788880840.329173:0:207999:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880840.329177:0:207999:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880840.329179:0:207999:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 368 niov 1 nkiov 1 00000100:00000001:1.0:1788880840.329188:0:207999:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.329190:0:207999:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306476505 00000100:00000001:1.0:1788880840.329194:0:207999:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306476503 00000100:00000001:1.0:1788880840.329198:0:207999:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306476505 00000100:00000001:1.0:1788880840.329200:0:207999:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306476503 00000100:00000001:1.0:1788880840.329202:0:207999:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306476505 00000100:00000001:1.0:1788880840.329208:0:207999:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306476503 00000100:00000001:1.0:1788880840.329211:0:207999:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306476505 00000100:00000001:1.0:1788880840.329214:0:207999:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306476503 00000100:00000001:1.0:1788880840.329217:0:207999:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.329220:0:207999:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.329222:0:207999:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880840.329224:0:207999:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880840.329226:0:207999:0:(client.c:2702:ptlrpc_set_wait()) set 00000000cd8de53b going to sleep for 11 seconds 00000100:00000001:1.0:1788880840.329228:0:207999:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880840.329230:0:207999:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880840.329290:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880840.329295:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000400:00000200:2.0:1788880840.329299:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880840.329315:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880840.329319:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000400:00000010:2.0:1788880840.329321:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000100:00000001:2.0:1788880840.329324:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880840.329327:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880840.329336:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880840.329339:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880856 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880840.329345:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880840.329347:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880840.330468:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.330478:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880840.330482:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.330485:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.330492:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880840.330503:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 8472 into portal 25 MB=0x6aa0250b0e380 00000400:00000200:0.0:1788880840.330509:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 8472/8472 into md 0xf81 [4] + 224 00000800:00000001:0.0:1788880840.330514:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.330532:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.330538:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.330541:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880840.330544:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880840.330547:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880856 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880840.330554:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=8472 offset=224 replen=8472 req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.330573:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880840.330580:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.330583:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.330613:0:207999:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880840.330619:0:207999:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880840.330620:0:207999:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880840.330622:0:207999:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dc9bac80. 00000400:00000200:1.0:1788880840.330626:0:207999:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e64ce9e0 00000400:00000010:1.0:1788880840.330628:0:207999:0:(lib-lnet.h:280:lnet_md_free()) kfreed 'md': 184 at 00000000e64ce9e0. 00000100:00000001:1.0:1788880840.330630:0:207999:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880840.330633:0:207999:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880840.330641:0:207999:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.330647:0:207999:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880840.330649:0:207999:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.330651:0:207999:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880840.330654:0:207999:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880840.330656:0:207999:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880840.330658:0:207999:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.330661:0:207999:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880840.330665:0:207999:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=68 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880840.330668:0:207999:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=68 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880840.330670:0:207999:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880840.330672:0:207999:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.330674:0:207999:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880840.330676:0:207999:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.330684:0:207999:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880840.330688:0:207999:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.330695:0:207999:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.330697:0:207999:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880840.330699:0:207999:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880840.330702:0:207999:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880856 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880840.330709:0:207999:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job llog_process_th:fa4856d5-51a8-499e-b3f8-b477743a3dec:207999:1875776780690304:192.168.204.111@tcp:502: 00000100:00000001:1.0:1788880840.330714:0:207999:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.330716:0:207999:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.330718:0:207999:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880840.330720:0:207999:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.330722:0:207999:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880856 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.330728:0:207999:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880840.330730:0:207999:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000cd8de53b. 00000100:00000001:1.0:1788880840.330733:0:207999:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880840.330734:0:207999:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00000040:00000001:1.0:1788880840.330740:0:207999:0:(llog_client.c:198:llog_client_next_block()) Process leaving 00000100:00000001:1.0:1788880840.330741:0:207999:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.330743:0:207999:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780690304/t0(0) o502->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 272/8472 e 0 to 0 dl 1788880856 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.330750:0:207999:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880840.330752:0:207999:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880840.330754:0:207999:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 16384 at 00000000664a6a04. 02000000:00000001:1.0:1788880840.330762:0:207999:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880840.330764:0:207999:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.330766:0:207999:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.330768:0:207999:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880840.330770:0:207999:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000000fc6edce. 02000000:00000001:1.0:1788880840.330773:0:207999:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880840.330775:0:207999:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880840.330780:0:207999:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:1.0:1788880840.330783:0:207999:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880840.330784:0:207999:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880840.330786:0:207999:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.330787:0:207999:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.330789:0:207999:0:(genops.c:1251:class_import_put()) Process leaving 00000040:00000001:1.0:1788880840.330791:0:207999:0:(lustre_log.h:485:llog_next_block()) Process leaving (rc=0 : 0 : 0) 00000040:00001000:1.0:1788880840.330794:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026000 type=0x10620000 idx=1 00000040:00001000:1.0:1788880840.330798:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 1 00000040:00001000:1.0:1788880840.330800:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0260e0 type=0x10620000 idx=2 00000040:00001000:1.0:1788880840.330801:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 2 00000040:00001000:1.0:1788880840.330802:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026158 type=0x10620000 idx=3 00000040:00001000:1.0:1788880840.330804:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 3 00000040:00001000:1.0:1788880840.330805:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026200 type=0x10620000 idx=4 00000040:00001000:1.0:1788880840.330806:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 4 00000040:00001000:1.0:1788880840.330807:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0262e0 type=0x10620000 idx=5 00000040:00001000:1.0:1788880840.330809:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 5 00000040:00001000:1.0:1788880840.330810:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0263c0 type=0x10620000 idx=6 00000040:00001000:1.0:1788880840.330812:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 6 00000040:00001000:1.0:1788880840.330813:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026438 type=0x10620000 idx=7 00000040:00001000:1.0:1788880840.330815:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 7 00000040:00001000:1.0:1788880840.330816:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0264e0 type=0x10620000 idx=8 00000040:00001000:1.0:1788880840.330817:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 8 00000040:00001000:1.0:1788880840.330818:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0265c0 type=0x10620000 idx=9 00000040:00001000:1.0:1788880840.330819:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 9 00000040:00001000:1.0:1788880840.330821:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0266a0 type=0x10620000 idx=10 00000040:00001000:1.0:1788880840.330822:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 10 00000040:00001000:1.0:1788880840.330823:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026718 type=0x10620000 idx=11 00000040:00001000:1.0:1788880840.330825:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 11 00000040:00001000:1.0:1788880840.330826:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0267f8 type=0x10620000 idx=12 00000040:00001000:1.0:1788880840.330828:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 12 00000040:00001000:1.0:1788880840.330829:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0268d8 type=0x10620000 idx=13 00000040:00001000:1.0:1788880840.330830:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 13 00000040:00001000:1.0:1788880840.330831:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026930 type=0x10620000 idx=14 00000040:00001000:1.0:1788880840.330833:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 14 00000040:00001000:1.0:1788880840.330834:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0269b0 type=0x10620000 idx=15 00000040:00001000:1.0:1788880840.330840:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 15 00000040:00001000:1.0:1788880840.330841:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026a40 type=0x10620000 idx=16 00000040:00001000:1.0:1788880840.330842:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 16 00000040:00001000:1.0:1788880840.330843:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026ae0 type=0x10620000 idx=17 00000040:00001000:1.0:1788880840.330845:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 17 00000040:00001000:1.0:1788880840.330846:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026bc0 type=0x10620000 idx=18 00000040:00001000:1.0:1788880840.330848:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 18 00000040:00001000:1.0:1788880840.330849:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026ca0 type=0x10620000 idx=19 00000040:00001000:1.0:1788880840.330850:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 19 00000040:00001000:1.0:1788880840.330852:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026cf0 type=0x10620000 idx=20 00000040:00001000:1.0:1788880840.330853:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 20 00000040:00001000:1.0:1788880840.330854:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026dd0 type=0x10620000 idx=21 00000040:00001000:1.0:1788880840.330855:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 21 00000040:00001000:1.0:1788880840.330856:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026eb0 type=0x10620000 idx=22 00000040:00001000:1.0:1788880840.330857:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 22 00000040:00001000:1.0:1788880840.330857:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026f08 type=0x10620000 idx=23 00000040:00001000:1.0:1788880840.330859:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 23 00000040:00001000:1.0:1788880840.330860:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be026f88 type=0x10620000 idx=24 00000040:00001000:1.0:1788880840.330861:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 24 00000040:00001000:1.0:1788880840.330862:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027018 type=0x10620000 idx=25 00000040:00001000:1.0:1788880840.330864:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 25 00000040:00001000:1.0:1788880840.330865:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0270b8 type=0x10620000 idx=26 00000040:00001000:1.0:1788880840.330866:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 26 00000040:00001000:1.0:1788880840.330867:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027198 type=0x10620000 idx=27 00000040:00001000:1.0:1788880840.330868:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 27 00000040:00001000:1.0:1788880840.330869:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027278 type=0x10620000 idx=28 00000040:00001000:1.0:1788880840.330871:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 28 00000040:00001000:1.0:1788880840.330872:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0272c8 type=0x10620000 idx=29 00000040:00001000:1.0:1788880840.330874:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 29 00000040:00001000:1.0:1788880840.330874:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0273a8 type=0x10620000 idx=30 00000040:00001000:1.0:1788880840.330876:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 30 00000040:00001000:1.0:1788880840.330876:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027488 type=0x10620000 idx=31 00000040:00001000:1.0:1788880840.330878:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 31 00000040:00001000:1.0:1788880840.330879:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0274e0 type=0x10620000 idx=32 00000040:00001000:1.0:1788880840.330880:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 32 00000040:00001000:1.0:1788880840.330886:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027560 type=0x10620000 idx=33 00000040:00001000:1.0:1788880840.330888:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 33 00000040:00001000:1.0:1788880840.330889:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be0275f0 type=0x10620000 idx=34 00000040:00001000:1.0:1788880840.330890:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 34 00000040:00001000:1.0:1788880840.330891:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027670 type=0x10620000 idx=35 00000040:00001000:1.0:1788880840.330892:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 35 00000040:00001000:1.0:1788880840.330893:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027750 type=0x10620000 idx=36 00000040:00001000:1.0:1788880840.330895:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 36 00000040:00001000:1.0:1788880840.330895:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027830 type=0x10620000 idx=37 00000040:00001000:1.0:1788880840.330897:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 37 00000040:00001000:1.0:1788880840.330897:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027880 type=0x10620000 idx=38 00000040:00001000:1.0:1788880840.330899:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 38 00000040:00001000:1.0:1788880840.330899:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027960 type=0x10620000 idx=39 00000040:00001000:1.0:1788880840.330900:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 39 00000040:00001000:1.0:1788880840.330901:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027a40 type=0x10620000 idx=40 00000040:00001000:1.0:1788880840.330903:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 40 00000040:00001000:1.0:1788880840.330904:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027a98 type=0x10620000 idx=41 00000040:00001000:1.0:1788880840.330905:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 41 00000040:00001000:1.0:1788880840.330906:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027b18 type=0x10620000 idx=42 00000040:00001000:1.0:1788880840.330908:0:207999:0:(llog.c:767:llog_process_thread()) skipping lrh_index 42 00000040:00001000:1.0:1788880840.330909:0:207999:0:(llog.c:671:llog_process_thread()) processing rec 0xffff8ca7be027ba8 type=0x10620000 idx=43 00000040:00001000:1.0:1788880840.330911:0:207999:0:(llog.c:784:llog_process_thread()) lrh_index: 43 lrh_len: 160 (1112 remains) 00000040:00001000:1.0:1788880840.330914:0:207999:0:(llog.c:805:llog_process_thread()) index: 43, lh_last_idx: 66 synced_idx: 0 lgh_last_idx: 66 00000020:00000001:1.0:1788880840.330917:0:207999:0:(obd_config.c:1862:class_config_llog_handler()) Process entered 00000020:00000010:1.0:1788880840.330923:0:207999:0:(obd_config.c:1997:class_config_llog_handler()) kmalloced '(inst_name)': 34 at 00000000ccfcbe42. 00000020:01000000:1.0:1788880840.330927:0:207999:0:(obd_config.c:2005:class_config_llog_handler()) cmd cf014, instance name: lustre-clilmv-ffff8ca787527000 00000020:00000010:1.0:1788880840.330929:0:207999:0:(obd_config.c:2057:class_config_llog_handler()) kmalloced '(lcfg_new)': 152 at 00000000cd8de53b. 00000020:00000080:1.0:1788880840.330932:0:207999:0:(obd_config.c:1470:class_process_config()) processing cmd: cf014 00000020:00000001:1.0:1788880840.330940:0:207999:0:(obd_class.h:689:obd_process_config()) Process entered 00000020:00000010:1.0:1788880840.330943:0:207999:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 000000000fc6edce. 00000020:00000010:1.0:1788880840.330947:0:207999:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 00000000a58a4768. 00000040:00000010:1.0:1788880840.330950:0:207999:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000edcbc44b. 00000020:00000010:1.0:1788880840.330954:0:207999:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000d5b2a266. 00000020:00000010:1.0:1788880840.330958:0:207999:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 0000000015664212. 00040000:00000010:1.0:1788880840.330966:0:207999:0:(lquota_lib.c:35:lquota_key_init()) kmalloced '(value)': 248 at 000000009908d3b1. 00040000:00000010:1.0:1788880840.330970:0:207999:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 0000000065770d1d. 00800000:00000001:1.0:1788880840.330975:0:207999:0:(lmv_obd.c:1377:lmv_process_config()) Process entered 00800000:00000001:1.0:1788880840.330981:0:207999:0:(lmv_obd.c:437:lmv_add_target()) Process entered 00800000:01000000:1.0:1788880840.330982:0:207999:0:(lmv_obd.c:439:lmv_add_target()) Target uuid: lustre-MDT0001_UUID. index 1 00800000:00000010:1.0:1788880840.330990:0:207999:0:(lmv_obd.c:448:lmv_add_target()) kmalloced '(tgt)': 368 at 000000003a17641d. 00020000:00000001:1.0:1788880840.330993:0:207999:0:(lu_tgt_descs.c:352:ltd_add_tgt()) Process entered 00020000:00000001:1.0:1788880840.330995:0:207999:0:(lu_tgt_descs.c:367:ltd_add_tgt()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00800000:00000001:1.0:1788880840.330998:0:207999:0:(lmv_obd.c:462:lmv_add_target()) Process leaving via out_tgt (rc=18446744073709551599 : -17 : 0xffffffffffffffef) 00800000:00000010:1.0:1788880840.331000:0:207999:0:(lmv_obd.c:479:lmv_add_target()) kfreed 'tgt': 368 at 000000003a17641d. 00800000:00000001:1.0:1788880840.331003:0:207999:0:(lmv_obd.c:1394:lmv_process_config()) Process leaving via out (rc=18446744073709551599 : -17 : 0xffffffffffffffef) 00800000:00000001:1.0:1788880840.331005:0:207999:0:(lmv_obd.c:1400:lmv_process_config()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00000020:00000010:1.0:1788880840.331008:0:207999:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 00000000a58a4768. 00000040:00000010:1.0:1788880840.331012:0:207999:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000edcbc44b. 00000020:00000010:1.0:1788880840.331014:0:207999:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 00000000d5b2a266. 00000020:00000010:1.0:1788880840.331018:0:207999:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 0000000015664212. 00040000:00000010:1.0:1788880840.331020:0:207999:0:(lquota_lib.c:35:lquota_key_fini()) kfreed 'info': 248 at 000000009908d3b1. 00040000:00000010:1.0:1788880840.331033:0:207999:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 0000000065770d1d. 00000020:00000010:1.0:1788880840.331035:0:207999:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 000000000fc6edce. 00000020:00000001:1.0:1788880840.331039:0:207999:0:(obd_class.h:704:obd_process_config()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00000020:00000001:1.0:1788880840.331041:0:207999:0:(obd_config.c:1667:class_process_config()) Process leaving via out (rc=18446744073709551599 : -17 : 0xffffffffffffffef) 00000020:00000010:1.0:1788880840.331044:0:207999:0:(obd_config.c:2090:class_config_llog_handler()) kfreed 'lcfg_new': 152 at 00000000cd8de53b. 00000020:00000010:1.0:1788880840.331046:0:207999:0:(obd_config.c:2092:class_config_llog_handler()) kfreed 'inst_name': 34 at 00000000ccfcbe42. 00000020:00020000:1.0:1788880840.331049:0:207999:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.111@tcp: cfg command failed: rc = -17 00000020:00000001:1.0:1788880840.333530:0:207999:0:(obd_config.c:2423:class_config_dump_handler()) Process entered 00000020:00000010:1.0:1788880840.333540:0:207999:0:(obd_config.c:2425:class_config_dump_handler()) kmalloced '(outstr)': 256 at 000000009908d3b1. 00000020:00000001:1.0:1788880840.333546:0:207999:0:(obd_config.c:2354:class_config_parse_rec()) Process entered 00000020:00000001:1.0:1788880840.333552:0:207999:0:(obd_config.c:2413:class_config_parse_rec()) Process leaving (rc=88 : 88 : 58) 00000020:02000400:1.0:1788880840.333554:0:207999:0:(obd_config.c:2431:class_config_dump_handler()) cmd=cf014 0:lustre-clilmv 1:lustre-MDT0001_UUID 2:1 3:1 4:lustre-MDT0001-mdc_UUID 00000020:00000010:1.0:1788880840.337073:0:207999:0:(obd_config.c:2437:class_config_dump_handler()) kfreed 'outstr': 256 at 000000009908d3b1. 00000020:00000001:1.0:1788880840.337089:0:207999:0:(obd_config.c:2438:class_config_dump_handler()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788880840.337092:0:207999:0:(obd_config.c:2106:class_config_llog_handler()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00000040:00000001:1.0:1788880840.337116:0:207999:0:(llog.c:843:llog_process_thread()) Process leaving via out (rc=18446744073709551599 : -17 : 0xffffffffffffffef) 00000040:00080000:1.0:1788880840.337120:0:207999:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 43 count 66 00000040:00000010:1.0:1788880840.337123:0:207999:0:(llog.c:899:llog_process_thread()) kvfree 'buf': 8192 at 000000000d09cd28. 00000020:00000010:1.0:1788880840.337133:0:207999:0:(lu_object.c:2058:lu_env_remove()) kfreed_rcu 'lei': 40 at 000000000cca38e9. 00000020:00000010:1.0:1788880840.337155:0:207999:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 000000005bd099ee. 00000040:00000010:1.0:1788880840.337163:0:207999:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000123d40c9. 00000020:00000010:1.0:1788880840.337168:0:207999:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 00000000a3e174ee. 00000020:00000010:1.0:1788880840.337173:0:207999:0:(tgt_main.c:741:tgt_key_fini()) slab-freed '(info)': 1288 at 000000008f8293fd. 00000020:00000010:1.0:1788880840.337179:0:207999:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 0000000025bc9394. 80000000:00000010:1.0:1788880840.337184:0:207999:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 00000000cf4d1652. 40000000:00000010:1.0:1788880840.337189:0:207999:0:(fid_handler.c:479:seq_key_fini()) kfreed 'info': 48 at 00000000b96c0748. 00100000:00000010:1.0:1788880840.337194:0:207999:0:(lfsck_lib.c:43:lfsck_key_fini()) kfreed 'info': 1800 at 000000005257fdb2. 00040000:00000010:1.0:1788880840.337198:0:207999:0:(lquota_lib.c:35:lquota_key_fini()) kfreed 'info': 248 at 00000000d7d1e0d5. 00040000:00000010:1.0:1788880840.337202:0:207999:0:(qmt_dev.c:418:qmt_key_fini()) kfreed 'info': 1256 at 00000000453588fe. 00040000:00000010:1.0:1788880840.337206:0:207999:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 00000000e7b82f46. 00000004:00000010:1.0:1788880840.337210:0:207999:0:(mdd_device.c:2420:mdd_key_fini()) kfreed 'info': 1872 at 000000007f7d1e46. 00000004:00000010:1.0:1788880840.337216:0:207999:0:(mdt_handler.c:8495:mdt_key_fini()) kfreed 'info': 2568 at 0000000013189f47. 00000004:00000010:1.0:1788880840.337223:0:207999:0:(lod_dev.c:2378:lod_key_fini()) kfreed 'info': 1224 at 000000006f45b8ae. 00002000:00000010:1.0:1788880840.337228:0:207999:0:(ofd_dev.c:3002:ofd_key_fini()) kfreed 'info': 816 at 000000002a02a4fa. 00000004:00000010:1.0:1788880840.337251:0:207999:0:(osp_dev.c:1888:osp_key_fini()) kfreed 'info': 616 at 00000000216a9979. 00000020:00000010:1.0:1788880840.337256:0:207999:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000659bd78d. 00000040:00000010:2.0:1788880840.337289:0:203401:0:(llog.c:994:llog_process_or_fork()) kfreed 'lpi': 96 at 00000000d9f2f285. 00000040:00000001:2.0:1788880840.337297:0:203401:0:(llog.c:995:llog_process_or_fork()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 00000020:01000000:2.0:1788880840.337311:0:203401:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 43-43 (rc=-17) 00000040:00000010:2.0:1788880840.337317:0:203401:0:(llog.c:70:llog_free_handle()) kvfree 'loghandle->lgh_hdr': 8192 at 00000000c3c95438. 00000040:00000010:2.0:1788880840.337322:0:203401:0:(llog.c:72:llog_free_handle()) kfreed 'loghandle': 352 at 000000002b77e71a. 00000020:00000001:2.0:1788880840.337324:0:203401:0:(obd_config.c:2151:class_config_parse_llog()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 10000000:00000001:2.0:1788880840.337327:0:203401:0:(mgc_request.c:1714:mgc_process_cfg_log()) Process leaving via out_pop (rc=18446744073709551599 : -17 : 0xffffffffffffffef) 00000020:00000010:2.0:1788880840.337333:0:203401:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 000000005309c375. 00000040:00000010:2.0:1788880840.337336:0:203401:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 0000000099f3ac40. 00000020:00000010:2.0:1788880840.337339:0:203401:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 00000000a71f1a29. 00000020:00000010:2.0:1788880840.337351:0:203401:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 00000000852b7c70. 80000000:00000010:2.0:1788880840.337354:0:203401:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 00000000b245c5fd. 00040000:00000010:2.0:1788880840.337357:0:203401:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 0000000056df9207. 20000000:00000010:2.0:1788880840.337361:0:203401:0:(mgs_handler.c:1787:mgs_key_fini()) kfreed 'info': 392 at 000000009c91be43. 00000020:00000010:2.0:1788880840.337366:0:203401:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 00000000f5272de5. 10000000:00000010:2.0:1788880840.337369:0:203401:0:(mgc_request.c:1730:mgc_process_cfg_log()) kfreed 'env': 56 at 000000000a91cda9. 10000000:01000000:2.0:1788880840.337374:0:203401:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'lustre-client' failed (-17). 10000000:00000001:2.0:1788880840.337378:0:203401:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:2.0:1788880840.337381:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:2.0:1788880840.337383:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:2.0:1788880840.337387:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 00000000c2b00415 refcount=3 00000020:00000001:2.0:1788880840.337389:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617250280345600 : -126823429206016 : ffff8ca79e8d9000) 00010000:00000001:2.0:1788880840.337392:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617250280345600 : -126823429206016 : ffff8ca79e8d9000) 00010000:00000001:2.0:1788880840.337395:0:203401:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:2.0:1788880840.337399:0:203401:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 00000000c2b00415/0x805d548a36a886f9 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7d5b expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880840.337409:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880840.337411:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:2.0:1788880840.337413:0:203401:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 00000000c2b00415/0x805d548a36a886f9 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7d5b expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:2.0:1788880840.337420:0:203401:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:2.0:1788880840.337421:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788880840.337422:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:2.0:1788880840.337424:0:203401:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880840.337426:0:203401:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=18446744073709551599 : -17 : ffffffffffffffef) 10000000:00020000:2.0:1788880840.337429:0:203401:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 10000000:00000001:2.0:1788880840.340175:0:203401:0:(mgc_request.c:621:do_requeue()) Process leaving 10000000:00000001:2.0:1788880840.340181:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:2.0:1788880840.340184:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-sptlrpc (000000007b8fd88a) refs 2 10000000:00000001:2.0:1788880840.340189:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:2.0:1788880840.340192:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:2.0:1788880840.340194:0:203401:0:(mgc_request.c:123:config_log_put()) log lustre-client(00000000c2974351) refs 3 10000000:00000001:2.0:1788880840.340197:0:203401:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:2.0:1788880840.340211:0:203401:0:(mgc_request.c:601:do_requeue()) Process entered 10000000:01000000:2.0:1788880840.340213:0:203401:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:00000001:2.0:1788880840.340215:0:203401:0:(mgc_request.c:1782:mgc_process_log()) Process entered 10000000:01000000:2.0:1788880840.340218:0:203401:0:(mgc_request.c:1801:mgc_process_log()) Process log lustre-sptlrpc-ffff8ca792d4cd40 from 1 10000000:00000001:2.0:1788880840.340221:0:203401:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:2.0:1788880840.340222:0:203401:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00000010:2.0:1788880840.340258:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880840.340264:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880840.340270:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880840.340274:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880840.340281:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880840.340289:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 000000008e3e56e7. 00000100:00000001:2.0:1788880840.340299:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880840.340319:0:203401:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:2.0:1788880840.340323:0:203401:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000040:2.0:1788880840.340330:0:203401:0:(ldlm_resource.c:1656:ldlm_resource_getref()) getref res: 000000008fce9a29 count: 2 00010000:00000001:2.0:1788880840.340333:0:203401:0:(ldlm_resource.c:1595:ldlm_resource_get()) Process leaving via found (rc=0 : 0 : 0x0) 00010000:00000001:2.0:1788880840.340336:0:203401:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:2.0:1788880840.340341:0:203401:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at 000000004a641a8f. 00000020:00000001:2.0:1788880840.340347:0:203401:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:2.0:1788880840.340350:0:203401:0:(lustre_handles.c:93:class_handle_hash()) added object 000000004a641a8f with handle 0x805d548a36a88700 to hash 00000020:00000001:2.0:1788880840.340353:0:203401:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:2.0:1788880840.340354:0:203401:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446617249862485504 : -126823847066112 : ffff8ca785a58600) 00010000:00000001:2.0:1788880840.340358:0:203401:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446617249862485504 : -126823847066112 : ffff8ca785a58600) 00010000:00000001:2.0:1788880840.340362:0:203401:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:2.0:1788880840.340366:0:203401:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880840.340369:0:203401:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.111@tcp lock: 000000004a641a8f/0x805d548a36a88700 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788880840.340379:0:203401:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.111@tcp lock: 000000004a641a8f/0x805d548a36a88700 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788880840.340391:0:203401:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.111@tcp lock: 000000004a641a8f/0x805d548a36a88700 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:2.0:1788880840.340409:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:2.0:1788880840.340412:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:2.0:1788880840.340419:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 0000000047bb7169. 00000100:00000001:2.0:1788880840.340422:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250063673984 : -126823645877632 : ffff8ca791a36a80) 00000100:00000001:2.0:1788880840.340426:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880840.340427:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000100:00000001:2.0:1788880840.340432:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880840.340435:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880840.340438:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:2.0:1788880840.340440:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880840.340451:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880840.340461:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880840.340463:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880840.340467:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880840.340471:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880840.340473:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880840.340477:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780690432:192.168.204.111@tcp:101: 00000100:00000001:2.0:1788880840.340482:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880840.340489:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880840.340491:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880840.340494:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880840.340499:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at 00000000de3147fa. 02000000:00000001:2.0:1788880840.340501:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880840.340508:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000087a6361a. 00000400:00000010:2.0:1788880840.340516:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cc94f399. 00000100:00000200:2.0:1788880840.340524:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875776780690432, portal 25 00000100:00000001:2.0:1788880840.340526:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880840.340527:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000100:00000040:2.0:1788880840.340533:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880840.340541:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880840.340550:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880840.340552:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000084794ee4. 00000100:00000200:2.0:1788880840.340555:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875776780690432, offset 0 00000400:00000200:2.0:1788880840.340563:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880840.340579:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880840.340584:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880840.340589:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880840.340593:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880840.340596:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880840.340600:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [472:472:256:4294967295] 192.168.204.111@tcp LPNI seq info [472:472:8:4294967295] 00000400:00000200:2.0:1788880840.340612:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880840.340617:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880840.340623:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e5360691. 00000800:00000200:2.0:1788880840.340631:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880840.340639:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880840.340644:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e5360691 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:2.0:1788880840.340659:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880840.340663:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306476517 00000100:00000001:2.0:1788880840.340673:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306476505 00000100:00000001:2.0:1788880840.340680:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306476517 00000100:00000001:2.0:1788880840.340685:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306476505 00000100:00000001:2.0:1788880840.340688:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306476517 00000100:00000001:2.0:1788880840.340693:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306476505 00000100:00000001:2.0:1788880840.340695:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306476517 00000100:00000001:2.0:1788880840.340698:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306476505 00000100:00000001:2.0:1788880840.340701:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880840.340704:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880840.340706:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880840.340708:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:2.0:1788880840.340718:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 0000000047bb7169 going to sleep for 11 seconds 00000100:00000001:2.0:1788880840.340721:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880840.340724:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880840.340804:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880840.340809:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e5360691. 00000400:00000200:0.0:1788880840.340813:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.340818:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880840.340822:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000084794ee4 00000400:00000010:0.0:1788880840.340824:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000084794ee4. 00000100:00000001:0.0:1788880840.340828:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880840.340831:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.340841:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880840.340844:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.340851:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880840.340853:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880840.342284:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.342298:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880840.342313:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.342316:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880840.342325:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880840.342338:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 344 into portal 25 MB=0x6aa0250b0e400 00000400:00000200:2.0:1788880840.342345:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 344/344 into md 0xf89 [1] + 224 00000800:00000001:2.0:1788880840.342351:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.342357:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880840.342359:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880840.342361:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880840.342365:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880840.342370:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880840.342381:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880840.342393:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880840.342398:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880840.342400:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.342461:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880840.342467:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880840.342468:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880840.342470:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000087a6361a. 00000400:00000200:1.0:1788880840.342473:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cc94f399 00000400:00000010:1.0:1788880840.342474:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cc94f399. 00000100:00000001:1.0:1788880840.342476:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880840.342478:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880840.342486:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.342492:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880840.342504:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.342507:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880840.342510:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880840.342521:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880840.342524:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.342529:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880840.342536:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=68 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880840.342540:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=68 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880840.342543:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880840.342545:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.342547:0:203401:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880840.342549:0:203401:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.342553:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880840.342557:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.342564:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.342567:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880840.342569:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880840.342573:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880840.342581:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780690432:192.168.204.111@tcp:101: 00000100:00000001:1.0:1788880840.342587:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.342589:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.342591:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880840.342593:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.342596:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.342602:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880840.342604:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 0000000047bb7169. 00000100:00000001:1.0:1788880840.342608:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880840.342610:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.342612:0:203401:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:1.0:1788880840.342615:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880840.342617:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880840.342626:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000004a641a8f refcount=4 00000020:00000001:1.0:1788880840.342628:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862485504 : -126823847066112 : ffff8ca785a58600) 00010000:00000001:1.0:1788880840.342631:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862485504 : -126823847066112 : ffff8ca785a58600) 00010000:00000040:1.0:1788880840.342637:0:203401:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: 000000004a641a8f, remote cookie: 0x92f11317f60a7d62, flags: 0x0 00010000:00000001:1.0:1788880840.342641:0:203401:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:1.0:1788880840.342646:0:203401:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:1.0:1788880840.342648:0:203401:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:1.0:1788880840.342650:0:203401:0:(ldlm_lock.c:1006:search_granted_lock()) Process leaving 00010000:00000001:1.0:1788880840.342652:0:203401:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:1.0:1788880840.342654:0:203401:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x0:0x0].0x0 (000000008fce9a29) refcount = 2 00010000:00000040:1.0:1788880840.342657:0:203401:0:(ldlm_resource.c:1879:ldlm_resource_dump()) Granted locks (in reverse order): 00010000:00000040:1.0:1788880840.342660:0:203401:0:(ldlm_resource.c:1882:ldlm_resource_dump()) ### ### ns: MGC192.168.204.111@tcp lock: 00000000c2b00415/0x805d548a36a886f9 lrc: 1/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7d5b expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788880840.342667:0:203401:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.111@tcp lock: 000000004a641a8f/0x805d548a36a88700 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7d62 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880840.342673:0:203401:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:1.0:1788880840.342675:0:203401:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:1.0:1788880840.342677:0:203401:0:(ldlm_lock.c:1915:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788880840.342687:0:203401:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:1.0:1788880840.342689:0:203401:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788880840.342692:0:203401:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.111@tcp lock: 000000004a641a8f/0x805d548a36a88700 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x92f11317f60a7d62 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880840.342698:0:203401:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:1.0:1788880840.342700:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.342702:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880840.342703:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.342704:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880840.342705:0:203401:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.342708:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.342711:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780690432/t0(0) o101->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 328/344 e 0 to 0 dl 1788880856 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.342718:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880840.342733:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880840.342736:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at 00000000de3147fa. 02000000:00000001:1.0:1788880840.342740:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880840.342742:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.342751:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.342755:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880840.342758:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 000000008e3e56e7. 02000000:00000001:1.0:1788880840.342763:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880840.342765:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880840.342767:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:1.0:1788880840.342771:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880840.342772:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:1.0:1788880840.342774:0:203401:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.342777:0:203401:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788880840.342779:0:203401:0:(mgc_request.c:108:config_log_get()) log lustre-sptlrpc (000000007b8fd88a) refs 3 10000000:00000001:1.0:1788880840.342781:0:203401:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.342782:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880840.342784:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880840.342785:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000004a641a8f refcount=3 00000020:00000001:1.0:1788880840.342786:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862485504 : -126823847066112 : ffff8ca785a58600) 00010000:00000001:1.0:1788880840.342788:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862485504 : -126823847066112 : ffff8ca785a58600) 00010000:00000001:1.0:1788880840.342789:0:203401:0:(ldlm_lock.c:2548:ldlm_lock_set_data()) Process entered 00010000:00000001:1.0:1788880840.342791:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.342791:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788880840.342792:0:203401:0:(ldlm_lock.c:2557:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.342795:0:203401:0:(mgc_request.c:1668:mgc_process_cfg_log()) Process entered 10000000:00000010:1.0:1788880840.342798:0:203401:0:(mgc_request.c:1682:mgc_process_cfg_log()) kmalloced '(env)': 56 at 00000000b96c0748. 00000020:00000010:1.0:1788880840.342801:0:203401:0:(lu_object.c:1772:keys_init()) kmalloced '(ctx->lc_value)': 320 at 0000000065770d1d. 00000020:00000010:1.0:1788880840.342806:0:203401:0:(lu_object.c:568:lu_global_key_init()) kmalloced '(value)': 512 at 0000000015664212. 00000040:00000010:1.0:1788880840.342809:0:203401:0:(llog_obd.c:218:llog_key_init()) kmalloced '(value)': 480 at 00000000edcbc44b. 00000020:00000010:1.0:1788880840.342813:0:203401:0:(dt_object.c:33:dt_global_key_init()) kmalloced '(value)': 1264 at 00000000d5b2a266. 00000020:00000010:1.0:1788880840.342818:0:203401:0:(update_records.c:1179:update_key_init()) kmalloced '(value)': 432 at 00000000a58a4768. 80000000:00000010:1.0:1788880840.342820:0:203401:0:(fld_handler.c:35:fld_key_init()) kmalloced '(value)': 72 at 00000000cf4d1652. 00040000:00000010:1.0:1788880840.342825:0:203401:0:(qsd_lib.c:51:qsd_key_init()) kmalloced '(value)': 408 at 000000003a17641d. 20000000:00000010:1.0:1788880840.342828:0:203401:0:(mgs_handler.c:1787:mgs_key_init()) kmalloced '(value)': 392 at 00000000760cd5b8. 10000000:00000040:1.0:1788880840.342832:0:203401:0:(lustre_log.h:387:llog_ctxt_get()) GETting ctxt 000000006d5d0799 : new refcount 2 00000020:00000001:1.0:1788880840.342839:0:203401:0:(obd_config.c:2120:class_config_parse_llog()) Process entered 00000020:00000040:1.0:1788880840.342841:0:203401:0:(obd_config.c:2122:class_config_parse_llog()) looking up llog lustre-sptlrpc 00000040:00000001:1.0:1788880840.342842:0:203401:0:(llog.c:1538:llog_open()) Process entered 00000040:00000010:1.0:1788880840.342845:0:203401:0:(llog.c:42:llog_alloc_handle()) kmalloced '(loghandle)': 352 at 00000000ace1117b. 00000040:00000001:1.0:1788880840.342848:0:203401:0:(llog_client.c:70:llog_client_open()) Process entered 00000020:00000040:1.0:1788880840.342850:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788880840.342853:0:203401:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000020:00000040:1.0:1788880840.342855:0:203401:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=5 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880840.342859:0:203401:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880840.342861:0:203401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880840.342865:0:203401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880840.342869:0:203401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at 0000000025bc9394. 00000100:00000001:1.0:1788880840.342872:0:203401:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.342877:0:203401:0:(client.c:3352:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788880840.342879:0:203401:0:(client.c:1156:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788880840.342884:0:203401:0:(client.c:1158:ptlrpc_prep_set()) kmalloced '(set)': 144 at 00000000cd8de53b. 00000100:00000001:1.0:1788880840.342886:0:203401:0:(client.c:1173:ptlrpc_prep_set()) Process leaving (rc=18446617250052434560 : -126823657117056 : ffff8ca790f7ea80) 00000100:00000001:1.0:1788880840.342889:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.342890:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000001:1.0:1788880840.342894:0:203401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880840.342895:0:203401:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.342897:0:203401:0:(client.c:2675:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788880840.342898:0:203401:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880840.342902:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.342909:0:203401:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880840.342910:0:203401:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.342914:0:203401:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880840.342917:0:203401:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880840.342919:0:203401:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880840.342922:0:203401:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780690560:192.168.204.111@tcp:501: 00000100:00000001:1.0:1788880840.342926:0:203401:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880840.342933:0:203401:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880840.342941:0:203401:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.342949:0:203401:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880840.342953:0:203401:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000123d40c9. 02000000:00000001:1.0:1788880840.342955:0:203401:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880840.342959:0:203401:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dc9bac80. 00000400:00000010:1.0:1788880840.342963:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000066e8f89d. 00000100:00000200:1.0:1788880840.342967:0:203401:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780690560, portal 25 00000100:00000001:1.0:1788880840.342970:0:203401:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880840.342971:0:203401:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000040:1.0:1788880840.342975:0:203401:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.342982:0:203401:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880840.342985:0:203401:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880840.342987:0:203401:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 00000100:00000200:1.0:1788880840.342990:0:203401:0:(niobuf.c:94:ptl_send_buf()) Sending 512 bytes to portal 26, xid 1875776780690560, offset 0 00000400:00000200:1.0:1788880840.342996:0:203401:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880840.343004:0:203401:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880840.343010:0:203401:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880840.343013:0:203401:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880840.343016:0:203401:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880840.343018:0:203401:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880840.343020:0:203401:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [473:473:256:4294967295] 192.168.204.111@tcp LPNI seq info [473:473:8:4294967295] 00000400:00000200:1.0:1788880840.343027:0:203401:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880840.343031:0:203401:0:(socklnd_cb.c:931:ksocknal_send()) sending 512 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880840.343035:0:203401:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000800:00000200:1.0:1788880840.343040:0:203401:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880840.343045:0:203401:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880840.343047:0:203401:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 608 niov 1 nkiov 1 00000100:00000001:1.0:1788880840.343063:0:203401:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.343066:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306476519 00000100:00000001:1.0:1788880840.343072:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306476517 00000100:00000001:1.0:1788880840.343075:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306476519 00000100:00000001:1.0:1788880840.343082:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306476517 00000100:00000001:1.0:1788880840.343084:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306476519 00000100:00000001:1.0:1788880840.343086:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306476517 00000100:00000001:1.0:1788880840.343087:0:203401:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306476519 00000100:00000001:1.0:1788880840.343089:0:203401:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306476517 00000100:00000001:1.0:1788880840.343090:0:203401:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880840.343091:0:203401:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880840.343093:0:203401:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880840.343094:0:203401:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788880840.343095:0:203401:0:(client.c:2702:ptlrpc_set_wait()) set 00000000cd8de53b going to sleep for 11 seconds 00000100:00000001:1.0:1788880840.343097:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880840.343098:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880840.343206:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880840.343209:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000400:00000200:0.0:1788880840.343214:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.343218:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880840.343221:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:0.0:1788880840.343223:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:0.0:1788880840.343226:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880840.343229:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.343317:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880840.343321:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 2 req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.343328:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880840.343330:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880840.344332:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.344340:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880840.344343:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.344344:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.344350:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880840.344358:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 272 into portal 25 MB=0x6aa0250b0e480 00000400:00000200:0.0:1788880840.344365:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 272/272 into md 0xf91 [1] + 224 00000800:00000001:0.0:1788880840.344369:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.344373:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880840.344375:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880840.344377:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880840.344381:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880840.344384:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880840.344391:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=272 offset=224 replen=272 req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880840.344402:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880840.344409:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880840.344411:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.344425:0:203401:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880840.344431:0:203401:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880840.344432:0:203401:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880840.344434:0:203401:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dc9bac80. 00000400:00000200:1.0:1788880840.344438:0:203401:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000066e8f89d 00000400:00000010:1.0:1788880840.344439:0:203401:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000066e8f89d. 00000100:00000001:1.0:1788880840.344442:0:203401:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880840.344446:0:203401:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880840.344455:0:203401:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.344461:0:203401:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880840.344463:0:203401:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.344466:0:203401:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880840.344468:0:203401:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880840.344470:0:203401:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880840.344472:0:203401:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880840.344474:0:203401:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880840.344478:0:203401:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=68 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880840.344481:0:203401:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=68 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880840.344484:0:203401:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:1.0:1788880840.344494:0:203401:0:(client.c:1497:ptlrpc_check_status()) @@@ check status: rc = -2 req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/0 rc 0/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.344500:0:203401:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000100:00000001:1.0:1788880840.344502:0:203401:0:(client.c:1784:after_reply()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000100:00000040:1.0:1788880840.344506:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Rpc:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.344512:0:203401:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551614 : -2 : 0xfffffffffffffffe) 00000100:00000001:1.0:1788880840.344515:0:203401:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880840.344517:0:203401:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880840.344520:0:203401:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Interpret:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880840.344527:0:203401:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:fa4856d5-51a8-499e-b3f8-b477743a3dec:203401:1875776780690560:192.168.204.111@tcp:501: 00000100:00000001:1.0:1788880840.344531:0:203401:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880840.344533:0:203401:0:(client.c:2766:ptlrpc_set_wait()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000100:00000001:1.0:1788880840.344536:0:203401:0:(client.c:1224:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788880840.344537:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.344540:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 2 fl Complete:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.344546:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880840.344548:0:203401:0:(client.c:3800:ptlrpc_reqset_free()) kfreed 'set': 144 at 00000000cd8de53b. 00000100:00000001:1.0:1788880840.344550:0:203401:0:(client.c:1261:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788880840.344552:0:203401:0:(client.c:3371:ptlrpc_queue_wait()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000040:00000001:1.0:1788880840.344554:0:203401:0:(llog_client.c:114:llog_client_open()) Process leaving via out (rc=18446744073709551614 : -2 : 0xfffffffffffffffe) 00000020:00000001:1.0:1788880840.344558:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.344570:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.344573:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880840.344574:0:203401:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880840.344579:0:203401:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780690560/t0(0) o501->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 512/272 e 0 to 0 dl 1788880856 ref 1 fl Complete:RQU/200/0 rc -2/-2 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880840.344585:0:203401:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880840.344591:0:203401:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880840.344593:0:203401:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000123d40c9. 02000000:00000001:1.0:1788880840.344596:0:203401:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880840.344597:0:203401:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880840.344598:0:203401:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880840.344600:0:203401:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880840.344602:0:203401:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at 0000000025bc9394. 02000000:00000001:1.0:1788880840.344605:0:203401:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880840.344607:0:203401:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880840.344608:0:203401:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:1.0:1788880840.344612:0:203401:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880840.344613:0:203401:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000040:00001000:1.0:1788880840.344616:0:203401:0:(llog.c:1559:llog_open()) MGC192.168.204.111@tcp: Failed to open llog lustre-sptlrpc: rc -2 00000040:00000010:1.0:1788880840.344620:0:203401:0:(llog.c:72:llog_free_handle()) kfreed 'loghandle': 352 at 00000000ace1117b. 00000040:00000001:1.0:1788880840.344623:0:203401:0:(llog.c:1563:llog_open()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 00000020:00000001:1.0:1788880840.344626:0:203401:0:(obd_config.c:2125:class_config_parse_llog()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 10000000:00000001:1.0:1788880840.344629:0:203401:0:(mgc_request.c:1714:mgc_process_cfg_log()) Process leaving via out_pop (rc=18446744073709551614 : -2 : 0xfffffffffffffffe) 00000020:00000010:1.0:1788880840.344635:0:203401:0:(lu_object.c:568:lu_global_key_fini()) kfreed 'info': 512 at 0000000015664212. 00000040:00000010:1.0:1788880840.344639:0:203401:0:(llog_obd.c:218:llog_key_fini()) kfreed 'info': 480 at 00000000edcbc44b. 00000020:00000010:1.0:1788880840.344643:0:203401:0:(dt_object.c:34:dt_global_key_fini()) kfreed 'info': 1264 at 00000000d5b2a266. 00000020:00000010:1.0:1788880840.344648:0:203401:0:(update_records.c:1175:update_key_fini()) kfreed 'info': 432 at 00000000a58a4768. 80000000:00000010:1.0:1788880840.344652:0:203401:0:(fld_handler.c:35:fld_key_fini()) kfreed 'info': 72 at 00000000cf4d1652. 00040000:00000010:1.0:1788880840.344656:0:203401:0:(qsd_lib.c:51:qsd_key_fini()) kfreed 'info': 408 at 000000003a17641d. 20000000:00000010:1.0:1788880840.344660:0:203401:0:(mgs_handler.c:1787:mgs_key_fini()) kfreed 'info': 392 at 00000000760cd5b8. 00000020:00000010:1.0:1788880840.344663:0:203401:0:(lu_object.c:1700:keys_fini()) kfreed 'ctx->lc_value': 320 at 0000000065770d1d. 10000000:00000010:1.0:1788880840.344665:0:203401:0:(mgc_request.c:1730:mgc_process_cfg_log()) kfreed 'env': 56 at 00000000b96c0748. 10000000:01000000:1.0:1788880840.344667:0:203401:0:(mgc_request.c:1878:mgc_process_log()) MGC192.168.204.111@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 10000000:00000001:1.0:1788880840.344671:0:203401:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:1.0:1788880840.344674:0:203401:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788880840.344675:0:203401:0:(lustre_handles.c:129:class_handle2object()) Process entered 00000020:00000040:1.0:1788880840.344678:0:203401:0:(lustre_handles.c:147:class_handle2object()) GET ldlm 000000004a641a8f refcount=3 00000020:00000001:1.0:1788880840.344680:0:203401:0:(lustre_handles.c:154:class_handle2object()) Process leaving (rc=18446617249862485504 : -126823847066112 : ffff8ca785a58600) 00010000:00000001:1.0:1788880840.344683:0:203401:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446617249862485504 : -126823847066112 : ffff8ca785a58600) 00010000:00000001:1.0:1788880840.344685:0:203401:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:1.0:1788880840.344694:0:203401:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.111@tcp lock: 000000004a641a8f/0x805d548a36a88700 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7d62 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880840.344702:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.344703:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:1.0:1788880840.344705:0:203401:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.111@tcp lock: 000000004a641a8f/0x805d548a36a88700 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x92f11317f60a7d62 expref: -99 pid: 203401 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788880840.344711:0:203401:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:1.0:1788880840.344713:0:203401:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788880840.344714:0:203401:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:1.0:1788880840.344715:0:203401:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788880840.344717:0:203401:0:(mgc_request.c:1906:mgc_process_log()) Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe) 10000000:00000001:1.0:1788880840.344720:0:203401:0:(mgc_request.c:621:do_requeue()) Process leaving 10000000:00000001:1.0:1788880840.344723:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:1.0:1788880840.344724:0:203401:0:(mgc_request.c:123:config_log_put()) log lustre-sptlrpc(000000007b8fd88a) refs 3 10000000:00000001:1.0:1788880840.344727:0:203401:0:(mgc_request.c:147:config_log_put()) Process leaving 00010000:00000001:1.0:1788880840.382370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880840.382378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880840.382384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880840.382385:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880840.382431:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880840.382435:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880840.382437:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880841.406355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880841.406362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880841.406366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880841.406367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880841.406449:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880841.406456:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880841.406460:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880842.430336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880842.430342:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880842.430347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880842.430348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880842.430402:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880842.430405:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880842.430406:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880843.390395:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880843.390403:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880843.390409:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000020:00000040:2.0:1788880843.390415:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880843.390420:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880843.390422:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880843.390427:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880843.390435:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000001:2.0:1788880843.390453:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880843.390463:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880843.390475:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880843.390477:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880843.390483:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_00+0] req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880843.390525:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880843.390528:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880843.390532:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880843.390535:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000078d5a15d. 00000020:00000040:2.0:1788880843.390538:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880843.390541:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880843.390543:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880843.390545:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880843.390546:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880843.390549:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f47bc5fc. 00000100:00100000:3.0:1788880843.390550:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:2.0:1788880843.390553:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880843.390553:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880843.390559:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880843.390560:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880843.390568:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880843.390569:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880843.390571:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880843.390572:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880843.390575:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880843.390578:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000078d5a15d] to pc [ptlrpcd_00_01+1] req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880843.390578:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880843.390585:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880843.390597:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880843.390600:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880843.390601:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880843.390605:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880843.390606:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880843.390610:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 02000000:00000001:3.0:1788880843.390611:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000020:00000040:2.0:1788880843.390613:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:3.0:1788880843.390615:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880843.390617:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880843.390621:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880843.390621:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880843.390626:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780690688:192.168.204.111@tcp:400: 02000000:00000001:2.0:1788880843.390630:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880843.390632:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000010:2.0:1788880843.390635:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f9812869. 00000100:00000001:2.0:1788880843.390640:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880843.390640:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880843.390647:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:3.0:1788880843.390651:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880843.390654:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880843.390656:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880843.390658:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_02+2] req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880843.390666:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880843.390672:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000081a10824. 02000000:00000001:3.0:1788880843.390675:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880843.390681:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000623a1d39. 00000100:00000001:2.0:1788880843.390686:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880843.390688:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000100:00000040:2.0:1788880843.390690:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:3.0:1788880843.390696:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780690688, portal 25 00000100:00000001:3.0:1788880843.390700:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880843.390702:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 00000100:00000040:2.0:1788880843.390704:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880843.390706:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880843.390712:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880843.390712:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11812) 00000100:00000001:1.0:1788880843.390715:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880843.390718:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880859 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880843.390722:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880843.390725:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880843.390730:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880843.390741:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880843.390745:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880843.390748:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880843.390748:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880843.390751:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000999bcd04. 00000100:00000200:3.0:1788880843.390754:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780690688, offset 0 02000000:00000001:1.0:1788880843.390759:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880843.390762:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880843.390766:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880843.390766:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788880843.390771:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780690944:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880843.390775:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880843.390778:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880843.390780:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880843.390783:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788880843.390783:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880843.390791:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:1.0:1788880843.390794:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000099f3ac40. 02000000:00000001:1.0:1788880843.390798:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880843.390798:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:1.0:1788880843.390804:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dc9bac80. 00000400:00000200:3.0:1788880843.390805:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:1.0:1788880843.390811:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000066e8f89d. 00000400:00000200:3.0:1788880843.390812:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:1.0:1788880843.390817:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780690944, portal 10 00000100:00000001:0.0:1788880843.390818:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880843.390820:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [474:474:256:4294967295] 192.168.204.111@tcp LPNI seq info [474:474:8:4294967295] 00000100:00100000:0.0:1788880843.390823:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:1.0:1788880843.390823:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880843.390826:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000100:00000001:0.0:1788880843.390827:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880843.390835:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788880843.390838:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880843.390839:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880843.390841:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880843.390846:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880843.390849:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880843.390852:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:3.0:1788880843.390856:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880843.390857:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880843.390863:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880843.390864:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000100:00000040:0.0:1788880843.390866:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880843.390868:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000062e8e6f4. 00000100:00000200:1.0:1788880843.390874:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780690944, offset 0 00000800:00000200:3.0:1788880843.390874:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880843.390884:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880843.390887:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880843.390888:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880843.390888:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:0.0:1788880843.390892:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:3.0:1788880843.390894:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 02000000:00000001:0.0:1788880843.390895:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880843.390899:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880843.390902:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880843.390904:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880843.390909:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880843.390909:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880843.390912:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00100000:0.0:1788880843.390914:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780690816:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880843.390915:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306479567 00000400:00000200:1.0:1788880843.390918:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880843.390924:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880843.390925:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306476519 00000400:00000200:1.0:1788880843.390926:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [475:475:255:4294967295] 192.168.204.111@tcp LPNI seq info [475:475:7:4294967295] 02000000:00000001:0.0:1788880843.390929:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880843.390930:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306479567 02000000:00000001:0.0:1788880843.390932:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880843.390938:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880843.390942:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880843.390942:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306476519 02000000:00000010:0.0:1788880843.390944:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a07dd7e3. 00000100:00000001:3.0:1788880843.390947:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306479567 02000000:00000001:0.0:1788880843.390951:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880843.390952:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306476519 00000800:00000200:1.0:1788880843.390956:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880843.390957:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ead0c626. 00000100:00000001:3.0:1788880843.390957:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306479567 00000800:00000010:1.0:1788880843.390961:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000100:00000001:3.0:1788880843.390963:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306476519 00000400:00000010:0.0:1788880843.390966:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 00000100:00000001:3.0:1788880843.390968:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880843.390969:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880843.390971:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880843.390972:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780690816, portal 10 00000100:00000001:3.0:1788880843.390974:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880843.390976:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880843.390977:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880843.390980:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938426496 : -126823771125120 : ffff8ca78a2c4a80) 00000100:00000001:3.0:1788880843.390980:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880843.390982:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880843.390985:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880843.390988:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880843.390991:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880843.390992:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880843.390998:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880843.391005:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880843.391006:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880843.391011:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880843.391011:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13388 usecs (14 jiffies) 00000400:00000010:0.0:1788880843.391015:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000fbbca8a. 00000100:00000001:1.0:1788880843.391016:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306474510 00000100:00000200:0.0:1788880843.391019:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780690816, offset 0 00000100:00000001:1.0:1788880843.391020:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306479567 00000100:00000001:1.0:1788880843.391025:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306474510 00000100:00000001:1.0:1788880843.391028:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306479567 00000400:00000200:0.0:1788880843.391029:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880843.391033:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306474510 00000100:00000001:1.0:1788880843.391036:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306479567 00000400:00000200:0.0:1788880843.391038:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880843.391039:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306474510 00000100:00000001:1.0:1788880843.391043:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306479567 00000400:00000200:0.0:1788880843.391045:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880843.391046:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880843.391049:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880843.391050:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880843.391053:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880843.391055:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880843.391056:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880843.391061:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880843.391062:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880843.391065:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880843.391067:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [476:476:254:4294967295] 192.168.204.111@tcp LPNI seq info [476:476:6:4294967295] 00000100:00000001:1.0:1788880843.391069:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880843.391072:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880843.391073:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880843.391075:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880843.391087:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880843.391092:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880843.391095:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ae4abc27. 00000800:00000200:0.0:1788880843.391099:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880843.391103:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880843.391106:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ae4abc27 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880843.391110:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880843.391113:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 187435 usecs (188 jiffies) 00000100:00000001:0.0:1788880843.391116:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306474684 00000100:00000001:0.0:1788880843.391118:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306479567 00000100:00000001:0.0:1788880843.391120:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306474684 00000100:00000001:0.0:1788880843.391122:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306479567 00000100:00000001:0.0:1788880843.391125:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306474684 00000100:00000001:0.0:1788880843.391126:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306479567 00000100:00000001:0.0:1788880843.391128:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306474684 00000100:00000001:0.0:1788880843.391129:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306479567 00000100:00000001:0.0:1788880843.391131:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880843.391133:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880843.391135:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880843.391138:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880843.391262:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880843.391266:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:2.0:1788880843.391270:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880843.391275:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880843.391281:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000999bcd04 00000400:00000010:2.0:1788880843.391283:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000999bcd04. 00000100:00000001:2.0:1788880843.391288:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880843.391291:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880859 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880843.391300:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880843.391303:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880859 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880843.391309:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880843.391310:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880843.391321:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880843.391322:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000400:00000200:2.0:1788880843.391331:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880843.391333:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880843.391336:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000062e8e6f4 00000400:00000010:2.0:1788880843.391337:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000062e8e6f4. 00000100:00000001:2.0:1788880843.391339:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880843.391342:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880843.391349:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880843.391350:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880843.391356:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880843.391357:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880843.391411:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880843.391414:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ae4abc27. 00000400:00000200:2.0:1788880843.391416:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880843.391419:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880843.391422:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000fbbca8a 00000400:00000010:2.0:1788880843.391423:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000fbbca8a. 00000100:00000001:2.0:1788880843.391426:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880843.391428:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880843.391434:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880843.391436:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880843.391441:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880843.391443:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880843.393209:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880843.393222:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880843.393226:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880843.393228:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880843.393275:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880843.393291:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0e500 00000400:00000200:0.0:1788880843.393298:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xf99 [1] + 224 00000800:00000001:0.0:1788880843.393305:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880843.393319:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880843.393322:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880843.393325:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880843.393331:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880843.393336:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880843.393345:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880843.393361:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880843.393368:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880843.393387:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880843.393391:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880843.393396:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:0.0:1788880843.393397:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880843.393399:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:0.0:1788880843.393401:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000010:3.0:1788880843.393402:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000623a1d39. 00000800:00000001:0.0:1788880843.393405:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880843.393406:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000800:00000001:0.0:1788880843.393409:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000010:3.0:1788880843.393410:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000800:00000001:0.0:1788880843.393413:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880843.393416:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:0.0:1788880843.393420:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000200:3.0:1788880843.393424:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880843.393431:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0e580 00000400:00000200:0.0:1788880843.393440:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xfa9 [1] + 224 00000100:00000200:3.0:1788880843.393443:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880843.393446:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880843.393451:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880843.393453:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880843.393457:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880843.393459:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880843.393462:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880843.393477:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880843.393493:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880843.393497:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880843.393499:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880843.393519:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880843.393523:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880843.393527:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880843.393528:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880843.393530:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ead0c626. 00000400:00000200:1.0:1788880843.393534:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000400:00000010:1.0:1788880843.393536:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000100:00000001:1.0:1788880843.393538:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880843.393542:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880843.393574:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:1.0:1788880843.393577:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880843.393579:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880843.393584:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880843.393587:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880843.393591:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880843.393591:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880843.393592:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880843.393593:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880843.393601:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880843.393602:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880843.393605:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880843.393607:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880843.393609:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880843.393611:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880843.393614:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:1.0:1788880843.393615:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880843.393619:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880843.393624:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=61 v=5 (5 5 5 5) 00000400:00000200:0.0:1788880843.393630:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:1.0:1788880843.393631:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=61 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880843.393637:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880843.393639:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880843.393641:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:0.0:1788880843.393645:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0e600 00010000:00000001:1.0:1788880843.393646:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880843.393648:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880843.393654:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00001000:3.0:1788880843.393654:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=71 v=5 (5 5 5 5) 00000100:00000040:1.0:1788880843.393657:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000200:0.0:1788880843.393658:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xfa1 [1] + 224 00000100:00001000:3.0:1788880843.393661:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=71 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880843.393662:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880843.393665:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880843.393667:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788880843.393672:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880843.393674:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:0.0:1788880843.393675:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880843.393678:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880843.393681:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00010000:00000001:3.0:1788880843.393684:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:0.0:1788880843.393690:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880843.393690:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880843.393697:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00010000:00000001:3.0:1788880843.393697:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880843.393698:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880843.393701:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880843.393702:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880843.393705:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880843.393709:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880843.393711:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880843.393739:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880843.393740:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00100000:1.0:1788880843.393741:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780690816:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880843.393747:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880843.393750:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880843.393752:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880843.393757:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880843.393759:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880843.393763:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000078d5a15d x1875776780690816/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880843.393772:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880843.393778:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:3.0:1788880843.393782:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780690688:192.168.204.111@tcp:400: 00000800:00000001:0.0:1788880843.393784:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880843.393791:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880843.393795:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880843.393796:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:1.0:1788880843.393801:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:3.0:1788880843.393802:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780690688/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880843.393805:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a07dd7e3. 02000000:00000001:1.0:1788880843.393811:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880843.393812:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880843.393813:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000020:00000001:1.0:1788880843.393814:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880843.393815:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880843.393816:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000020:00000040:1.0:1788880843.393818:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000010:3.0:1788880843.393820:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000081a10824. 00000400:00000001:2.0:1788880843.393821:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000001:1.0:1788880843.393822:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000001:2.0:1788880843.393824:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000010:1.0:1788880843.393827:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f47bc5fc. 00000400:00000010:2.0:1788880843.393827:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dc9bac80. 02000000:00000001:3.0:1788880843.393827:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880843.393831:0:201286:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000001:1.0:1788880843.393834:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000200:2.0:1788880843.393834:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000066e8f89d 00000020:00000040:3.0:1788880843.393834:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 02000000:00000001:1.0:1788880843.393838:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000010:2.0:1788880843.393839:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000066e8f89d. 00000020:00000001:3.0:1788880843.393839:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000010:1.0:1788880843.393842:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000078d5a15d. 02000000:00000010:3.0:1788880843.393844:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 00000100:00000001:2.0:1788880843.393845:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880843.393848:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880843.393850:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880843.393850:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880843.393853:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880843.393855:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:3.0:1788880843.393856:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000200:2.0:1788880843.393859:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880843.393862:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880843.393863:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880843.393865:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880843.393867:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880843.393870:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880843.393873:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880843.393876:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880843.393878:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880843.393882:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880843.393884:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880843.393886:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880843.393893:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880843.393896:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880843.393896:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880843.393901:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880843.393903:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880843.393905:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880843.393906:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880843.393908:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880843.393912:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=71 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880843.393915:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=5 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880843.393917:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880843.393919:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880843.393920:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880843.393923:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880843.393930:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880843.393932:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880843.393933:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880843.393935:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880843.393936:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880843.393938:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880843.393944:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880843.393946:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880843.393947:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880843.393949:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880843.393955:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780690944:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880843.393968:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880843.393970:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880843.393972:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780690944/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880859 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880843.393978:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880843.393980:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880843.393982:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000099f3ac40. 02000000:00000001:2.0:1788880843.393984:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880843.393985:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880843.393986:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880843.393987:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880843.393989:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f9812869. 02000000:00000001:2.0:1788880843.393991:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880843.393992:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880843.393993:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:2.0:1788880843.393996:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880843.393997:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880843.393998:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880843.394002:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880843.394003:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880843.394005:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880843.394006:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880843.454339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880843.454345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880843.454349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880843.454350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880843.454400:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880843.454403:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880843.454406:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880844.478327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880844.478332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880844.478336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880844.478337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880844.478397:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880844.478401:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880844.478404:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880845.502339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880845.502346:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880845.502348:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880845.502350:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880845.502354:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880845.502356:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880845.502357:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880845.502358:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880845.502361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880845.502362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880845.502431:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880845.502436:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880845.502438:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880846.526347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880846.526353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880846.526356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880846.526357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880846.526413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880846.526417:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880846.526420:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880847.550356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880847.550364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880847.550371:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880847.550375:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880847.550378:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880847.550381:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880847.550384:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880847.550387:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880847.550397:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880847.550401:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880847.550482:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880847.550489:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880847.550494:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880848.510349:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306479567, at 4306479616 00000100:00000040:1.0:1788880848.510349:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306479567, at 4306479616 00000100:00000040:2.0:1788880848.510349:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306479567, at 4306479616 00000100:00000010:1.0:1788880848.510366:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000668fd8a. 00000100:00000010:2.0:1788880848.510371:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d54626ec. 00000100:00000010:0.0:1788880848.510372:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d6fe56. 00000100:00000040:2.0:1788880848.510387:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880848.510392:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000040:3.0:1788880848.510406:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306479567, at 4306479616 00000100:00000010:2.0:1788880848.510409:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880848.510416:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:3.0:1788880848.510418:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001bac11fc. 00000100:00000001:2.0:1788880848.510420:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880848.510423:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880848.510427:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880848.510434:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f9812869. 00000100:00000001:2.0:1788880848.510439:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880848.510447:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880848.510454:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880848.510456:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880848.510460:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_03+3] req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880848.510483:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880848.510485:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880848.510487:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880848.510489:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000020:00000040:2.0:1788880848.510490:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880848.510492:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880848.510493:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880848.510495:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880848.510498:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:2.0:1788880848.510501:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880848.510504:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880848.510509:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880848.510513:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880848.510514:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880848.510516:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880848.510518:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880848.510522:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_00+0] req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880848.510528:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880848.510543:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880848.510545:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880848.510548:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880848.510549:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880848.510553:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880848.510553:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880848.510557:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880848.510561:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780691072:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880848.510566:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880848.510566:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880848.510567:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880848.510570:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:2.0:1788880848.510570:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 00000100:00000001:0.0:1788880848.510572:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788880848.510573:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000020:00000040:2.0:1788880848.510573:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880848.510577:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880848.510577:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:0.0:1788880848.510578:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880848.510580:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880848.510582:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:2.0:1788880848.510583:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880848.510588:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 02000000:00000010:1.0:1788880848.510589:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000c21a3ff8. 00000100:00000001:2.0:1788880848.510591:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880848.510593:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880848.510597:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880848.510601:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880848.510602:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880848.510603:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c21d159e. 02000000:00000001:0.0:1788880848.510605:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880848.510606:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788880848.510608:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880848.510608:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880848.510610:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880848.510612:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_01+1] req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880848.510613:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780691200:192.168.204.111@tcp:400: 00000400:00000010:1.0:1788880848.510613:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d71e0867. 00000100:00000001:0.0:1788880848.510617:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880848.510620:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880848.510625:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880848.510627:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880848.510628:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:0.0:1788880848.510629:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880848.510629:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780691072, portal 25 00000100:00000040:2.0:1788880848.510632:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11817) 02000000:00000001:0.0:1788880848.510634:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880848.510636:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000010:0.0:1788880848.510637:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f5272de5. 00000100:00000001:1.0:1788880848.510638:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 02000000:00000001:0.0:1788880848.510642:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880848.510648:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000003d054356. 00000100:00000040:1.0:1788880848.510649:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880864 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880848.510658:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880848.510663:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880848.510666:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ff62fd67. 00000400:00000010:1.0:1788880848.510668:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000641230df. 00000100:00000200:0.0:1788880848.510671:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780691200, portal 10 00000100:00000200:1.0:1788880848.510673:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780691072, offset 0 00000100:00000001:0.0:1788880848.510675:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880848.510678:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 00000400:00000200:1.0:1788880848.510683:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880848.510688:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880848.510693:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880848.510695:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880848.510696:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:0.0:1788880848.510699:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000096cf0c4a. 00000400:00000200:1.0:1788880848.510702:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:0.0:1788880848.510703:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780691200, offset 0 00000400:00000200:1.0:1788880848.510707:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880848.510711:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880848.510714:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880848.510718:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880848.510721:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [477:477:256:4294967295] 192.168.204.111@tcp LPNI seq info [477:477:8:4294967295] 00000400:00000200:1.0:1788880848.510732:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880848.510739:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880848.510744:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 00000800:00000200:1.0:1788880848.510751:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880848.510760:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880848.510764:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880848.510765:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880848.510774:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880848.510778:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880848.510778:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880848.510782:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880848.510784:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306484687 00000400:00000200:0.0:1788880848.510788:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:1.0:1788880848.510789:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000400:00000200:0.0:1788880848.510794:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [478:478:255:4294967295] 192.168.204.111@tcp LPNI seq info [478:478:7:4294967295] 00000100:00000001:1.0:1788880848.510795:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1788880848.510800:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306484687 00000100:00000010:1.0:1788880848.510803:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000100:00000001:1.0:1788880848.510806:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880848.510811:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880848.510812:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306484687 00000800:00000200:0.0:1788880848.510820:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880848.510821:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000800:00000010:0.0:1788880848.510826:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000bd68c00a. 00000100:00000001:1.0:1788880848.510826:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880848.510828:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880848.510829:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306484687 00000100:00000001:3.0:1788880848.510833:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880848.510834:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:1.0:1788880848.510834:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000100:00000001:1.0:1788880848.510840:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880848.510842:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880848.510844:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880848.510844:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880848.510845:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880848.510848:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880848.510849:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880848.510850:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000bd68c00a type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880848.510852:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880848.510854:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880848.510857:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880848.510859:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880848.510860:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:0.0:1788880848.510862:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 186113 usecs (187 jiffies) 00000100:00000001:1.0:1788880848.510864:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:2.0:1788880848.510865:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000100:00000001:0.0:1788880848.510867:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306479803 00000100:00000001:1.0:1788880848.510868:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880848.510872:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880848.510875:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880848.510877:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880848.510880:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306484687 00000100:00000001:1.0:1788880848.510881:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880848.510881:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880848.510884:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306479803 00000100:00000001:3.0:1788880848.510884:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880848.510887:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880848.510888:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880848.510890:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306484687 02000000:00000001:3.0:1788880848.510890:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788880848.510893:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306479803 02000000:00000001:3.0:1788880848.510894:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880848.510897:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000641230df 02000000:00000001:3.0:1788880848.510897:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880848.510901:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306484687 00000400:00000010:2.0:1788880848.510901:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000641230df. 00000100:00100000:3.0:1788880848.510904:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780691328:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880848.510905:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306479803 00000100:00000001:2.0:1788880848.510907:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880848.510908:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880848.510910:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306484687 02000000:00000001:3.0:1788880848.510912:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880848.510913:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788880848.510913:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880864 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880848.510915:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880848.510917:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880848.510919:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880848.510921:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880848.510924:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880848.510926:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880848.510927:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880848.510928:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:2.0:1788880848.510928:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880864 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880848.510930:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880848.510931:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880848.510933:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880848.510934:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880848.510948:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880848.510950:0:201278:0:(events.c:68:request_out_callback()) Process leaving 02000000:00000010:3.0:1788880848.510957:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002679a1a9. 02000000:00000001:3.0:1788880848.510960:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880848.510963:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000623a1d39. 00000400:00000010:3.0:1788880848.510967:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000100:00000200:3.0:1788880848.510969:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780691328, portal 10 00000100:00000001:3.0:1788880848.510970:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880848.510971:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938435456 : -126823771116160 : ffff8ca78a2c6d80) 00000100:00000040:3.0:1788880848.510976:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880848.510986:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1788880848.510987:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:3.0:1788880848.510990:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880848.510991:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000bd68c00a. 00000400:00000010:3.0:1788880848.510993:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000400:00000200:2.0:1788880848.510996:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:3.0:1788880848.510996:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780691328, offset 0 00000400:00000200:2.0:1788880848.511004:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880848.511005:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880848.511012:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880848.511014:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880848.511016:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880848.511018:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880848.511019:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880848.511021:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [479:479:255:4294967295] 192.168.204.111@tcp LPNI seq info [479:479:7:4294967295] 00000400:00000200:3.0:1788880848.511026:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:2.0:1788880848.511030:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000096cf0c4a 00000800:00000200:3.0:1788880848.511030:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880848.511033:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000096cf0c4a. 00000800:00000010:3.0:1788880848.511035:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:2.0:1788880848.511036:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880848.511041:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880848.511047:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880848.511048:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880848.511051:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880848.511052:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880848.511079:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880848.511086:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880848.511088:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880848.511099:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880848.511103:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13321 usecs (14 jiffies) 00000100:00000001:3.0:1788880848.511105:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306479630 00000100:00000001:3.0:1788880848.511107:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306484687 00000100:00000001:3.0:1788880848.511109:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306479630 00000100:00000001:3.0:1788880848.511110:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306484687 00000100:00000001:3.0:1788880848.511111:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306479630 00000100:00000001:3.0:1788880848.511112:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306484687 00000100:00000001:3.0:1788880848.511113:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306479630 00000100:00000001:3.0:1788880848.511113:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306484687 00000100:00000001:3.0:1788880848.511119:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880848.511121:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880848.511123:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880848.511124:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880848.511127:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880848.511128:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880848.511130:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880848.511131:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880848.511132:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880848.511133:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880848.511184:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880848.511187:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:0.0:1788880848.511190:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880848.511194:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880848.511197:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00000400:00000010:0.0:1788880848.511198:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000100:00000001:0.0:1788880848.511201:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880848.511204:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880848.511210:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880848.511212:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880848.511218:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880848.511219:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880848.513131:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880848.513188:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880848.513193:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880848.513196:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880848.513205:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880848.513217:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0e700 00000400:00000200:0.0:1788880848.513230:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xfb5 [1] + 224 00000800:00000001:0.0:1788880848.513255:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880848.513259:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880848.513262:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880848.513266:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880848.513270:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880848.513276:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880848.513286:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880848.513298:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880848.513306:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880848.513309:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880848.513311:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880848.513312:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880848.513317:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880848.513324:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0e780 00000400:00000200:0.0:1788880848.513329:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xfc1 [1] + 224 00000800:00000001:0.0:1788880848.513332:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880848.513335:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880848.513337:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880848.513340:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880848.513343:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880848.513346:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880848.513353:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880848.513383:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880848.513389:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880848.513393:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880848.513395:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880848.513398:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880848.513403:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880848.513410:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0e680 00000100:00000001:1.0:1788880848.513412:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880848.513417:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xfb1 [1] + 224 00000100:00000001:1.0:1788880848.513417:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880848.513423:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880848.513424:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880848.513427:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880848.513429:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000623a1d39. 00000800:00000001:0.0:1788880848.513430:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880848.513435:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880848.513435:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000400:00000010:1.0:1788880848.513439:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000400:00000200:0.0:1788880848.513440:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880848.513446:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880848.513447:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880848.513453:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880848.513454:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880848.513485:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880848.513486:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880848.513497:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880848.513499:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880848.513501:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880848.513505:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880848.513507:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880848.513522:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880848.513525:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880848.513527:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880848.513531:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880848.513536:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880848.513537:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=76 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880848.513558:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=10 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880848.513562:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880848.513564:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880848.513567:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880848.513573:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880848.513573:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880848.513577:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880848.513582:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00010000:00000001:1.0:1788880848.513584:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880848.513585:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880848.513589:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c21d159e. 00000100:00000001:1.0:1788880848.513593:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000200:3.0:1788880848.513593:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d71e0867 00000100:00000040:1.0:1788880848.513598:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000010:3.0:1788880848.513598:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d71e0867. 00000100:00000001:1.0:1788880848.513602:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880848.513604:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880848.513605:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880848.513611:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880848.513613:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880848.513639:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880848.513642:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:3.0:1788880848.513643:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880848.513652:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880848.513653:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880848.513655:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880848.513656:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880848.513657:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880848.513660:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880848.513662:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880848.513663:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880848.513666:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880848.513671:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=76 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880848.513679:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=76 v=5 (1 1 1 1) 00000100:00100000:1.0:1788880848.513681:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780691328:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880848.513686:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880848.513689:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880848.513689:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880848.513692:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880848.513694:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00010000:00000001:3.0:1788880848.513695:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880848.513699:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000175072aa x1875776780691328/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880848.513699:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880848.513707:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880848.513710:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880848.513714:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002679a1a9. 00000100:00000040:3.0:1788880848.513716:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880848.513719:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880848.513723:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880848.513726:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880848.513731:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880848.513731:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880848.513736:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880848.513738:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880848.513741:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 00000100:00000040:3.0:1788880848.513742:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880848.513753:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00100000:3.0:1788880848.513757:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780691072:192.168.204.111@tcp:400: 02000000:00000001:1.0:1788880848.513758:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880848.513761:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000175072aa. 00000100:00000001:3.0:1788880848.513764:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880848.513766:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880848.513768:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880848.513769:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880848.513773:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880848.513774:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780691072/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880848.513780:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880848.513782:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880848.513784:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880848.513786:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880848.513788:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880848.513789:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880848.513791:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000c21a3ff8. 02000000:00000001:3.0:1788880848.513796:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880848.513798:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880848.513799:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880848.513802:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880848.513803:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f9812869. 02000000:00000001:3.0:1788880848.513806:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880848.513808:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880848.513809:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:3.0:1788880848.513812:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880848.513813:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880848.513814:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880848.513819:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880848.513820:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880848.513822:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880848.513824:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880848.514198:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880848.514201:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880848.514205:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880848.514207:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880848.514208:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000003d054356. 00000400:00000200:2.0:1788880848.514212:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ff62fd67 00000400:00000010:2.0:1788880848.514213:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ff62fd67. 00000100:00000001:2.0:1788880848.514216:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880848.514219:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880848.514226:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880848.514247:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880848.514249:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880848.514251:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880848.514252:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880848.514253:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880848.514260:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880848.514262:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880848.514265:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=66 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880848.514268:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=66 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880848.514270:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880848.514271:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880848.514272:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880848.514275:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880848.514276:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880848.514277:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880848.514279:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880848.514280:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880848.514283:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880848.514289:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880848.514290:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880848.514291:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880848.514294:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880848.514300:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780691200:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880848.514303:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880848.514305:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880848.514306:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780691200/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880864 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880848.514311:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880848.514312:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880848.514314:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f5272de5. 02000000:00000001:2.0:1788880848.514316:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880848.514318:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880848.514318:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880848.514320:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880848.514321:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 02000000:00000001:2.0:1788880848.514323:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880848.514329:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880848.514330:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000100:00000001:2.0:1788880848.514333:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880848.514334:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880848.514335:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880848.514338:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880848.514340:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880848.514341:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880848.514342:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880848.574334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880848.574340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880848.574344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880848.574345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880848.574393:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880848.574397:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880848.574399:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00000001:1.0:1788880849.534331:0:203401:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000001:1.0:1788880849.534338:0:203401:0:(mgc_request.c:120:config_log_put()) Process leaving 00010000:00000001:1.0:1788880849.598341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880849.598349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880849.598356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880849.598357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880849.598419:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880849.598424:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880849.598427:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880850.622369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880850.622379:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880850.622386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880850.622387:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880850.622448:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880850.622472:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880850.622475:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880851.646344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880851.646354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880851.646362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880851.646363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880851.646397:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880851.646401:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880851.646404:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880852.670324:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880852.670331:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880852.670338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880852.670339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880852.670394:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880852.670398:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880852.670400:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880853.630336:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306484687, at 4306484736 00000100:00000040:1.0:1788880853.630336:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306484687, at 4306484736 00000100:00000040:3.0:1788880853.630336:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306484687, at 4306484736 00000100:00000040:2.0:1788880853.630348:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:0.0:1788880853.630353:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000001:2.0:1788880853.630354:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880853.630354:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000010:1.0:1788880853.630355:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000010:2.0:1788880853.630365:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000020:00000040:2.0:1788880853.630372:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880853.630376:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880853.630379:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880853.630384:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880853.630391:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:2.0:1788880853.630398:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880853.630408:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880853.630419:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880853.630421:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880853.630426:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_02+2] req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880853.630449:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880853.630450:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880853.630453:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880853.630455:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002084a2b4. 00000100:00000001:1.0:1788880853.630466:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880853.630466:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880853.630469:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880853.630471:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880853.630471:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880853.630473:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880853.630475:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880853.630476:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880853.630479:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880853.630483:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880853.630485:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000100:00000001:2.0:1788880853.630491:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880853.630498:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880853.630500:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880853.630501:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880853.630507:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880853.630511:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880853.630513:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880853.630514:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880853.630517:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880853.630519:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780691456:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880853.630525:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002084a2b4] to pc [ptlrpcd_00_03+3] req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880853.630525:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880853.630530:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880853.630534:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880853.630537:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880853.630541:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e5dc9915. 02000000:00000001:3.0:1788880853.630543:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880853.630545:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880853.630549:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880853.630549:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000002629a0e9. 00000100:00000001:2.0:1788880853.630555:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880853.630560:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000884e74b8. 00000400:00000010:3.0:1788880853.630560:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 00000100:00000001:1.0:1788880853.630563:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880853.630563:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880853.630565:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880853.630568:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000200:3.0:1788880853.630568:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780691456, portal 25 02000000:00000001:2.0:1788880853.630570:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880853.630573:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880853.630573:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880853.630574:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880853.630576:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 02000000:00000010:2.0:1788880853.630577:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5d1e8ed. 00000100:00000040:1.0:1788880853.630580:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880853.630582:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880853.630582:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880869 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880853.630590:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880853.630593:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880853.630594:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880853.630596:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880853.630599:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880853.630601:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880853.630603:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880853.630606:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880853.630610:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cf82a3fa. 00000100:00000001:2.0:1788880853.630611:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:1.0:1788880853.630612:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780691584:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880853.630614:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880853.630614:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780691456, offset 0 00000100:00000001:1.0:1788880853.630619:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880853.630622:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880853.630622:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000884e74b8] to pc [ptlrpcd_00_00+0] req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880853.630623:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880853.630625:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880853.630627:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880853.630631:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000001363d535. 00000400:00000200:3.0:1788880853.630638:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788880853.630642:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880853.630643:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:1.0:1788880853.630648:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000100:00000001:2.0:1788880853.630648:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880853.630648:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:2.0:1788880853.630654:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:1.0:1788880853.630655:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000400:00000200:3.0:1788880853.630655:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000200:1.0:1788880853.630658:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780691584, portal 10 00000400:00000200:3.0:1788880853.630658:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880853.630660:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880853.630663:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938429184 : -126823771122432 : ffff8ca78a2c5500) 00000400:00000200:3.0:1788880853.630664:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [480:480:256:4294967295] 192.168.204.111@tcp LPNI seq info [480:480:8:4294967295] 00000100:00000001:0.0:1788880853.630667:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880853.630669:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880853.630671:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880853.630674:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880853.630675:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880853.630677:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11822) 00000400:00000200:3.0:1788880853.630682:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880853.630684:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880853.630688:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880853.630693:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306484687, at 4306484736 00000100:00000040:1.0:1788880853.630696:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880853.630701:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000400:00000010:1.0:1788880853.630702:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000061cfcb81. 00000100:00000001:0.0:1788880853.630705:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000200:3.0:1788880853.630705:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:1.0:1788880853.630707:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780691584, offset 0 00000100:00000001:0.0:1788880853.630708:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880853.630712:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 02000000:00000001:0.0:1788880853.630713:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880853.630714:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880853.630718:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788880853.630719:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880853.630721:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880853.630723:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880853.630724:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880853.630725:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880853.630729:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880853.630730:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880853.630730:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880853.630734:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788880853.630735:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00100000:0.0:1788880853.630737:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780691712:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880853.630739:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880853.630744:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880853.630745:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788880853.630745:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [481:481:255:4294967295] 192.168.204.111@tcp LPNI seq info [481:481:7:4294967295] 02000000:00000001:0.0:1788880853.630749:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880853.630749:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306489807 02000000:00000001:0.0:1788880853.630752:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880853.630755:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008ea3b290. 02000000:00000001:0.0:1788880853.630757:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880853.630760:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 02000000:00000010:0.0:1788880853.630763:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a07dd7e3. 00000100:00000001:3.0:1788880853.630765:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306489807 00000400:00000200:1.0:1788880853.630766:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880853.630768:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880853.630771:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001632a9ba. 00000800:00000200:1.0:1788880853.630773:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880853.630774:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000048a5223f. 00000100:00000001:3.0:1788880853.630775:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000010:1.0:1788880853.630776:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000100:00000001:3.0:1788880853.630780:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306489807 00000800:00000200:1.0:1788880853.630781:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:0.0:1788880853.630782:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000010:3.0:1788880853.630784:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001bac11fc. 00000800:00000200:1.0:1788880853.630787:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880853.630789:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000200:0.0:1788880853.630790:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780691712, portal 10 00000800:00000200:1.0:1788880853.630790:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880853.630793:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306489807 00000100:00000001:0.0:1788880853.630795:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880853.630797:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880853.630797:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000026aaaaf2. 00000100:00000001:0.0:1788880853.630798:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938423808 : -126823771127808 : ffff8ca78a2c4000) 00000100:00000001:3.0:1788880853.630800:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000040:1.0:1788880853.630801:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 184817 usecs (185 jiffies) 00000100:00000001:1.0:1788880853.630804:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306484921 00000100:00000001:3.0:1788880853.630805:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880853.630806:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306489807 00000100:00000040:0.0:1788880853.630809:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880853.630809:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306484921 00000100:00000001:3.0:1788880853.630809:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880853.630814:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306489807 00000100:00000001:1.0:1788880853.630817:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306484921 00000100:00000001:3.0:1788880853.630818:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880853.630821:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306489807 00000100:00000001:3.0:1788880853.630821:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880853.630825:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306484921 00000100:00000001:0.0:1788880853.630828:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880853.630828:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306489807 00000100:00000001:3.0:1788880853.630830:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880853.630832:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880853.630833:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880853.630833:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880853.630837:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880853.630837:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880853.630837:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788880853.630838:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e73139ca. 00000100:00000001:1.0:1788880853.630841:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880853.630841:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000010:2.0:1788880853.630842:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:1.0:1788880853.630843:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880853.630844:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780691712, offset 0 00000100:00000001:3.0:1788880853.630844:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880853.630846:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880853.630849:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880853.630849:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880853.630851:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880853.630855:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880853.630856:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880853.630859:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880853.630860:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880853.630864:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880853.630867:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880853.630868:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880853.630872:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880853.630875:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880853.630878:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880853.630881:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880853.630884:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [482:482:254:4294967295] 192.168.204.111@tcp LPNI seq info [482:482:6:4294967295] 00000400:00000200:0.0:1788880853.630891:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880853.630896:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880853.630896:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cf82a3fa 00000800:00000010:0.0:1788880853.630900:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000087d24527. 00000400:00000010:2.0:1788880853.630900:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cf82a3fa. 00000100:00000001:2.0:1788880853.630906:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:0.0:1788880853.630908:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880853.630912:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880869 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880853.630918:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880853.630923:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880853.630926:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880869 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880853.630930:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000087d24527 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880853.630935:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880853.630937:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880853.630938:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000040:0.0:1788880853.630942:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13251 usecs (14 jiffies) 00000100:00000001:0.0:1788880853.630946:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306484750 00000100:00000001:0.0:1788880853.630948:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306489807 00000100:00000001:0.0:1788880853.630950:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306484750 00000800:00000200:2.0:1788880853.630950:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880853.630953:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000100:00000001:0.0:1788880853.630955:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306489807 00000400:00000200:2.0:1788880853.630957:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880853.630958:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306484750 00000400:00000200:2.0:1788880853.630962:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880853.630963:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306489807 00000400:00000200:2.0:1788880853.630964:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000061cfcb81 00000400:00000010:2.0:1788880853.630967:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000061cfcb81. 00000100:00000001:0.0:1788880853.630969:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306484750 00000100:00000001:2.0:1788880853.630970:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880853.630974:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306489807 00000100:00000200:2.0:1788880853.630974:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880853.630980:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880853.630986:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880853.630987:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880853.630990:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880853.630993:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880853.630998:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880853.631002:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880853.631005:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880853.631010:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880853.631011:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880853.631012:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880853.631014:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880853.631018:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880853.631020:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880853.631038:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880853.631040:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000087d24527. 00000400:00000200:2.0:1788880853.631042:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880853.631045:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880853.631046:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e73139ca 00000400:00000010:2.0:1788880853.631060:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e73139ca. 00000100:00000001:2.0:1788880853.631062:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880853.631065:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880853.631069:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880853.631070:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880853.631074:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880853.631075:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880853.632363:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880853.632374:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880853.632377:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880853.632380:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880853.632387:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880853.632398:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0e800 00000400:00000200:0.0:1788880853.632404:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xfc9 [1] + 224 00000800:00000001:0.0:1788880853.632411:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880853.632418:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880853.632421:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880853.632431:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880853.632436:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880853.632439:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880853.632449:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880853.632465:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880853.632472:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880853.632475:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880853.632483:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880853.632486:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880853.632490:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880853.632492:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880853.632494:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000002629a0e9. 00000400:00000200:3.0:1788880853.632498:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 00000400:00000010:3.0:1788880853.632500:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000100:00000001:3.0:1788880853.632502:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880853.632505:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880853.632514:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880853.632521:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880853.632523:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880853.632525:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880853.632528:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880853.632530:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880853.632532:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880853.632535:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880853.632539:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=81 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880853.632543:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=81 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880853.632545:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880853.632546:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880853.632548:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880853.632551:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880853.632553:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880853.632557:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880853.632628:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880853.632630:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880853.632632:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880853.632635:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880853.632643:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780691456:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880853.632647:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880853.632650:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880853.632652:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780691456/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880853.632658:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880853.632660:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880853.632663:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000800:00000001:0.0:1788880853.632665:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880853.632669:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880853.632671:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880853.632676:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880853.632683:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0e880 00000400:00000200:0.0:1788880853.632688:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xfd1 [1] + 224 00000800:00000001:0.0:1788880853.632692:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880853.632698:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880853.632700:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000010:3.0:1788880853.632700:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e5dc9915. 00000400:00000200:0.0:1788880853.632704:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880853.632708:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880853.632711:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880853.632718:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880853.632735:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880853.632740:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880853.632756:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880853.632758:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000800:00000001:0.0:1788880853.632759:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000001:3.0:1788880853.632761:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880853.632763:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000800:00000001:0.0:1788880853.632765:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880853.632766:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:3.0:1788880853.632766:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880853.632769:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 00000100:00000001:1.0:1788880853.632771:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880853.632772:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880853.632775:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000200:0.0:1788880853.632776:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:1.0:1788880853.632776:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000010:3.0:1788880853.632777:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000400:00000001:1.0:1788880853.632781:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880853.632782:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880853.632784:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880853.632786:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880853.632788:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0e900 00000400:00000010:1.0:1788880853.632790:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000100:00000001:3.0:1788880853.632793:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788880853.632795:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000100:00000001:3.0:1788880853.632796:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880853.632800:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:1.0:1788880853.632805:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:0.0:1788880853.632809:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xfd9 [1] + 224 00000100:00000200:1.0:1788880853.632812:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880853.632817:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880853.632824:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880853.632826:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880853.632829:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880853.632838:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880853.632839:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880853.632840:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880853.632843:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880853.632844:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880853.632846:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880853.632849:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880853.632851:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000200:0.0:1788880853.632853:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880853.632853:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880853.632856:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880853.632859:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880853.632864:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=71 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880853.632870:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=71 v=5 (1 1 1 1) 00000100:00000040:0.0:1788880853.632874:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880853.632874:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880853.632876:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880853.632879:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880853.632882:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880853.632886:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880853.632888:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880853.632891:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880853.632892:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880853.632895:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880853.632901:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880853.632902:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880853.632903:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880853.632906:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880853.632909:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788880853.632910:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880853.632914:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1788880853.632922:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780691584:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880853.632932:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880853.632934:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880853.632937:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002084a2b4 x1875776780691584/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880853.632937:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880853.632939:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880853.632941:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880853.632942:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880853.632944:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000010:2.0:1788880853.632944:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000048a5223f. 02000000:00000001:1.0:1788880853.632945:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880853.632947:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000001363d535. 00000400:00000200:2.0:1788880853.632947:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:2.0:1788880853.632948:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 02000000:00000001:1.0:1788880853.632949:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880853.632950:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880853.632950:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000040:1.0:1788880853.632952:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880853.632953:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000200:2.0:1788880853.632953:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880853.632955:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 02000000:00000001:1.0:1788880853.632957:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880853.632958:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880853.632959:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002084a2b4. 00000100:00000200:2.0:1788880853.632961:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880853.632962:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880853.632963:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880853.632964:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880853.632967:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880853.632969:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880853.632970:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880853.632971:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880853.632974:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880853.632976:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880853.632977:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880853.632977:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880853.632978:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880853.632979:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880853.632980:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880853.632983:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=81 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880853.632986:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=15 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880853.632988:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880853.632989:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880853.632991:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880853.632994:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880853.633001:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880853.633003:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880853.633004:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880853.633006:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880853.633007:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880853.633009:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880853.633016:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880853.633017:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880853.633018:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880853.633019:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880853.633024:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780691712:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880853.633026:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880853.633035:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880853.633037:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000884e74b8 x1875776780691712/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880869 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880853.633043:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880853.633045:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880853.633056:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a07dd7e3. 02000000:00000001:2.0:1788880853.633059:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880853.633065:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880853.633066:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880853.633069:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880853.633070:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5d1e8ed. 02000000:00000001:2.0:1788880853.633072:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880853.633073:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880853.633074:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000884e74b8. 00000100:00000001:2.0:1788880853.633077:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880853.633078:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880853.633079:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880853.633083:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880853.633084:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880853.633085:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880853.633087:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880853.694338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880853.694345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880853.694350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880853.694351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880853.694408:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880853.694413:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880853.694416:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880854.718354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880854.718361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880854.718368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880854.718369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880854.718433:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880854.718437:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880854.718439:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880855.742298:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880855.742306:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880855.742308:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880855.742310:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880855.742314:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880855.742316:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880855.742325:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880855.742327:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880855.742331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880855.742332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880855.742357:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880855.742361:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880855.742363:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880856.766304:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880856.766309:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880856.766314:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880856.766315:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880856.766346:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880856.766350:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880856.766353:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880857.790344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880857.790359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880857.790366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880857.790369:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880857.790371:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880857.790375:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880857.790378:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880857.790380:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880857.790381:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880857.790383:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880857.790440:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880857.790445:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880857.790448:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880858.750287:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306489807, at 4306489856 00000100:00000040:3.0:1788880858.750292:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306489807, at 4306489856 00000100:00000040:2.0:1788880858.750294:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306489807, at 4306489856 00000100:00000010:1.0:1788880858.750304:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001632a9ba. 00000100:00000010:3.0:1788880858.750313:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000026aaaaf2. 00000100:00000010:2.0:1788880858.750322:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001bac11fc. 00000100:00000040:0.0:1788880858.750371:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880858.750377:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880858.750382:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000b4a4221. 00000020:00000040:0.0:1788880858.750386:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880858.750390:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880858.750393:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880858.750397:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880858.750404:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003806e31f. 00000100:00000001:0.0:1788880858.750410:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880858.750418:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880858.750428:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880858.750431:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880858.750436:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000b4a4221] to pc [ptlrpcd_00_01+1] req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880858.750471:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880858.750474:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880858.750477:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880858.750480:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d4e901df. 00000020:00000040:0.0:1788880858.750482:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880858.750484:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880858.750485:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880858.750487:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880858.750488:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880858.750491:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880858.750492:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880858.750497:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000013bf63f4. 00000100:00000040:2.0:1788880858.750497:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880858.750498:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880858.750502:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880858.750503:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880858.750511:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:0.0:1788880858.750513:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880858.750513:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880858.750518:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880858.750521:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880858.750526:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788880858.750527:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880858.750530:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880858.750532:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780691840:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880858.750536:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d4e901df] to pc [ptlrpcd_00_02+2] req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880858.750545:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880858.750551:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880858.750553:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880858.750555:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880858.750558:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000088ba1b50. 02000000:00000001:2.0:1788880858.750560:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880858.750563:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880858.750565:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000002869d45. 00000100:00000040:0.0:1788880858.750567:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:2.0:1788880858.750570:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009543d7d4. 00000100:00000001:0.0:1788880858.750571:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880858.750573:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:0.0:1788880858.750576:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000fd20c80. 00000100:00000200:2.0:1788880858.750576:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780691840, portal 25 00000100:00000001:3.0:1788880858.750577:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880858.750579:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:0.0:1788880858.750581:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880858.750581:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880858.750584:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849373056 : -126822860178560 : ffff8ca7c0783b80) 00000100:00000040:3.0:1788880858.750586:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880858.750587:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880858.750589:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:2.0:1788880858.750591:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880874 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880858.750593:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880858.750599:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000044c69dd2. 00000100:00000001:0.0:1788880858.750604:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880858.750608:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880858.750611:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880858.750613:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880858.750614:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880858.750615:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880858.750618:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880858.750621:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880858.750622:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880858.750626:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c0c07443. 00000100:00100000:3.0:1788880858.750629:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780691968:192.168.204.111@tcp:400: 00000100:00000200:2.0:1788880858.750631:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780691840, offset 0 00000100:00000001:0.0:1788880858.750634:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880858.750637:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880858.750639:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880858.750642:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880858.750643:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880858.750643:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880858.750644:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880858.750650:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000115c4ce2. 00000100:00000040:0.0:1788880858.750653:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000fd20c80] to pc [ptlrpcd_00_03+3] req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880858.750654:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880858.750657:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:3.0:1788880858.750662:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000002629a0e9. 00000400:00000200:2.0:1788880858.750665:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880858.750670:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:3.0:1788880858.750670:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 00000400:00000200:2.0:1788880858.750677:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880858.750678:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880858.750678:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780691968, portal 10 00000400:00000200:2.0:1788880858.750682:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:0.0:1788880858.750684:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880858.750684:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788880858.750688:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [483:483:256:4294967295] 192.168.204.111@tcp LPNI seq info [483:483:8:4294967295] 00000100:00000001:3.0:1788880858.750688:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849369472 : -126822860182144 : ffff8ca7c0782d80) 00000100:00000040:0.0:1788880858.750692:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788880858.750699:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11827) 00000100:00000040:3.0:1788880858.750701:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880858.750709:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880858.750712:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880858.750714:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306489807, at 4306489856 00000100:00000040:3.0:1788880858.750719:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880858.750722:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880858.750723:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008ea3b290. 00000400:00000010:3.0:1788880858.750723:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000082116f14. 00000100:00000200:3.0:1788880858.750729:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780691968, offset 0 00000800:00000010:2.0:1788880858.750731:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5d1e8ed. 00000400:00000200:3.0:1788880858.750735:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880858.750741:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:2.0:1788880858.750741:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880858.750743:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880858.750746:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880858.750749:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:3.0:1788880858.750750:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880858.750754:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:0.0:1788880858.750756:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880858.750761:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5d1e8ed type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880858.750761:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880858.750766:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880858.750771:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880858.750775:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880858.750776:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:3.0:1788880858.750776:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [484:484:255:4294967295] 192.168.204.111@tcp LPNI seq info [484:484:7:4294967295] 00000100:00000001:0.0:1788880858.750779:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880858.750783:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880858.750785:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306494927 02000000:00000001:0.0:1788880858.750786:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880858.750790:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:2.0:1788880858.750792:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005fb7b615. 02000000:00000001:0.0:1788880858.750793:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880858.750797:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880858.750800:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00100000:0.0:1788880858.750801:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780692096:192.168.204.111@tcp:400: 00000800:00000010:3.0:1788880858.750804:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000001:2.0:1788880858.750805:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306494927 00000100:00000001:0.0:1788880858.750807:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000010:2.0:1788880858.750811:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008b80730b. 02000000:00000001:0.0:1788880858.750812:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880858.750815:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880858.750816:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:3.0:1788880858.750816:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:0.0:1788880858.750819:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880858.750821:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306494927 02000000:00000010:0.0:1788880858.750825:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000010c9710f. 00000100:00000010:2.0:1788880858.750825:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fbf478f2. 00000800:00000200:3.0:1788880858.750826:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880858.750828:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 02000000:00000001:0.0:1788880858.750831:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880858.750832:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880858.750833:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306494927 00000400:00000010:0.0:1788880858.750836:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000f4196098. 00000100:00000010:2.0:1788880858.750839:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045d7c07c. 00000100:00000001:3.0:1788880858.750840:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880858.750842:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000010:0.0:1788880858.750843:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002aa6ba25. 00000100:00000040:3.0:1788880858.750846:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 183527 usecs (184 jiffies) 00000100:00000200:0.0:1788880858.750849:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780692096, portal 10 00000100:00000001:2.0:1788880858.750850:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880858.750851:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306490040 00000100:00000001:0.0:1788880858.750854:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880858.750857:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849357824 : -126822860193792 : ffff8ca7c0780000) 00000100:00000001:2.0:1788880858.750857:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880858.750857:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306494927 00000100:00000001:2.0:1788880858.750863:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880858.750863:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306490040 00000100:00000001:3.0:1788880858.750866:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306494927 00000100:00000040:0.0:1788880858.750868:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880858.750868:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880858.750869:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306490040 00000100:00000001:3.0:1788880858.750874:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306494927 00000100:00000001:3.0:1788880858.750878:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306490040 00000100:00000001:2.0:1788880858.750879:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880858.750882:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306494927 00000100:00000001:2.0:1788880858.750884:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880858.750888:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880858.750890:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880858.750891:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880858.750893:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880858.750894:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880858.750898:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880858.750899:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880858.750899:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880858.750902:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 00000100:00000001:2.0:1788880858.750905:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880858.750905:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880858.750906:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780692096, offset 0 00000100:00000001:3.0:1788880858.750913:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880858.750916:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880858.750917:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880858.750922:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880858.750924:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880858.750926:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880858.750928:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880858.750931:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880858.750933:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880858.750939:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880858.750939:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880858.750945:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880858.750962:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [485:485:254:4294967295] 192.168.204.111@tcp LPNI seq info [485:485:6:4294967295] 00000800:00000200:2.0:1788880858.750967:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880858.750970:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5d1e8ed. 00000400:00000200:0.0:1788880858.750977:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:2.0:1788880858.750977:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880858.750987:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000200:0.0:1788880858.750988:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880858.750994:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000fc8f9b7c. 00000400:00000200:2.0:1788880858.750995:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c0c07443 00000400:00000010:2.0:1788880858.750999:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c0c07443. 00000800:00000200:0.0:1788880858.751000:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880858.751004:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:0.0:1788880858.751008:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:2.0:1788880858.751012:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880874 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880858.751013:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000fc8f9b7c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880858.751019:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880858.751024:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13176 usecs (14 jiffies) 00000100:00000001:0.0:1788880858.751028:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306489870 00000100:00000001:2.0:1788880858.751029:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880858.751032:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306494927 00000100:00000040:2.0:1788880858.751035:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880874 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880858.751036:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306489870 00000100:00000001:0.0:1788880858.751038:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306494927 00000100:00000001:0.0:1788880858.751039:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306489870 00000100:00000001:0.0:1788880858.751041:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306494927 00000100:00000001:0.0:1788880858.751042:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306489870 00000100:00000001:0.0:1788880858.751044:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306494927 00000100:00000001:0.0:1788880858.751053:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880858.751057:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880858.751061:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880858.751064:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880858.751068:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880858.751070:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880858.751071:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880858.751073:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880858.751076:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880858.751077:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880858.751079:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880858.751081:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880858.751087:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880858.751090:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:2.0:1788880858.751093:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880858.751097:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880858.751100:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000082116f14 00000400:00000010:2.0:1788880858.751101:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000082116f14. 00000100:00000001:2.0:1788880858.751103:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880858.751106:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880858.751112:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880858.751113:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880858.751119:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880858.751121:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880858.751178:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880858.751179:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000fc8f9b7c. 00000400:00000200:2.0:1788880858.751182:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880858.751184:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880858.751186:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000400:00000010:2.0:1788880858.751187:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000001:2.0:1788880858.751188:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880858.751190:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880858.751195:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880858.751196:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880858.751199:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880858.751200:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880858.752682:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880858.752693:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880858.752696:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880858.752711:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880858.752718:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880858.752729:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0e980 00000400:00000200:0.0:1788880858.752736:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xfe1 [1] + 224 00000800:00000001:0.0:1788880858.752742:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880858.752746:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880858.752748:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880858.752751:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880858.752755:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880858.752759:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880858.752768:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880858.752784:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880858.752792:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880858.752796:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880858.752799:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880858.752801:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880858.752807:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880858.752815:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ea00 00000100:00000001:2.0:1788880858.752816:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880858.752819:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880858.752831:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xfe9 [1] + 224 00000800:00000001:0.0:1788880858.752836:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:2.0:1788880858.752838:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880858.752841:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:0.0:1788880858.752842:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880858.752843:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000002869d45. 00000400:00000200:0.0:1788880858.752846:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880858.752849:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009543d7d4 00000400:00000200:0.0:1788880858.752853:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:2.0:1788880858.752853:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009543d7d4. 00000100:00000001:2.0:1788880858.752859:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880858.752860:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880858.752865:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880858.752867:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880858.752890:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880858.752892:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880858.752911:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880858.752915:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880858.752917:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880858.752920:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880858.752921:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880858.752922:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880858.752925:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880858.752927:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880858.752929:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:2.0:1788880858.752931:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=86 v=5 (5 5 5 5) 00000800:00000001:0.0:1788880858.752934:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880858.752936:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=86 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880858.752938:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:0.0:1788880858.752939:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880858.752940:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880858.752941:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880858.752943:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880858.752945:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788880858.752946:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000001:3.0:1788880858.752947:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880858.752961:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00010000:00000001:2.0:1788880858.752962:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880858.752963:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000002629a0e9. 00000100:00000001:2.0:1788880858.752965:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880858.752970:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880858.752970:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 00000100:00000040:2.0:1788880858.752973:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880858.752975:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000100:00000001:3.0:1788880858.752981:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:0.0:1788880858.752985:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ea80 00000100:00000001:2.0:1788880858.752985:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880858.752989:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:3.0:1788880858.752989:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880858.752992:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880858.752997:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xff1 [1] + 224 00000100:00000040:2.0:1788880858.752999:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880858.753007:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:2.0:1788880858.753019:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780691840:192.168.204.111@tcp:400: 00000100:00000200:3.0:1788880858.753019:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880858.753026:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880858.753027:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880858.753030:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:0.0:1788880858.753032:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880858.753033:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880858.753034:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880858.753038:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:2.0:1788880858.753040:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000b4a4221 x1875776780691840/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880858.753041:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:3.0:1788880858.753042:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880858.753045:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880858.753048:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880858.753050:0:201280:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1788880858.753054:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880858.753055:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:0.0:1788880858.753058:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880858.753059:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880858.753062:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000088ba1b50. 00000100:00001000:3.0:1788880858.753062:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=76 v=5 (5 5 5 5) 02000000:00000001:2.0:1788880858.753067:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880858.753069:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00001000:3.0:1788880858.753071:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=76 v=5 (1 1 1 1) 00000020:00000040:2.0:1788880858.753072:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880858.753076:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880858.753080:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003806e31f. 00000100:00000001:3.0:1788880858.753081:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:0.0:1788880858.753083:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880858.753085:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880858.753086:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880858.753089:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880858.753091:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000b4a4221. 00010000:00000001:3.0:1788880858.753091:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880858.753096:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880858.753097:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880858.753100:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880858.753102:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880858.753103:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880858.753105:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880858.753108:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880858.753108:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880858.753110:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880858.753110:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880858.753114:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880858.753115:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880858.753118:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880858.753119:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880858.753122:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880858.753127:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880858.753129:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880858.753131:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880858.753133:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880858.753146:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880858.753151:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880858.753154:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780691968:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880858.753155:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880858.753160:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880858.753162:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880858.753162:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880858.753165:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000010:1.0:1788880858.753166:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000f4196098. 00000100:00000040:3.0:1788880858.753170:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d4e901df x1875776780691968/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880858.753171:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002aa6ba25 00000400:00000010:1.0:1788880858.753175:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002aa6ba25. 00000100:00000001:1.0:1788880858.753179:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880858.753182:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:1.0:1788880858.753185:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880858.753186:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880858.753189:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000115c4ce2. 02000000:00000001:3.0:1788880858.753194:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880858.753196:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880858.753199:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000200:1.0:1788880858.753200:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880858.753202:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880858.753205:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000013bf63f4. 02000000:00000001:3.0:1788880858.753209:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880858.753212:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880858.753213:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000010:3.0:1788880858.753214:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d4e901df. 00000100:00000001:1.0:1788880858.753216:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880858.753218:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880858.753219:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880858.753221:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880858.753222:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880858.753224:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880858.753227:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880858.753228:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880858.753231:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880858.753253:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880858.753257:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880858.753260:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880858.753263:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880858.753267:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=86 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880858.753270:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=20 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880858.753272:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880858.753273:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880858.753275:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880858.753278:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880858.753286:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880858.753288:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880858.753289:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880858.753291:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880858.753292:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880858.753294:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880858.753300:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880858.753302:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880858.753303:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880858.753305:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880858.753312:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780692096:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880858.753316:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880858.753317:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880858.753319:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000fd20c80 x1875776780692096/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880874 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880858.753325:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880858.753326:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880858.753328:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000010c9710f. 02000000:00000001:1.0:1788880858.753331:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880858.753336:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880858.753337:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880858.753340:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880858.753341:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000044c69dd2. 02000000:00000001:1.0:1788880858.753343:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880858.753345:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880858.753346:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000fd20c80. 00000100:00000001:1.0:1788880858.753348:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880858.753349:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880858.753351:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880858.753354:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880858.753355:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880858.753357:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880858.753358:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880858.814329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880858.814334:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880858.814339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880858.814340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880858.814374:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880858.814377:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880858.814380:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880859.838346:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880859.838353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880859.838360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880859.838362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880859.838437:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880859.838440:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880859.838442:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880860.862341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880860.862346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880860.862352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880860.862353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880860.862383:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880860.862386:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880860.862389:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880861.886428:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880861.886434:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880861.886437:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880861.886438:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880861.886496:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880861.886500:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880861.886503:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880862.910301:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880862.910308:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880862.910314:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880862.910315:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880862.910362:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880862.910366:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880862.910368:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880863.870299:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306494927, at 4306494976 00000100:00000040:3.0:1788880863.870305:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306494927, at 4306494976 00000100:00000040:0.0:1788880863.870306:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306494927, at 4306494976 00000100:00000040:2.0:1788880863.870314:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306494927, at 4306494976 00000100:00000010:3.0:1788880863.870316:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045d7c07c. 00000100:00000010:0.0:1788880863.870318:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005fb7b615. 00000100:00000010:2.0:1788880863.870324:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fbf478f2. 00000100:00000010:1.0:1788880863.870344:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008b80730b. 00000100:00000040:0.0:1788880863.870355:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880863.870359:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880863.870364:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000363de9c9. 00000020:00000040:0.0:1788880863.870368:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880863.870371:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880863.870373:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880863.870377:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880863.870383:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000044c69dd2. 00000100:00000001:0.0:1788880863.870389:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880863.870398:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880863.870405:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880863.870407:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880863.870411:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000363de9c9] to pc [ptlrpcd_00_00+0] req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880863.870428:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880863.870429:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880863.870432:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880863.870434:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c2a263b1. 00000020:00000040:0.0:1788880863.870436:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880863.870437:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880863.870438:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880863.870439:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880863.870441:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000013bf63f4. 00000100:00000001:2.0:1788880863.870452:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880863.870456:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880863.870458:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880863.870461:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880863.870462:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880863.870468:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880863.870470:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880863.870471:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880863.870474:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c2a263b1] to pc [ptlrpcd_00_01+1] req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880863.870489:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880863.870494:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880863.870496:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880863.870497:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880863.870498:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880863.870498:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880863.870500:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880863.870502:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880863.870505:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cdd8dec9. 00000100:00000040:1.0:1788880863.870506:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1788880863.870509:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:2.0:1788880863.870513:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880863.870517:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880863.870518:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880863.870520:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880863.870521:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880863.870522:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880863.870522:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880863.870525:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880863.870528:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003806e31f. 02000000:00000001:1.0:1788880863.870528:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00100000:2.0:1788880863.870530:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780692224:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880863.870532:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880863.870532:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880863.870539:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880863.870539:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880863.870539:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00100000:1.0:1788880863.870547:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780692352:192.168.204.111@tcp:400: 02000000:00000001:2.0:1788880863.870547:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880863.870548:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880863.870551:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880863.870552:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880863.870556:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880863.870558:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880863.870559:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cdd8dec9] to pc [ptlrpcd_00_02+2] req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880863.870563:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000027d4cece. 02000000:00000001:1.0:1788880863.870565:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880863.870568:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880863.870569:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880863.870571:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880863.870575:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000001363d535. 02000000:00000001:1.0:1788880863.870578:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880863.870578:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000002869d45. 00000400:00000010:1.0:1788880863.870583:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000100:00000001:0.0:1788880863.870585:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880863.870587:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009543d7d4. 00000400:00000010:1.0:1788880863.870588:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000100:00000040:0.0:1788880863.870589:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880863.870595:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000200:1.0:1788880863.870595:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780692352, portal 10 00000100:00000200:2.0:1788880863.870595:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780692224, portal 25 00000100:00000001:1.0:1788880863.870601:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880863.870602:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880863.870603:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11832) 00000100:00000001:1.0:1788880863.870604:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364992 : -126822860186624 : ffff8ca7c0781c00) 00000100:00000001:2.0:1788880863.870605:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849365888 : -126822860185728 : ffff8ca7c0781f80) 00000100:00000001:0.0:1788880863.870611:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880863.870614:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880863.870614:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880879 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880863.870616:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:0.0:1788880863.870634:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880863.870643:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880863.870643:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880863.870644:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788880863.870651:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880863.870652:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880863.870657:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000038c5ac58. 00000400:00000010:2.0:1788880863.870658:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c0c07443. 00000100:00000001:0.0:1788880863.870659:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880863.870662:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780692352, offset 0 00000100:00000001:0.0:1788880863.870663:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880863.870663:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780692224, offset 0 00000100:00000001:0.0:1788880863.870665:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880863.870669:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880863.870673:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880863.870677:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880863.870683:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880863.870686:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880863.870689:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880863.870689:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880863.870691:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880863.870692:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880863.870694:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880863.870696:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880863.870696:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880863.870702:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cdd8dec9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780692480:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880863.870703:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880863.870707:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880863.870710:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1788880863.870711:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:0.0:1788880863.870712:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880863.870714:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880863.870716:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000010:0.0:1788880863.870717:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000948807ec. 02000000:00000001:0.0:1788880863.870720:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880863.870722:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:0.0:1788880863.870724:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dd8b4e58. 00000400:00000200:2.0:1788880863.870728:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [486:486:256:4294967295] 192.168.204.111@tcp LPNI seq info [486:486:8:4294967295] 00000400:00000010:0.0:1788880863.870730:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000100:00000200:0.0:1788880863.870735:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780692480, portal 10 00000400:00000200:2.0:1788880863.870740:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880863.870745:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880863.870749:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849372160 : -126822860179456 : ffff8ca7c0783800) 00000800:00000200:2.0:1788880863.870749:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880863.870750:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880863.870756:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:2.0:1788880863.870757:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5d1e8ed. 00000100:00000040:0.0:1788880863.870760:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880863.870761:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880863.870767:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1788880863.870771:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880863.870773:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880863.870776:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880863.870782:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880863.870786:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [487:487:255:4294967295] 192.168.204.111@tcp LPNI seq info [487:487:7:4294967295] 00000400:00000010:0.0:1788880863.870787:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000800:00000200:2.0:1788880863.870790:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:0.0:1788880863.870792:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780692480, offset 0 00000800:00000200:2.0:1788880863.870797:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5d1e8ed type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880863.870803:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880863.870804:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880863.870815:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880863.870817:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880863.870818:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000252f4482. 00000400:00000200:0.0:1788880863.870819:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880863.870826:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:1.0:1788880863.870826:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880863.870826:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306500047 00000100:00000010:2.0:1788880863.870834:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fbf478f2. 00000800:00000200:1.0:1788880863.870835:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880863.870839:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:1.0:1788880863.870839:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000252f4482 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880863.870840:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1788880863.870845:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880863.870847:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306500047 00000400:00000200:0.0:1788880863.870848:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:1.0:1788880863.870851:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 182256 usecs (183 jiffies) 00000100:00000001:1.0:1788880863.870864:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306495159 00000100:00000010:2.0:1788880863.870865:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006738d3fe. 00000400:00000200:0.0:1788880863.870869:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880863.870870:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306500047 00000100:00000001:2.0:1788880863.870871:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880863.870876:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [488:488:254:4294967295] 192.168.204.111@tcp LPNI seq info [488:488:6:4294967295] 00000100:00000001:2.0:1788880863.870878:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306500047 00000100:00000001:1.0:1788880863.870882:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306495159 00000100:00000010:2.0:1788880863.870884:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ad5555dd. 00000100:00000001:1.0:1788880863.870887:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306500047 00000100:00000001:2.0:1788880863.870889:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880863.870893:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306495159 00000100:00000001:2.0:1788880863.870896:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306500047 00000400:00000200:0.0:1788880863.870899:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880863.870899:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306500047 00000100:00000010:2.0:1788880863.870901:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fa7e66e6. 00000100:00000001:1.0:1788880863.870903:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306495159 00000100:00000001:2.0:1788880863.870908:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:3.0:1788880863.870910:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:0.0:1788880863.870912:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880863.870914:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306500047 00000100:00000001:2.0:1788880863.870914:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:3.0:1788880863.870916:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5d1e8ed. 00000800:00000010:0.0:1788880863.870919:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000fc8f9b7c. 00000100:00000001:1.0:1788880863.870919:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880863.870920:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880863.870924:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880863.870926:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880863.870928:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880863.870929:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880863.870931:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880863.870932:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788880863.870937:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880863.870938:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880863.870939:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:0.0:1788880863.870941:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000fc8f9b7c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880863.870942:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880863.870942:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880863.870947:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880863.870949:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880863.870950:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880863.870950:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880863.870951:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880863.870953:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880863.870955:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880863.870956:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13105 usecs (14 jiffies) 00000100:00000001:1.0:1788880863.870956:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880863.870961:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880863.870963:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306494990 00000400:00000200:3.0:1788880863.870966:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880863.870967:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306500047 00000100:00000001:0.0:1788880863.870973:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306494990 00000400:00000200:3.0:1788880863.870974:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880863.870979:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306500047 00000400:00000200:3.0:1788880863.870980:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c0c07443 00000100:00000001:0.0:1788880863.870983:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306494990 00000400:00000010:3.0:1788880863.870983:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c0c07443. 00000100:00000001:0.0:1788880863.870988:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306500047 00000100:00000001:3.0:1788880863.870989:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880863.870993:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306494990 00000100:00000200:3.0:1788880863.870995:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880879 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880863.870998:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306500047 00000100:00000001:0.0:1788880863.871003:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880863.871009:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880863.871010:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880863.871012:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880863.871016:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880863.871022:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880879 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880863.871030:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880863.871032:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880863.871044:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880863.871046:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000252f4482. 00000400:00000200:3.0:1788880863.871049:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880863.871053:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880863.871055:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000038c5ac58 00000400:00000010:3.0:1788880863.871056:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000038c5ac58. 00000100:00000001:3.0:1788880863.871059:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880863.871060:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880863.871065:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880863.871066:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880863.871070:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880863.871071:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880863.871092:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880863.871093:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000fc8f9b7c. 00000400:00000200:3.0:1788880863.871095:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880863.871097:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880863.871098:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000400:00000010:3.0:1788880863.871099:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00000100:00000001:3.0:1788880863.871100:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880863.871106:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880863.871110:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880863.871111:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880863.871115:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880863.871116:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880863.872454:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880863.872461:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880863.872463:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880863.872464:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880863.872469:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880863.872477:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0eb00 00000400:00000200:2.0:1788880863.872484:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0xff9 [1] + 224 00000800:00000001:2.0:1788880863.872495:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880863.872500:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880863.872502:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880863.872506:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880863.872510:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880863.872514:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880863.872524:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880863.872540:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880863.872547:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880863.872552:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880863.872553:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880863.872554:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880863.872555:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880863.872559:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880863.872560:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880863.872563:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880863.872565:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000002869d45. 00000400:00000200:2.0:1788880863.872568:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880863.872569:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009543d7d4 00000400:00000200:2.0:1788880863.872578:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0eb80 00000400:00000010:3.0:1788880863.872600:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009543d7d4. 00000100:00000001:3.0:1788880863.872603:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880863.872608:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880863.872609:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0xffd [1] + 224 00000800:00000001:2.0:1788880863.872620:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880863.872628:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880863.872630:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880863.872634:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880863.872637:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880863.872640:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880863.872649:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880863.872659:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880863.872663:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880863.872668:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880863.872668:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880863.872672:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880863.872675:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880863.872678:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880863.872680:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880863.872682:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880863.872685:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880863.872685:0:201285:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:2.0:1788880863.872688:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 02000000:00000001:3.0:1788880863.872689:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880863.872692:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880863.872694:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880863.872696:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:3.0:1788880863.872697:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880863.872700:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788880863.872701:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ec00 00000400:00000010:1.0:1788880863.872703:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000100:00001000:3.0:1788880863.872704:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=91 v=5 (5 5 5 5) 00000400:00000200:1.0:1788880863.872708:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000100:00001000:3.0:1788880863.872709:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=91 v=5 (1 1 1 1) 00000400:00000010:1.0:1788880863.872711:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:3.0:1788880863.872713:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880863.872715:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880863.872715:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880863.872719:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1009 [1] + 224 00000100:00000200:1.0:1788880863.872721:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880863.872731:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788880863.872734:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880863.872739:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880863.872739:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880863.872742:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880863.872743:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:2.0:1788880863.872744:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880863.872745:0:201286:0:(client.c:1580:after_reply()) Process entered 00010000:00000001:3.0:1788880863.872747:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880863.872748:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880863.872750:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880863.872753:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880863.872754:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880863.872754:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880863.872755:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880863.872762:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=81 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880863.872763:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1788880863.872765:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880863.872767:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=81 v=5 (1 1 1 1) 00000100:00000200:2.0:1788880863.872769:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880863.872774:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880863.872777:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880863.872781:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880863.872785:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880863.872789:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880863.872791:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880863.872795:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000040:2.0:1788880863.872796:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880863.872797:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880863.872806:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880863.872826:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880863.872831:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880863.872831:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880863.872834:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880863.872836:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880863.872840:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880863.872841:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880863.872847:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880863.872849:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:3.0:1788880863.872849:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880863.872868:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1788880863.872885:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780692224:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880863.872886:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880863.872890:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880863.872896:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880863.872896:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788880863.872900:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880863.872900:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000010:0.0:1788880863.872904:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dd8b4e58. 00000100:00000040:3.0:1788880863.872906:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000363de9c9 x1875776780692224/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880863.872909:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780692352:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880863.872917:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000100:00000001:3.0:1788880863.872917:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880863.872919:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880863.872920:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000010:0.0:1788880863.872921:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:1.0:1788880863.872923:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000010:3.0:1788880863.872924:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000027d4cece. 00000100:00000001:0.0:1788880863.872926:0:201288:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1788880863.872929:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:1.0:1788880863.872931:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c2a263b1 x1875776780692352/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880863.872932:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000200:0.0:1788880863.872933:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:3.0:1788880863.872935:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788880863.872940:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880863.872943:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000044c69dd2. 02000000:00000001:3.0:1788880863.872948:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880863.872950:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880863.872953:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000363de9c9. 00000100:00000001:3.0:1788880863.872958:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880863.872959:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880863.872964:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880863.872966:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:0.0:1788880863.872968:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880863.872969:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880863.872970:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880863.872973:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880863.872974:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000001363d535. 00000100:00000001:3.0:1788880863.872977:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880863.872980:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880863.872982:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880863.872985:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880863.872986:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000040:1.0:1788880863.872988:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880863.872990:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880863.872993:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880863.872995:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000010:1.0:1788880863.872996:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000013bf63f4. 02000000:00000001:0.0:1788880863.872998:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880863.873001:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:1.0:1788880863.873001:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880863.873005:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880863.873005:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880863.873008:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c2a263b1. 02000000:00000001:0.0:1788880863.873009:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880863.873013:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880863.873015:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880863.873016:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=91 v=5 (5 5 5 5) 00000100:00000001:1.0:1788880863.873018:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880863.873021:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=25 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880863.873024:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880863.873025:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880863.873028:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880863.873029:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880863.873033:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:0.0:1788880863.873034:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880863.873036:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:0.0:1788880863.873040:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880863.873048:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880863.873051:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880863.873052:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880863.873053:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880863.873055:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880863.873057:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880863.873064:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880863.873065:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880863.873067:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880863.873069:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880863.873086:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cdd8dec9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780692480:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880863.873090:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880863.873092:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880863.873094:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cdd8dec9 x1875776780692480/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880879 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880863.873100:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880863.873101:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880863.873103:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000948807ec. 02000000:00000001:0.0:1788880863.873105:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880863.873107:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880863.873107:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880863.873109:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880863.873110:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003806e31f. 02000000:00000001:0.0:1788880863.873111:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880863.873112:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880863.873113:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cdd8dec9. 00000100:00000001:0.0:1788880863.873115:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880863.873115:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880863.873116:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880863.873119:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880863.873120:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880863.873121:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880863.873122:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880863.934310:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880863.934317:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880863.934325:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880863.934326:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880863.934371:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880863.934375:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880863.934376:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880864.958363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880864.958371:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880864.958377:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880864.958378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880864.958451:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880864.958455:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880864.958467:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880865.982345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880865.982356:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880865.982358:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880865.982375:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880865.982379:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880865.982380:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880865.982382:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880865.982383:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880865.982386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880865.982387:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880865.982427:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880865.982430:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880865.982432:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880867.006309:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880867.006314:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880867.006319:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880867.006320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880867.006389:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880867.006395:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880867.006417:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880868.030499:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880868.030506:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880868.030512:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880868.030515:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880868.030517:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880868.030520:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880868.030523:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880868.030525:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880868.030527:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880868.030529:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880868.030619:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880868.030623:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880868.030630:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880868.990350:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306500047, at 4306500096 00000100:00000040:1.0:1788880868.990350:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306500047, at 4306500096 00000100:00000040:2.0:1788880868.990350:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306500047, at 4306500096 00000100:00000040:3.0:1788880868.990350:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306500047, at 4306500096 00000100:00000010:1.0:1788880868.990370:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006738d3fe. 00000100:00000010:3.0:1788880868.990372:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fa7e66e6. 00000100:00000010:0.0:1788880868.990374:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fbf478f2. 00000100:00000010:2.0:1788880868.990375:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ad5555dd. 00000100:00000040:3.0:1788880868.990415:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880868.990420:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880868.990430:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880868.990434:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880868.990439:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880868.990442:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880868.990447:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880868.990454:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:3.0:1788880868.990461:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880868.990471:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880868.990479:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880868.990481:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880868.990486:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_03+3] req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880868.990506:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880868.990508:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880868.990511:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880868.990515:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000020:00000040:3.0:1788880868.990517:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880868.990518:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880868.990520:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880868.990521:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880868.990523:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880868.990524:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:2.0:1788880868.990527:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880868.990528:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880868.990530:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880868.990531:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880868.990533:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880868.990535:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880868.990540:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:0.0:1788880868.990543:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880868.990552:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880868.990554:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880868.990557:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880868.990561:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_00+0] req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880868.990567:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880868.990571:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880868.990575:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880868.990578:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880868.990579:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780692608:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880868.990582:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880868.990588:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880868.990588:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880868.990591:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000100:00000001:2.0:1788880868.990593:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:3.0:1788880868.990595:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880868.990596:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880868.990597:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880868.990597:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880868.990600:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880868.990601:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880868.990601:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880868.990603:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880868.990604:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880868.990607:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880868.990607:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880868.990608:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000040:2.0:1788880868.990611:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880868.990613:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880868.990614:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f77f54b9. 02000000:00000001:0.0:1788880868.990619:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880868.990620:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:0.0:1788880868.990631:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dd8b4e58. 00000100:00000001:3.0:1788880868.990635:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880868.990637:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880868.990641:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_01+1] req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880868.990642:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000100:00000200:0.0:1788880868.990652:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780692608, portal 25 00000100:00000001:0.0:1788880868.990658:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880868.990658:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880868.990662:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000001:2.0:1788880868.990662:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880868.990668:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880868.990671:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880868.990672:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880884 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880868.990672:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880868.990676:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880868.990678:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880868.990686:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780692736:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880868.990687:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788880868.990695:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880868.990695:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880868.990697:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11837) 02000000:00000001:2.0:1788880868.990699:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880868.990700:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880868.990703:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880868.990703:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880868.990704:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880868.990706:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880868.990708:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880868.990712:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880868.990714:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008ea7dcae. 00000400:00000010:0.0:1788880868.990715:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000115221d8. 02000000:00000001:2.0:1788880868.990720:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880868.990721:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780692608, offset 0 00000100:00000001:1.0:1788880868.990726:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:2.0:1788880868.990727:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000deb55c04. 00000100:00000001:1.0:1788880868.990729:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880868.990733:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880868.990733:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880868.990737:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880868.990740:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880868.990740:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000044b1d7fc. 00000100:00100000:1.0:1788880868.990744:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780692864:192.168.204.111@tcp:400: 00000100:00000200:2.0:1788880868.990745:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780692736, portal 10 00000100:00000001:2.0:1788880868.990748:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880868.990750:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880868.990750:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000100:00000001:1.0:1788880868.990768:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880868.990769:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880868.990770:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880868.990773:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880868.990776:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788880868.990777:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880868.990781:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880868.990782:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880868.990784:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1788880868.990787:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:1.0:1788880868.990788:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000633adef2. 00000400:00000200:0.0:1788880868.990790:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:2.0:1788880868.990791:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e1dae22. 02000000:00000001:1.0:1788880868.990794:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880868.990796:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780692736, offset 0 00000400:00000200:0.0:1788880868.990798:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [489:489:256:4294967295] 192.168.204.111@tcp LPNI seq info [489:489:8:4294967295] 00000400:00000010:1.0:1788880868.990800:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000400:00000200:2.0:1788880868.990805:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880868.990813:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000100:00000200:1.0:1788880868.990817:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780692864, portal 10 00000100:00000001:1.0:1788880868.990819:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880868.990819:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000400:00000200:0.0:1788880868.990825:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:1.0:1788880868.990832:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880868.990836:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880868.990837:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788880868.990846:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003806e31f. 00000400:00000200:2.0:1788880868.990846:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880868.990847:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880868.990852:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880868.990854:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:1.0:1788880868.990856:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000974f52f4. 00000800:00000200:0.0:1788880868.990859:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:1.0:1788880868.990861:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780692864, offset 0 00000400:00000200:2.0:1788880868.990861:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880868.990868:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880868.990873:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880868.990876:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [490:490:255:4294967295] 192.168.204.111@tcp LPNI seq info [490:490:7:4294967295] 00000800:00000200:0.0:1788880868.990881:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880868.990888:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003806e31f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880868.990891:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880868.990902:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880868.990906:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880868.990907:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880868.990908:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306505167 00000800:00000010:2.0:1788880868.990912:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008eccda10. 00000100:00000010:0.0:1788880868.990914:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005b8bebe2. 00000400:00000200:1.0:1788880868.990920:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880868.990921:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:2.0:1788880868.990924:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880868.990927:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306505167 00000400:00000200:1.0:1788880868.990928:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:0.0:1788880868.990932:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009ac8d70b. 00000400:00000200:1.0:1788880868.990936:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1788880868.990936:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880868.990937:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:2.0:1788880868.990942:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008eccda10 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880868.990943:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306505167 00000400:00000200:1.0:1788880868.990943:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880868.990948:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880868.990949:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006050354c. 00000400:00000200:1.0:1788880868.990952:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [491:491:254:4294967295] 192.168.204.111@tcp LPNI seq info [491:491:6:4294967295] 00000100:00000001:0.0:1788880868.990954:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000040:2.0:1788880868.990955:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 181002 usecs (182 jiffies) 00000100:00000001:2.0:1788880868.990959:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306500278 00000100:00000001:0.0:1788880868.990960:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306505167 00000100:00000001:2.0:1788880868.990963:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306505167 00000100:00000010:0.0:1788880868.990966:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000da3f50d1. 00000100:00000001:2.0:1788880868.990968:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306500278 00000100:00000001:0.0:1788880868.990969:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788880868.990972:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880868.990973:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306505167 00000100:00000001:0.0:1788880868.990975:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1788880868.990976:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880868.990979:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306500278 00000100:00000001:0.0:1788880868.990981:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880868.990984:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306505167 00000100:00000001:0.0:1788880868.990986:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880868.990987:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880868.990987:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306500278 00000100:00000001:0.0:1788880868.990990:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880868.990992:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306505167 00000800:00000010:3.0:1788880868.990993:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003806e31f. 00000100:00000001:2.0:1788880868.990994:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:1.0:1788880868.990995:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000100:00000001:2.0:1788880868.990998:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880868.990999:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:3.0:1788880868.991002:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880868.991003:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880868.991003:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880868.991005:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880868.991008:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880868.991009:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880868.991012:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880868.991014:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880868.991015:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:1.0:1788880868.991017:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880868.991017:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880868.991018:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880868.991019:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880868.991024:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880868.991025:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880868.991027:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880868.991028:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000115221d8 00000100:00000001:2.0:1788880868.991029:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880868.991032:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880868.991034:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880868.991034:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000115221d8. 00000100:00000040:1.0:1788880868.991043:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 13033 usecs (14 jiffies) 00000100:00000001:3.0:1788880868.991044:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880868.991047:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306500110 00000100:00000001:1.0:1788880868.991049:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306505167 00000100:00000001:1.0:1788880868.991053:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306500110 00000100:00000001:1.0:1788880868.991057:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306505167 00000100:00000200:3.0:1788880868.991058:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880884 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880868.991063:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306500110 00000100:00000001:1.0:1788880868.991068:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306505167 00000100:00000001:1.0:1788880868.991071:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306500110 00000100:00000001:1.0:1788880868.991073:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306505167 00000100:00000001:3.0:1788880868.991073:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880868.991077:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880868.991079:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880884 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880868.991080:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880868.991085:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880868.991088:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880868.991093:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880868.991094:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880868.991095:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880868.991098:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880868.991100:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880868.991101:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880868.991102:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880868.991104:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880868.991110:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880868.991113:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008eccda10. 00000400:00000200:3.0:1788880868.991116:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880868.991118:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880868.991120:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e1dae22 00000400:00000010:3.0:1788880868.991121:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e1dae22. 00000100:00000001:3.0:1788880868.991122:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880868.991124:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880868.991129:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880868.991130:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880868.991134:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880868.991135:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880868.991198:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880868.991201:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:3.0:1788880868.991205:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880868.991208:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880868.991210:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000974f52f4 00000400:00000010:3.0:1788880868.991211:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000974f52f4. 00000100:00000001:3.0:1788880868.991214:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880868.991216:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880868.991222:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880868.991223:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880868.991251:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880868.991253:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880868.993122:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880868.993133:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880868.993136:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880868.993137:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880868.993143:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880868.993152:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ed00 00000400:00000200:3.0:1788880868.993158:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1019 [1] + 224 00000800:00000001:3.0:1788880868.993163:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880868.993164:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880868.993166:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880868.993168:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880868.993171:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880868.993174:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880868.993225:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880868.993253:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880868.993263:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880868.993265:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880868.993270:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880868.993272:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880868.993273:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000deb55c04. 00000400:00000200:2.0:1788880868.993277:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000044b1d7fc 00000400:00000010:2.0:1788880868.993284:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000044b1d7fc. 00000800:00000001:3.0:1788880868.993284:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880868.993287:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:3.0:1788880868.993289:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880868.993293:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880868.993293:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880868.993296:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880868.993305:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000200:2.0:1788880868.993308:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880868.993313:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0ec80 00000400:00000200:3.0:1788880868.993321:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1011 [1] + 224 00000100:00000001:2.0:1788880868.993322:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880868.993326:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880868.993326:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880868.993328:0:201285:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:3.0:1788880868.993330:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880868.993331:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000200:3.0:1788880868.993332:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880868.993334:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880868.993336:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880868.993337:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:2.0:1788880868.993339:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880868.993346:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:2.0:1788880868.993347:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=86 v=5 (5 5 5 5) 00000100:00000200:3.0:1788880868.993350:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:2.0:1788880868.993352:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=86 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880868.993359:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880868.993361:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880868.993365:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:3.0:1788880868.993366:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880868.993369:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880868.993374:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880868.993376:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880868.993378:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880868.993379:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880868.993381:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:2.0:1788880868.993384:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880868.993389:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880868.993391:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000800:00000001:3.0:1788880868.993392:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788880868.993393:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880868.993396:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880868.993397:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880868.993401:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880868.993404:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788880868.993404:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880868.993408:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00100000:2.0:1788880868.993408:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780692736:192.168.204.111@tcp:400: 00000400:00000001:0.0:1788880868.993413:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:3.0:1788880868.993415:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:0.0:1788880868.993417:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880868.993418:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880868.993420:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dd8b4e58. 00000400:00000200:3.0:1788880868.993423:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ed80 00000400:00000200:0.0:1788880868.993425:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000100:00000001:2.0:1788880868.993426:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000010:0.0:1788880868.993429:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000040:2.0:1788880868.993432:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780692736/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880868.993434:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:3.0:1788880868.993438:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1021 [1] + 224 00000100:00000001:2.0:1788880868.993444:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000800:00000001:3.0:1788880868.993445:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880868.993446:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880868.993448:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880868.993451:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008ea7dcae. 00000800:00000001:3.0:1788880868.993451:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880868.993456:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:2.0:1788880868.993457:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880868.993461:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000200:0.0:1788880868.993464:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:2.0:1788880868.993465:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000200:3.0:1788880868.993465:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000020:00000001:2.0:1788880868.993471:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880868.993472:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880868.993477:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880868.993481:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 00000100:00000001:0.0:1788880868.993485:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880868.993485:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880868.993486:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880868.993488:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880868.993489:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880868.993492:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:0.0:1788880868.993493:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880868.993497:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:0.0:1788880868.993498:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880868.993500:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880868.993501:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880868.993505:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880868.993505:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880868.993509:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880868.993511:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00001000:0.0:1788880868.993512:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=96 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880868.993514:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880868.993515:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:0.0:1788880868.993520:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=96 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880868.993520:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880868.993523:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880868.993524:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880868.993526:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880868.993529:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880868.993530:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880868.993532:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880868.993536:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880868.993538:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880868.993543:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880868.993547:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880868.993549:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880868.993550:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880868.993552:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880868.993553:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880868.993553:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880868.993554:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880868.993556:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000100:00000040:0.0:1788880868.993557:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880868.993559:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000400:00000010:1.0:1788880868.993561:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:1.0:1788880868.993564:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:0.0:1788880868.993565:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780692608:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880868.993566:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880868.993569:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880868.993571:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880868.993573:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780692608/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880868.993574:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880868.993580:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880868.993581:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788880868.993581:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:0.0:1788880868.993588:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f77f54b9. 00000100:00000001:1.0:1788880868.993588:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880868.993590:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880868.993590:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880868.993591:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:0.0:1788880868.993592:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880868.993592:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000040:0.0:1788880868.993593:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880868.993593:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880868.993595:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:1.0:1788880868.993595:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880868.993596:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:0.0:1788880868.993599:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00001000:1.0:1788880868.993599:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=96 v=5 (5 5 5 5) 02000000:00000001:0.0:1788880868.993600:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880868.993601:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00001000:1.0:1788880868.993601:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=30 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880868.993603:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880868.993603:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880868.993605:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880868.993605:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880868.993606:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880868.993606:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880868.993610:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00080000:1.0:1788880868.993610:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880868.993611:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880868.993613:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880868.993615:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880868.993617:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880868.993619:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880868.993620:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880868.993622:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880868.993623:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880868.993625:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880868.993631:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880868.993633:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880868.993634:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880868.993636:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880868.993643:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780692864:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880868.993647:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880868.993649:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880868.993651:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780692864/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880884 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880868.993657:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880868.993658:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880868.993663:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000633adef2. 02000000:00000001:1.0:1788880868.993666:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880868.993667:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880868.993667:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880868.993669:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880868.993669:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:1.0:1788880868.993671:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880868.993672:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880868.993672:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:1.0:1788880868.993674:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880868.993675:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880868.993676:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880868.993679:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880868.993679:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880868.993681:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880868.993682:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880869.054310:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880869.054317:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880869.054323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880869.054324:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880869.054366:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880869.054370:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880869.054373:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880870.078365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880870.078374:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880870.078381:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880870.078382:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880870.078443:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880870.078448:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880870.078451:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880871.102332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880871.102343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880871.102355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880871.102357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880871.102394:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880871.102400:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880871.102403:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880872.126341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880872.126349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880872.126354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880872.126356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880872.126427:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880872.126431:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880872.126434:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880873.150353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880873.150360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880873.150367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880873.150368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880873.150432:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880873.150458:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880873.150461:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880874.110334:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306505167, at 4306505216 00000100:00000040:2.0:1788880874.110334:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306505167, at 4306505216 00000100:00000040:3.0:1788880874.110341:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880874.110344:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306505167, at 4306505216 00000100:00000001:3.0:1788880874.110351:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880874.110353:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006050354c. 00000100:00000010:0.0:1788880874.110355:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005b8bebe2. 00000100:00000010:1.0:1788880874.110356:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009ac8d70b. 00000100:00000010:3.0:1788880874.110361:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000020:00000040:3.0:1788880874.110367:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880874.110372:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880874.110376:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880874.110382:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880874.110390:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:3.0:1788880874.110398:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880874.110409:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880874.110432:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880874.110434:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880874.110441:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_02+2] req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880874.110464:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880874.110466:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880874.110470:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880874.110472:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000020:00000040:3.0:1788880874.110475:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880874.110478:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880874.110479:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880874.110481:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880874.110483:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880874.110485:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:2.0:1788880874.110489:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880874.110490:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880874.110494:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880874.110496:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880874.110499:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880874.110503:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880874.110505:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880874.110513:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880874.110517:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880874.110519:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880874.110521:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880874.110531:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_03+3] req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880874.110535:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880874.110539:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880874.110544:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880874.110548:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780692992:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880874.110551:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880874.110555:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880874.110556:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880874.110560:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:2.0:1788880874.110561:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880874.110563:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880874.110563:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 02000000:00000001:2.0:1788880874.110568:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000020:00000040:3.0:1788880874.110569:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880874.110573:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880874.110573:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003816d5cd. 00000100:00000001:3.0:1788880874.110573:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880874.110578:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880874.110578:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880874.110579:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880874.110582:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880874.110583:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880874.110587:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000deb55c04. 02000000:00000010:3.0:1788880874.110589:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000040:0.0:1788880874.110590:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880874.110596:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880874.110597:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000044b1d7fc. 00000100:00000001:0.0:1788880874.110603:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788880874.110606:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880874.110607:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880874.110608:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780692992, portal 25 02000000:00000001:0.0:1788880874.110612:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880874.110614:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:0.0:1788880874.110616:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880874.110618:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 02000000:00000001:0.0:1788880874.110622:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880874.110629:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880890 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880874.110630:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780693120:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880874.110630:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880874.110633:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880874.110639:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880874.110642:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_00+0] req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880874.110644:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880874.110648:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880874.110649:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880874.110653:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880874.110667:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:0.0:1788880874.110669:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000826ae8f4. 00000400:00000010:2.0:1788880874.110672:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e1dae22. 00000100:00000200:2.0:1788880874.110677:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780692992, offset 0 02000000:00000001:0.0:1788880874.110678:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880874.110686:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dd8b4e58. 00000100:00000001:3.0:1788880874.110687:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880874.110692:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880874.110693:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880874.110698:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:0.0:1788880874.110700:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000100:00000040:3.0:1788880874.110704:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11842) 00000100:00000200:0.0:1788880874.110706:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780693120, portal 10 00000400:00000200:2.0:1788880874.110709:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880874.110710:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880874.110714:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000001:1.0:1788880874.110714:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880874.110718:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880874.110720:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880874.110720:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306505167, at 4306505216 00000100:00000001:1.0:1788880874.110723:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880874.110727:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880874.110727:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:3.0:1788880874.110728:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000da3f50d1. 00000100:00000040:1.0:1788880874.110733:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880874.110736:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880874.110745:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880874.110751:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880874.110753:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880874.110756:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [492:492:256:4294967295] 192.168.204.111@tcp LPNI seq info [492:492:8:4294967295] 00000100:00000001:1.0:1788880874.110757:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:0.0:1788880874.110758:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880874.110759:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880874.110761:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880874.110763:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a09430ba. 00000100:00000200:0.0:1788880874.110768:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780693120, offset 0 02000000:00000001:1.0:1788880874.110768:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880874.110772:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880874.110778:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880874.110779:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880874.110783:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00100000:1.0:1788880874.110787:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780693248:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880874.110793:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:2.0:1788880874.110794:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880874.110798:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880874.110801:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880874.110801:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 02000000:00000001:1.0:1788880874.110802:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880874.110808:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880874.110809:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:1.0:1788880874.110813:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000026532f47. 00000400:00000200:0.0:1788880874.110814:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880874.110815:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:1.0:1788880874.110819:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880874.110820:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880874.110825:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:1.0:1788880874.110826:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000400:00000200:0.0:1788880874.110832:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [493:493:255:4294967295] 192.168.204.111@tcp LPNI seq info [493:493:7:4294967295] 00000400:00000010:1.0:1788880874.110835:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000800:00000200:2.0:1788880874.110836:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:1.0:1788880874.110842:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780693248, portal 10 00000800:00000200:2.0:1788880874.110843:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880874.110849:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880874.110853:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880874.110853:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000040:1.0:1788880874.110863:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880874.110865:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880874.110865:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880874.110873:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e494c472. 00000100:00000001:2.0:1788880874.110874:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306510287 00000100:00000001:1.0:1788880874.110879:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:2.0:1788880874.110882:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ad5555dd. 00000800:00000200:0.0:1788880874.110883:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:1.0:1788880874.110889:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880874.110891:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:0.0:1788880874.110894:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:1.0:1788880874.110894:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000100:00000001:2.0:1788880874.110899:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306510287 00000800:00000200:0.0:1788880874.110900:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e494c472 type 1, nob 320 niov 1 nkiov 1 00000100:00000200:1.0:1788880874.110900:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780693248, offset 0 00000100:00000010:2.0:1788880874.110906:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fbf478f2. 00000100:00000001:0.0:1788880874.110908:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880874.110912:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:0.0:1788880874.110913:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 179768 usecs (180 jiffies) 00000400:00000200:1.0:1788880874.110913:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880874.110919:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306510287 00000100:00000001:0.0:1788880874.110920:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306505396 00000400:00000200:1.0:1788880874.110925:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880874.110926:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306510287 00000100:00000010:2.0:1788880874.110926:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fa7e66e6. 00000100:00000001:0.0:1788880874.110932:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306505396 00000400:00000200:1.0:1788880874.110933:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880874.110934:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:3.0:1788880874.110935:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:1.0:1788880874.110941:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880874.110942:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306510287 00000800:00000010:3.0:1788880874.110942:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000100:00000001:2.0:1788880874.110943:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306510287 00000100:00000001:0.0:1788880874.110949:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306505396 00000400:00000200:3.0:1788880874.110950:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000010:2.0:1788880874.110951:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006738d3fe. 00000100:00000001:0.0:1788880874.110953:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306510287 00000100:00000001:2.0:1788880874.110957:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880874.110958:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306505396 00000400:00000200:1.0:1788880874.110961:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880874.110964:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880874.110964:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880874.110965:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306510287 00000100:00000001:2.0:1788880874.110969:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880874.110971:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880874.110977:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880874.110977:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880874.110981:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880874.110984:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880874.110985:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [494:494:254:4294967295] 192.168.204.111@tcp LPNI seq info [494:494:6:4294967295] 00000100:00000001:2.0:1788880874.110989:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880874.110990:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880874.110992:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880874.110995:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880874.110995:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880874.111000:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880874.111003:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880874.111003:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880874.111005:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880874.111008:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880874.111013:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880874.111015:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e1dae22 00000100:00000001:0.0:1788880874.111016:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880874.111016:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880874.111017:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880874.111021:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880874.111021:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e1dae22. 00000800:00000010:1.0:1788880874.111024:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5b428a0. 00000100:00000001:0.0:1788880874.111028:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880874.111030:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:1.0:1788880874.111033:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:3.0:1788880874.111038:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880890 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880874.111043:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880874.111049:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5b428a0 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880874.111055:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880874.111055:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880874.111060:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880890 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880874.111061:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12967 usecs (13 jiffies) 00000100:00000001:1.0:1788880874.111066:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306505229 00000100:00000001:1.0:1788880874.111071:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306510287 00000100:00000001:1.0:1788880874.111077:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306505229 00000100:00000001:3.0:1788880874.111078:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880874.111083:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306510287 00000100:00000001:3.0:1788880874.111083:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880874.111087:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306505229 00000100:00000001:1.0:1788880874.111089:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306510287 00000100:00000001:1.0:1788880874.111091:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306505229 00000100:00000001:1.0:1788880874.111094:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306510287 00000100:00000001:1.0:1788880874.111095:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880874.111098:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880874.111098:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880874.111101:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880874.111102:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e494c472. 00000100:00000001:1.0:1788880874.111106:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880874.111108:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880874.111112:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:3.0:1788880874.111114:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880874.111115:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:3.0:1788880874.111117:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a09430ba 00000100:00000001:1.0:1788880874.111119:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880874.111121:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a09430ba. 00000100:00000001:1.0:1788880874.111123:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880874.111125:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880874.111127:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880874.111129:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880874.111132:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880874.111140:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880874.111142:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880874.111148:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880874.111150:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880874.111204:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880874.111206:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5b428a0. 00000400:00000200:3.0:1788880874.111209:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880874.111212:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880874.111215:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000400:00000010:3.0:1788880874.111216:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:3.0:1788880874.111218:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880874.111221:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880874.111267:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880874.111270:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880874.111276:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880874.111277:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880874.113369:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880874.113382:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880874.113385:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880874.113387:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880874.113395:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880874.113407:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ee80 00000400:00000200:3.0:1788880874.113417:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1031 [1] + 224 00000800:00000001:3.0:1788880874.113466:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880874.113469:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880874.113472:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880874.113476:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880874.113480:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880874.113485:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880874.113496:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880874.113513:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880874.113520:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880874.113523:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880874.113527:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880874.113529:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880874.113533:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880874.113545:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ef00 00000400:00000200:3.0:1788880874.113551:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1039 [1] + 224 00000800:00000001:3.0:1788880874.113555:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880874.113566:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880874.113567:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880874.113569:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880874.113578:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880874.113579:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880874.113589:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:0.0:1788880874.113592:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:3.0:1788880874.113597:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880874.113599:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880874.113604:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dd8b4e58. 00000400:00000200:0.0:1788880874.113616:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000100:00000040:3.0:1788880874.113616:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880874.113621:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:0.0:1788880874.113629:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880874.113640:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880874.113644:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880874.113651:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880874.113669:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880874.113670:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880874.113678:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880874.113684:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880874.113701:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880874.113703:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:3.0:1788880874.113703:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880874.113708:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880874.113713:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880874.113715:0:201288:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:3.0:1788880874.113720:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0ee00 00000400:00000001:1.0:1788880874.113722:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:0.0:1788880874.113723:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000001:1.0:1788880874.113726:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880874.113729:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000010:1.0:1788880874.113732:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 00000100:00000001:0.0:1788880874.113734:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880874.113740:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000400:00000010:1.0:1788880874.113743:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:1.0:1788880874.113746:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:3.0:1788880874.113749:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1029 [1] + 224 02000000:00000001:0.0:1788880874.113757:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880874.113760:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788880874.113762:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880874.113771:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880874.113772:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=92 v=5 (5 5 5 5) 00000100:00000200:1.0:1788880874.113780:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880874.113780:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00001000:0.0:1788880874.113783:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=92 v=5 (1 1 1 1) 00000400:00000200:3.0:1788880874.113794:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880874.113795:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880874.113800:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880874.113807:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:0.0:1788880874.113808:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880874.113811:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:0.0:1788880874.113815:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880874.113816:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880874.113816:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880874.113822:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880874.113830:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880874.113831:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:1.0:1788880874.113834:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:0.0:1788880874.113836:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000040:3.0:1788880874.113837:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880874.113839:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880874.113843:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880874.113848:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880874.113848:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880874.113871:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880874.113871:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=102 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880874.113873:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880874.113885:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:1.0:1788880874.113893:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=36 v=5 (1 0 0 0) 00000800:00000001:3.0:1788880874.113895:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880874.113905:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880874.113915:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880874.113915:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880874.113916:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880874.113922:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788880874.113924:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880874.113927:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788880874.113929:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00010000:00000001:1.0:1788880874.113929:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000010:2.0:1788880874.113930:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000deb55c04. 00000100:00000001:0.0:1788880874.113936:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880874.113939:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000044b1d7fc 00010000:00080000:1.0:1788880874.113944:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880874.113944:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000044b1d7fc. 00000100:00000040:0.0:1788880874.113949:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880874.113952:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880874.113973:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880874.113974:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880874.113981:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880874.113985:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880874.113992:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880874.113998:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880874.114012:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880874.114021:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780693120:192.168.204.111@tcp:400: 00000100:00000200:2.0:1788880874.114031:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880874.114040:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880874.114050:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880874.114054:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880874.114061:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780693120/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880874.114061:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880874.114062:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880874.114066:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880874.114071:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880874.114078:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:1.0:1788880874.114084:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880874.114085:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880874.114087:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880874.114089:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:0.0:1788880874.114093:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880874.114094:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880874.114098:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000826ae8f4. 02000000:00000001:2.0:1788880874.114101:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880874.114107:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:2.0:1788880874.114109:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=102 v=5 (5 5 5 5) 00000100:00100000:1.0:1788880874.114111:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780693248:192.168.204.111@tcp:400: 00000020:00000001:0.0:1788880874.114112:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880874.114120:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00001000:2.0:1788880874.114121:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=102 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880874.114123:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:0.0:1788880874.114128:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880874.114128:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880874.114130:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880874.114134:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880874.114136:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 00010000:00000001:2.0:1788880874.114138:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:1.0:1788880874.114139:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780693248/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880874.114143:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880874.114145:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880874.114147:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880874.114153:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:2.0:1788880874.114155:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880874.114159:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:1.0:1788880874.114161:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000010:0.0:1788880874.114164:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 02000000:00000010:1.0:1788880874.114166:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000026532f47. 00000100:00000001:0.0:1788880874.114173:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:1.0:1788880874.114175:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880874.114175:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880874.114177:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880874.114180:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880874.114185:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880874.114187:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880874.114188:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000020:00000001:1.0:1788880874.114190:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880874.114193:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880874.114199:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 00000100:00000040:2.0:1788880874.114203:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880874.114205:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:1.0:1788880874.114205:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880874.114210:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880874.114210:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880874.114219:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:0.0:1788880874.114220:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880874.114226:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880874.114227:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00100000:2.0:1788880874.114273:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780692992:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880874.114281:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880874.114283:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880874.114285:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880874.114288:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880874.114292:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780692992/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880890 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880874.114296:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880874.114299:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880874.114303:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880874.114305:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880874.114307:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880874.114308:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880874.114319:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003816d5cd. 02000000:00000001:2.0:1788880874.114323:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880874.114324:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880874.114326:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880874.114328:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880874.114329:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 02000000:00000001:2.0:1788880874.114332:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880874.114333:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880874.114334:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000001:2.0:1788880874.114337:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880874.114338:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880874.114339:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880874.114343:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880874.114344:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880874.114346:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880874.114348:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880874.174364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880874.174369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880874.174374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880874.174374:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880874.174431:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880874.174434:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880874.174436:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880875.198278:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880875.198287:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880875.198289:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880875.198292:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880875.198297:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880875.198311:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880875.198313:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880875.198316:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880875.198321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880875.198323:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880875.198364:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880875.198368:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880875.198370:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880876.222309:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880876.222314:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880876.222319:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880876.222320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880876.222362:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880876.222365:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880876.222367:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880877.246373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880877.246380:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880877.246387:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880877.246391:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880877.246393:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880877.246397:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880877.246400:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880877.246402:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880877.246404:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880877.246406:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880877.246435:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880877.246439:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880877.246441:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880878.270356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880878.270363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880878.270369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880878.270369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880878.270437:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880878.270443:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880878.270447:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880879.230353:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306510287, at 4306510336 00000100:00000040:2.0:1788880879.230353:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306510287, at 4306510336 00000100:00000010:0.0:1788880879.230370:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ad5555dd. 00000100:00000010:2.0:1788880879.230371:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fa7e66e6. 00000100:00000040:1.0:1788880879.230420:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306510287, at 4306510336 00000100:00000010:1.0:1788880879.230426:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fbf478f2. 00000100:00000040:3.0:1788880879.230484:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306510287, at 4306510336 00000100:00000040:1.0:1788880879.230488:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880879.230492:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880879.230498:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000020:00000040:1.0:1788880879.230501:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880879.230506:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000010:3.0:1788880879.230507:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006738d3fe. 02000000:00000001:1.0:1788880879.230508:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880879.230513:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880879.230520:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:1.0:1788880879.230525:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880879.230533:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880879.230552:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880879.230554:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880879.230570:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000077a1dd13] to pc [ptlrpcd_00_01+1] req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880879.230595:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880879.230598:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880879.230602:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880879.230605:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b5bd1bf6. 00000020:00000040:1.0:1788880879.230608:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880879.230611:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880879.230613:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880879.230614:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880879.230617:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:3.0:1788880879.230618:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880879.230619:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880879.230624:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880879.230631:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880879.230631:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880879.230633:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880879.230636:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b5bd1bf6] to pc [ptlrpcd_00_02+2] req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880879.230638:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880879.230638:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880879.230644:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880879.230648:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880879.230656:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880879.230658:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880879.230659:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880879.230663:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880879.230663:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880879.230666:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000003b122924. 00000020:00000040:1.0:1788880879.230670:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880879.230670:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880879.230671:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880879.230672:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880879.230674:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880879.230676:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880879.230676:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 02000000:00000001:3.0:1788880879.230676:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880879.230682:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880879.230683:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880879.230684:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5b428a0. 00000100:00000001:0.0:1788880879.230687:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:3.0:1788880879.230688:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780693376:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880879.230691:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880879.230692:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880879.230693:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880879.230696:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880879.230698:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880879.230700:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880879.230701:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880879.230704:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880879.230705:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880879.230707:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880879.230710:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880879.230713:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:2.0:1788880879.230715:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880879.230716:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880879.230716:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880879.230722:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003b122924] to pc [ptlrpcd_00_03+3] req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880879.230724:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000039ebda7d. 02000000:00000001:3.0:1788880879.230730:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880879.230740:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000002629a0e9. 00000100:00000001:2.0:1788880879.230742:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880879.230745:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880879.230745:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880879.230749:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880879.230749:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 02000000:00000001:2.0:1788880879.230750:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:1.0:1788880879.230755:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:2.0:1788880879.230755:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:3.0:1788880879.230758:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780693376, portal 25 00000100:00000040:1.0:1788880879.230761:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11848) 02000000:00000001:2.0:1788880879.230761:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880879.230762:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880879.230767:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00100000:2.0:1788880879.230769:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b5bd1bf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780693504:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880879.230775:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880879.230776:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880895 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880879.230780:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880879.230782:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880879.230784:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880879.230787:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 02000000:00000001:2.0:1788880879.230788:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880879.230791:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:2.0:1788880879.230793:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003816d5cd. 02000000:00000001:2.0:1788880879.230798:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880879.230799:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880879.230801:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880879.230802:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880879.230804:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000deb55c04. 00000100:00000001:0.0:1788880879.230807:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880879.230809:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880879.230811:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:2.0:1788880879.230811:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000044b1d7fc. 00000100:00000001:0.0:1788880879.230814:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:3.0:1788880879.230814:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000035d99bc2. 00000100:00000200:2.0:1788880879.230816:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780693504, portal 10 00000100:00000001:2.0:1788880879.230820:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000200:3.0:1788880879.230820:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780693376, offset 0 00000100:00000040:0.0:1788880879.230824:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880879.230824:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938059520 : -126823771492096 : ffff8ca78a26b100) 00000400:00000200:3.0:1788880879.230832:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880879.230835:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880879.230846:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880879.230849:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880879.230850:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788880879.230850:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880879.230855:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880879.230859:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880879.230859:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788880879.230860:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788880879.230864:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009543d7d4. 02000000:00000001:0.0:1788880879.230865:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880879.230868:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:2.0:1788880879.230870:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780693504, offset 0 00000100:00100000:0.0:1788880879.230872:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780693632:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880879.230879:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1788880879.230882:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1788880879.230884:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880879.230888:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880879.230890:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880879.230891:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880879.230892:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788880879.230897:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [495:495:256:4294967295] 192.168.204.111@tcp LPNI seq info [495:495:8:4294967295] 02000000:00000010:0.0:1788880879.230904:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008b528dd3. 02000000:00000001:0.0:1788880879.230908:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880879.230913:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dd8b4e58. 00000400:00000200:3.0:1788880879.230914:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:0.0:1788880879.230920:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000800:00000200:3.0:1788880879.230925:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:0.0:1788880879.230926:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780693632, portal 10 00000400:00000200:2.0:1788880879.230926:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880879.230932:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000010:3.0:1788880879.230933:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000100:00000001:0.0:1788880879.230935:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938062208 : -126823771489408 : ffff8ca78a26bb80) 00000400:00000200:2.0:1788880879.230935:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880879.230941:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880879.230945:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:0.0:1788880879.230946:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880879.230948:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880879.230954:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788880879.230959:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880879.230964:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [496:496:255:4294967295] 192.168.204.111@tcp LPNI seq info [496:496:7:4294967295] 00000100:00000001:0.0:1788880879.230966:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:3.0:1788880879.230967:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000040:0.0:1788880879.230973:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880879.230979:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d6cef011. 00000400:00000200:2.0:1788880879.230980:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:0.0:1788880879.230983:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780693632, offset 0 00000100:00000001:3.0:1788880879.230990:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880879.230991:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880879.230996:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306515407 00000800:00000200:2.0:1788880879.230997:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880879.231001:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880879.231005:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000eaa15d72. 00000100:00000010:3.0:1788880879.231005:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f093018f. 00000400:00000200:0.0:1788880879.231006:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880879.231012:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880879.231014:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880879.231016:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1788880879.231017:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880879.231022:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306515407 00000400:00000200:0.0:1788880879.231023:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788880879.231028:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880879.231031:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [497:497:254:4294967295] 192.168.204.111@tcp LPNI seq info [497:497:6:4294967295] 00000800:00000200:2.0:1788880879.231035:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000eaa15d72 type 1, nob 320 niov 1 nkiov 1 00000100:00000010:3.0:1788880879.231035:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009636b993. 00000100:00000001:2.0:1788880879.231042:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880879.231043:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880879.231046:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880879.231049:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306515407 00000100:00000040:2.0:1788880879.231050:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 178556 usecs (179 jiffies) 00000800:00000200:0.0:1788880879.231055:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880879.231056:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306510515 00000100:00000010:3.0:1788880879.231058:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000eeb663f. 00000800:00000010:0.0:1788880879.231061:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000087d24527. 00000800:00000200:1.0:1788880879.231062:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880879.231062:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306515407 00000100:00000001:3.0:1788880879.231063:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880879.231068:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306510515 00000100:00000001:3.0:1788880879.231070:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306515407 00000800:00000010:1.0:1788880879.231071:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000800:00000200:0.0:1788880879.231072:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880879.231073:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306515407 00000100:00000010:3.0:1788880879.231077:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004800b45d. 00000400:00000200:1.0:1788880879.231080:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880879.231080:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306510515 00000100:00000001:3.0:1788880879.231082:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:0.0:1788880879.231085:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880879.231086:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306515407 00000100:00000001:3.0:1788880879.231089:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880879.231091:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000087d24527 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880879.231093:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306510515 00000400:00000200:1.0:1788880879.231094:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880879.231096:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880879.231098:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880879.231100:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306515407 00000400:00000200:1.0:1788880879.231102:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000035d99bc2 00000100:00000001:3.0:1788880879.231102:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880879.231104:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12905 usecs (13 jiffies) 00000100:00000001:2.0:1788880879.231105:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788880879.231107:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000035d99bc2. 00000100:00000001:3.0:1788880879.231107:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880879.231112:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306510349 00000100:00000001:2.0:1788880879.231112:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880879.231116:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880879.231116:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880879.231117:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306515407 00000100:00000001:2.0:1788880879.231118:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880879.231120:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000200:1.0:1788880879.231123:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880895 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880879.231124:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306510349 00000100:00000001:2.0:1788880879.231126:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880879.231126:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880879.231130:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880879.231132:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880879.231135:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880879.231137:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880879.231139:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880879.231139:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880879.231143:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880895 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880879.231143:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880879.231146:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306515407 00000100:00000001:3.0:1788880879.231146:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880879.231152:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306510349 00000100:00000001:0.0:1788880879.231157:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306515407 00000100:00000001:0.0:1788880879.231161:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306510349 00000100:00000001:1.0:1788880879.231161:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880879.231165:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880879.231166:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306515407 00000100:00000001:0.0:1788880879.231170:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880879.231172:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880879.231175:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880879.231177:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880879.231183:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880879.231186:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000eaa15d72. 00000400:00000200:1.0:1788880879.231188:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880879.231190:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880879.231193:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009543d7d4 00000400:00000010:1.0:1788880879.231195:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009543d7d4. 00000100:00000001:1.0:1788880879.231205:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880879.231208:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880879.231215:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880879.231217:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880879.231224:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880879.231225:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880879.231296:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880879.231298:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000087d24527. 00000400:00000200:1.0:1788880879.231300:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880879.231303:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880879.231304:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d6cef011 00000400:00000010:1.0:1788880879.231305:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d6cef011. 00000100:00000001:1.0:1788880879.231307:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880879.231309:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880879.231316:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880879.231318:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880879.231324:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880879.231325:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788880879.232958:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880879.232970:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880879.232973:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880879.232976:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880879.232983:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880879.232994:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0ef80 00000400:00000200:3.0:1788880879.233115:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1041 [1] + 224 00000800:00000001:3.0:1788880879.233121:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880879.233125:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880879.233127:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880879.233129:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880879.233133:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880879.233137:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880879.233143:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880879.233193:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880879.233196:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880879.233259:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880879.233273:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880879.233274:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880879.233277:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880879.233279:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000002629a0e9. 00000400:00000200:1.0:1788880879.233284:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 00000400:00000010:1.0:1788880879.233286:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000100:00000001:1.0:1788880879.233288:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880879.233292:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880879.233299:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880879.233314:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880879.233316:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880879.233319:0:201286:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:3.0:1788880879.233320:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880879.233322:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:3.0:1788880879.233324:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880879.233325:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:3.0:1788880879.233326:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880879.233329:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880879.233333:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 02000000:00000001:1.0:1788880879.233335:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880879.233339:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f080 00000100:00001000:1.0:1788880879.233342:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=107 v=5 (5 5 5 5) 00000400:00000200:3.0:1788880879.233345:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1051 [1] + 224 00000100:00001000:1.0:1788880879.233351:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=107 v=5 (1 1 1 1) 00000800:00000001:3.0:1788880879.233351:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880879.233354:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880879.233357:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:3.0:1788880879.233357:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880879.233361:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880879.233364:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880879.233366:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880879.233366:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880879.233372:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880879.233375:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880879.233378:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880879.233386:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880879.233388:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880879.233410:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880879.233414:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880879.233416:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880879.233417:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880879.233421:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880879.233422:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880879.233426:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880879.233429:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880879.233431:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1788880879.233436:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780693376:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880879.233438:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880879.233447:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880879.233447:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f000 00000100:00000001:1.0:1788880879.233451:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:3.0:1788880879.233454:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1045 [1] + 224 00000100:00000001:0.0:1788880879.233455:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880879.233457:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780693376/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880879.233459:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788880879.233461:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880879.233464:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880879.233468:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880879.233469:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000010:0.0:1788880879.233470:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dd8b4e58. 02000000:00000001:1.0:1788880879.233473:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000200:0.0:1788880879.233475:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000800:00000001:3.0:1788880879.233475:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880879.233476:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000039ebda7d. 00000400:00000200:3.0:1788880879.233480:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1788880879.233483:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000200:3.0:1788880879.233484:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:0.0:1788880879.233486:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000020:00000001:1.0:1788880879.233487:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880879.233489:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880879.233491:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880879.233496:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880879.233496:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000001:1.0:1788880879.233500:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000200:3.0:1788880879.233503:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880879.233505:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 00000100:00000200:0.0:1788880879.233516:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880879.233518:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880879.233519:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880879.233523:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880879.233526:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:1.0:1788880879.233532:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880879.233536:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880879.233537:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880879.233539:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880879.233542:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880879.233542:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880879.233545:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880879.233545:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880879.233547:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880879.233550:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880879.233551:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000001:3.0:1788880879.233551:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:0.0:1788880879.233552:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880879.233555:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880879.233556:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880879.233558:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=107 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880879.233571:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=41 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880879.233572:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880879.233575:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880879.233576:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880879.233577:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880879.233581:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880879.233591:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880879.233592:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880879.233596:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:0.0:1788880879.233600:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788880879.233602:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880879.233603:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000001:2.0:1788880879.233605:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:0.0:1788880879.233606:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880879.233609:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000010:2.0:1788880879.233609:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000deb55c04. 00000100:00000001:0.0:1788880879.233612:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880879.233614:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000044b1d7fc 00000100:00000040:0.0:1788880879.233618:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880879.233619:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000044b1d7fc. 00000100:00000001:2.0:1788880879.233623:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880879.233628:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880879.233633:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880879.233636:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880879.233636:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880879.233644:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880879.233653:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880879.233663:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780693632:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880879.233667:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880879.233669:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880879.233671:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880879.233672:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880879.233675:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:0.0:1788880879.233677:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000003b122924 x1875776780693632/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880879.233678:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880879.233681:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880879.233684:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880879.233689:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880879.233689:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880879.233692:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880879.233694:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008b528dd3. 00000100:00001000:2.0:1788880879.233695:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=97 v=5 (5 5 5 5) 02000000:00000001:0.0:1788880879.233697:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880879.233699:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880879.233701:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00001000:2.0:1788880879.233701:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=97 v=5 (1 1 1 1) 00000020:00000001:0.0:1788880879.233704:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880879.233706:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000010:0.0:1788880879.233707:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5b428a0. 00000100:00000001:2.0:1788880879.233709:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880879.233710:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00010000:00000001:2.0:1788880879.233712:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:0.0:1788880879.233713:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880879.233715:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000003b122924. 00010000:00000001:2.0:1788880879.233716:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880879.233718:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880879.233720:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788880879.233725:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880879.233725:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880879.233726:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880879.233730:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880879.233732:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880879.233733:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880879.233734:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880879.233737:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880879.233740:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880879.233740:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880879.233749:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880879.233752:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880879.233753:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880879.233756:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880879.233763:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b5bd1bf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780693504:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880879.233767:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880879.233768:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880879.233770:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b5bd1bf6 x1875776780693504/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880895 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880879.233781:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880879.233783:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880879.233784:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003816d5cd. 02000000:00000001:2.0:1788880879.233787:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880879.233788:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880879.233789:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880879.233792:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880879.233793:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 02000000:00000001:2.0:1788880879.233796:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880879.233797:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880879.233798:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b5bd1bf6. 00000100:00000001:2.0:1788880879.233800:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880879.233801:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880879.233803:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880879.233807:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880879.233808:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880879.233810:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880879.233811:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880879.294349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880879.294355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880879.294359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880879.294360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880879.294392:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880879.294395:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880879.294398:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880880.318343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880880.318350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880880.318354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880880.318355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880880.318414:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880880.318417:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880880.318420:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880881.342363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880881.342371:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880881.342376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880881.342377:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880881.342409:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880881.342414:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880881.342417:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880882.366348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880882.366356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880882.366361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880882.366362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880882.366444:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880882.366448:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880882.366451:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880883.390358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880883.390372:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880883.390378:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880883.390379:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880883.390410:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880883.390414:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880883.390417:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788880884.350293:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306515407, at 4306515456 00000100:00000010:3.0:1788880884.350308:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004800b45d. 00000100:00000040:1.0:1788880884.350333:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306515407, at 4306515456 00000100:00000010:1.0:1788880884.350341:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009636b993. 00000100:00000040:2.0:1788880884.350460:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880884.350480:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880884.350485:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000884e74b8. 00000020:00000040:2.0:1788880884.350489:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880884.350494:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880884.350497:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880884.350501:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880884.350507:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 00000100:00000001:2.0:1788880884.350513:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880884.350523:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880884.350540:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880884.350543:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880884.350547:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000884e74b8] to pc [ptlrpcd_00_00+0] req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880884.350570:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880884.350573:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880884.350576:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880884.350579:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 00000020:00000040:2.0:1788880884.350582:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880884.350582:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880884.350587:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00100000:3.0:1788880884.350588:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 02000000:00000001:2.0:1788880884.350589:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880884.350591:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880884.350593:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880884.350597:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e53a8f34. 00000100:00000001:3.0:1788880884.350599:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880884.350601:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880884.350603:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880884.350606:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880884.350609:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880884.350614:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880884.350618:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880884.350623:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880884.350627:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880884.350628:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880884.350630:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880884.350633:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880884.350634:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880884.350635:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880884.350637:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_01+1] req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880884.350642:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880884.350645:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880884.350650:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880884.350653:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780693760:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880884.350656:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880884.350661:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880884.350661:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880884.350663:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880884.350666:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880884.350666:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880884.350670:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e28573ff. 00000100:00000001:2.0:1788880884.350674:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:3.0:1788880884.350674:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880884.350676:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788880884.350677:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000010:2.0:1788880884.350679:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000629edf55. 00000100:00000001:1.0:1788880884.350681:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:2.0:1788880884.350681:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000010:3.0:1788880884.350681:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004092e92f. 00000100:00000001:2.0:1788880884.350686:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880884.350688:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788880884.350688:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880884.350690:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880884.350690:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880884.350690:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005b101ac6. 00000100:00000001:1.0:1788880884.350694:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880884.350696:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c54a5c6f. 00000100:00000200:3.0:1788880884.350699:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780693760, portal 25 00000100:00000001:3.0:1788880884.350701:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880884.350704:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880884.350704:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938423808 : -126823771127808 : ffff8ca78a2c4000) 00000100:00000001:2.0:1788880884.350707:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880884.350709:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880884.350710:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880900 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880884.350715:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880884.350715:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880884.350720:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880884.350726:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880884.350730:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ec90f18d. 00000100:00000001:1.0:1788880884.350732:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880884.350732:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000200:3.0:1788880884.350734:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780693760, offset 0 00000100:00000001:1.0:1788880884.350735:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880884.350735:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880884.350738:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880884.350740:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880884.350740:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000629edf55] to pc [ptlrpcd_00_02+2] req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880884.350744:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880884.350744:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788880884.350750:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780693888:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880884.350753:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880884.350756:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880884.350758:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880884.350761:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788880884.350761:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880884.350766:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880884.350767:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002974b115. 00000400:00000200:3.0:1788880884.350768:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880884.350769:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880884.350770:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880884.350773:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:3.0:1788880884.350774:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880884.350775:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1788880884.350775:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000db88a7b4. 00000100:00000001:0.0:1788880884.350780:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880884.350781:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880884.350783:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:1.0:1788880884.350783:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000100:00000040:2.0:1788880884.350783:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11853) 00000400:00000200:3.0:1788880884.350785:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:1.0:1788880884.350788:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780693888, portal 10 00000100:00000040:0.0:1788880884.350790:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880884.350794:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880884.350797:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000400:00000200:3.0:1788880884.350799:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [498:498:256:4294967295] 192.168.204.111@tcp LPNI seq info [498:498:8:4294967295] 00000100:00000001:0.0:1788880884.350808:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880884.350809:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880884.350810:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880884.350817:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:3.0:1788880884.350820:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880884.350825:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880884.350830:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880884.350834:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:3.0:1788880884.350834:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00100000:0.0:1788880884.350837:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780694016:192.168.204.111@tcp:400: 00000800:00000010:3.0:1788880884.350841:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000001:0.0:1788880884.350842:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880884.350842:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880884.350847:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880884.350847:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1788880884.350848:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 02000000:00000001:0.0:1788880884.350850:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880884.350852:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880884.350853:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780693888, offset 0 00000800:00000200:3.0:1788880884.350853:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:0.0:1788880884.350855:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880884.350860:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000039bd304c. 02000000:00000001:0.0:1788880884.350863:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880884.350867:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880884.350867:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:0.0:1788880884.350869:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dd8b4e58. 00000800:00000200:3.0:1788880884.350872:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000400:00000010:0.0:1788880884.350875:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000400:00000200:1.0:1788880884.350878:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:0.0:1788880884.350881:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780694016, portal 10 00000400:00000200:1.0:1788880884.350883:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880884.350885:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880884.350886:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880884.350890:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880884.350891:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306520527 00000400:00000200:1.0:1788880884.350894:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880884.350895:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938432768 : -126823771118848 : ffff8ca78a2c6300) 00000100:00000001:3.0:1788880884.350899:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306515407 00000400:00000200:1.0:1788880884.350901:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880884.350905:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306520527 00000100:00000040:0.0:1788880884.350906:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880884.350908:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [499:499:255:4294967295] 192.168.204.111@tcp LPNI seq info [499:499:7:4294967295] 00000100:00000010:3.0:1788880884.350916:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004800b45d. 00000100:00000001:3.0:1788880884.350923:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880884.350928:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880884.350929:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880884.350929:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306520527 00000100:00000040:0.0:1788880884.350935:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880884.350936:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306515407 00000400:00000010:0.0:1788880884.350940:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000001:3.0:1788880884.350941:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306520527 00000800:00000200:1.0:1788880884.350943:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:0.0:1788880884.350946:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780694016, offset 0 00000100:00000010:3.0:1788880884.350947:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d54626ec. 00000800:00000010:1.0:1788880884.350949:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000100:00000001:3.0:1788880884.350952:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:0.0:1788880884.350956:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880884.350957:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880884.350958:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880884.350962:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880884.350964:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880884.350967:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880884.350969:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880884.350973:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880884.350973:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880884.350974:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880884.350977:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880884.350979:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880884.350981:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880884.350982:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880884.350982:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880884.350987:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788880884.350987:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 177360 usecs (178 jiffies) 00000100:00000001:3.0:1788880884.350987:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880884.350993:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880884.350994:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [500:500:254:4294967295] 192.168.204.111@tcp LPNI seq info [500:500:6:4294967295] 00000100:00000001:1.0:1788880884.350994:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306515634 00000100:00000001:1.0:1788880884.351000:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306520527 00000800:00000200:2.0:1788880884.351001:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880884.351004:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306515634 00000800:00000010:2.0:1788880884.351004:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:1.0:1788880884.351009:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306520527 00000400:00000200:0.0:1788880884.351010:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:2.0:1788880884.351011:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880884.351012:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306515634 00000100:00000001:1.0:1788880884.351016:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306520527 00000100:00000001:1.0:1788880884.351019:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306515634 00000400:00000200:2.0:1788880884.351021:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000200:0.0:1788880884.351022:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880884.351023:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306520527 00000400:00000200:2.0:1788880884.351026:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ec90f18d 00000100:00000001:1.0:1788880884.351027:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:0.0:1788880884.351028:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000bd68c00a. 00000100:00000001:1.0:1788880884.351029:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880884.351029:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ec90f18d. 00000100:00000001:1.0:1788880884.351033:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880884.351036:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880884.351036:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880884.351037:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880884.351042:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880900 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880884.351045:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880884.351049:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000bd68c00a type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880884.351054:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880884.351058:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12838 usecs (13 jiffies) 00000100:00000001:2.0:1788880884.351061:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880884.351062:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306515469 00000100:00000001:0.0:1788880884.351066:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306520527 00000100:00000040:2.0:1788880884.351067:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880900 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880884.351071:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306515469 00000100:00000001:0.0:1788880884.351076:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306520527 00000100:00000001:0.0:1788880884.351082:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306515469 00000100:00000001:2.0:1788880884.351083:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880884.351086:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306520527 00000100:00000001:2.0:1788880884.351087:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880884.351091:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306515469 00000100:00000001:0.0:1788880884.351099:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306520527 00000100:00000001:0.0:1788880884.351101:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880884.351104:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880884.351105:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880884.351106:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880884.351108:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000100:00000001:0.0:1788880884.351110:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880884.351112:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880884.351116:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880884.351117:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880884.351120:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880884.351125:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880884.351126:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000100:00000001:0.0:1788880884.351128:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:2.0:1788880884.351128:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:0.0:1788880884.351131:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880884.351132:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880884.351135:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880884.351137:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880884.351143:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880884.351144:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880884.351149:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880884.351151:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880884.351162:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880884.351164:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000bd68c00a. 00000400:00000200:2.0:1788880884.351167:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880884.351170:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880884.351172:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000400:00000010:2.0:1788880884.351174:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:2.0:1788880884.351176:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880884.351179:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880884.351186:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880884.351188:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880884.351194:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880884.351195:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880884.352526:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880884.352536:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880884.352538:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880884.352539:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880884.352545:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880884.352561:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0f100 00000400:00000200:1.0:1788880884.352566:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1059 [1] + 224 00000800:00000001:1.0:1788880884.352572:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880884.352575:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880884.352576:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880884.352579:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880884.352582:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880884.352584:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880884.352590:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880884.352599:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880884.352604:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880884.352607:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880884.352608:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:1.0:1788880884.352609:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880884.352612:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880884.352612:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880884.352617:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:3.0:1788880884.352619:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:1.0:1788880884.352622:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f180 00000400:00000001:3.0:1788880884.352622:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880884.352626:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004092e92f. 00000400:00000200:3.0:1788880884.352678:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005b101ac6 00000400:00000010:3.0:1788880884.352679:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005b101ac6. 00000100:00000001:3.0:1788880884.352681:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788880884.352683:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1061 [1] + 224 00000100:00000200:3.0:1788880884.352686:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880884.352689:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880884.352693:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880884.352696:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:3.0:1788880884.352698:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880884.352701:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880884.352704:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880884.352706:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880884.352710:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880884.352719:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880884.352724:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880884.352728:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880884.352730:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880884.352733:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880884.352737:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880884.352740:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880884.352742:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880884.352742:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880884.352744:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880884.352745:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880884.352747:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000200:1.0:1788880884.352748:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f200 00000400:00000001:2.0:1788880884.352749:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880884.352750:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000001:2.0:1788880884.352752:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880884.352754:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880884.352755:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000db88a7b4. 02000000:00000001:3.0:1788880884.352759:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880884.352760:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000400:00000010:2.0:1788880884.352763:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00001000:3.0:1788880884.352764:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=112 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880884.352766:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788880884.352769:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1069 [1] + 224 00000100:00001000:3.0:1788880884.352769:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=112 v=5 (1 1 1 1) 00000100:00000200:2.0:1788880884.352772:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880884.352777:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:1.0:1788880884.352778:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880884.352780:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880884.352783:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:1.0:1788880884.352785:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880884.352785:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880884.352790:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880884.352791:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880884.352796:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:3.0:1788880884.352799:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880884.352801:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880884.352802:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880884.352805:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880884.352822:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880884.352824:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880884.352826:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880884.352827:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880884.352827:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880884.352831:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880884.352831:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880884.352834:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880884.352836:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880884.352838:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880884.352839:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880884.352843:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880884.352844:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880884.352849:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:2.0:1788880884.352850:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=102 v=5 (5 5 5 5) 00000100:00100000:3.0:1788880884.352850:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780693760:192.168.204.111@tcp:400: 00000800:00000001:1.0:1788880884.352854:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788880884.352856:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=102 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880884.352860:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880884.352861:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880884.352863:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880884.352864:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00010000:00000001:2.0:1788880884.352867:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:3.0:1788880884.352868:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000884e74b8 x1875776780693760/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880884.352875:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:2.0:1788880884.352878:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880884.352879:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880884.352885:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000001:0.0:1788880884.352886:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880884.352887:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:2.0:1788880884.352889:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:0.0:1788880884.352890:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:3.0:1788880884.352891:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000010:0.0:1788880884.352894:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dd8b4e58. 00000100:00000001:2.0:1788880884.352895:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000010:3.0:1788880884.352895:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e28573ff. 00000100:00000001:2.0:1788880884.352899:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880884.352901:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 02000000:00000001:3.0:1788880884.352901:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:2.0:1788880884.352904:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880884.352905:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000020:00000001:3.0:1788880884.352905:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880884.352908:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880884.352915:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880884.352919:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880884.352919:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000020:00000001:3.0:1788880884.352920:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788880884.352921:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1788880884.352924:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 00000100:00000040:2.0:1788880884.352925:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880884.352929:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880884.352931:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880884.352934:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880884.352937:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000884e74b8. 00000100:00000001:3.0:1788880884.352942:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880884.352944:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880884.352948:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:2.0:1788880884.352949:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780693888:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880884.352954:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880884.352957:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880884.352957:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880884.352958:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880884.352960:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880884.352964:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880884.352967:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780693888/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880884.352968:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880884.352969:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880884.352972:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880884.352976:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880884.352978:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880884.352979:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880884.352980:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880884.352981:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880884.352986:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=112 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880884.352993:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00001000:0.0:1788880884.352994:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=46 v=5 (1 0 0 0) 02000000:00000001:2.0:1788880884.352997:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880884.352998:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000010:2.0:1788880884.352999:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002974b115. 00000100:00000001:0.0:1788880884.353000:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880884.353003:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:2.0:1788880884.353004:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880884.353006:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880884.353006:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880884.353008:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880884.353008:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e53a8f34. 02000000:00000001:2.0:1788880884.353012:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880884.353013:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00010000:00080000:0.0:1788880884.353015:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788880884.353015:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 00000100:00000001:2.0:1788880884.353019:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880884.353020:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880884.353022:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880884.353027:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880884.353029:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880884.353032:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880884.353032:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880884.353034:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880884.353037:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880884.353040:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880884.353042:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880884.353044:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880884.353047:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880884.353055:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880884.353057:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880884.353058:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880884.353062:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880884.353067:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780694016:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880884.353070:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880884.353071:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880884.353072:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000629edf55 x1875776780694016/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880900 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880884.353076:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880884.353077:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880884.353078:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000039bd304c. 02000000:00000001:0.0:1788880884.353081:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880884.353082:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880884.353083:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880884.353084:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880884.353085:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c54a5c6f. 02000000:00000001:0.0:1788880884.353087:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880884.353088:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880884.353089:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000629edf55. 00000100:00000001:0.0:1788880884.353091:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880884.353095:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880884.353096:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880884.353099:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880884.353099:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880884.353101:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880884.353102:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880884.414342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880884.414353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880884.414360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880884.414362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880884.414428:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880884.414432:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880884.414435:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880885.438385:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880885.438398:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880885.438401:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880885.438405:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880885.438411:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880885.438414:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880885.438416:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880885.438418:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880885.438423:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880885.438424:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880885.438475:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880885.438480:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880885.438484:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880886.462303:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880886.462309:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880886.462314:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880886.462315:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880886.462361:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880886.462364:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880886.462366:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880887.486351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880887.486355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880887.486361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880887.486364:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880887.486366:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880887.486369:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880887.486373:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880887.486375:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880887.486377:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880887.486379:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880887.486424:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880887.486428:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880887.486431:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880888.510305:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880888.510310:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880888.510321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880888.510322:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880888.510375:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880888.510378:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880888.510380:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880889.470343:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306520527, at 4306520576 00000100:00000040:1.0:1788880889.470343:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306520527, at 4306520576 00000100:00000040:2.0:1788880889.470343:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306520527, at 4306520576 00000100:00000010:1.0:1788880889.470360:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004800b45d. 00000100:00000010:2.0:1788880889.470362:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000eeb663f. 00000100:00000010:0.0:1788880889.470374:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f093018f. 00000100:00000040:3.0:1788880889.470387:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306520527, at 4306520576 00000100:00000010:3.0:1788880889.470393:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d54626ec. 00000100:00000040:1.0:1788880889.470399:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880889.470403:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880889.470407:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000020:00000040:1.0:1788880889.470410:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880889.470414:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880889.470416:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880889.470419:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880889.470425:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:1.0:1788880889.470430:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880889.470438:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880889.470445:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880889.470447:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880889.470451:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000077a1dd13] to pc [ptlrpcd_00_03+3] req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880889.470472:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880889.470474:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880889.470476:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880889.470478:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c09231ec. 00000020:00000040:1.0:1788880889.470480:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880889.470481:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880889.470482:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880889.470483:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880889.470485:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000252f4482. 00000100:00000001:1.0:1788880889.470487:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880889.470489:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880889.470494:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880889.470495:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880889.470497:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c09231ec] to pc [ptlrpcd_00_00+0] req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880889.470497:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880889.470503:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880889.470508:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880889.470515:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880889.470516:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880889.470516:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880889.470520:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880889.470522:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880889.470529:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880889.470529:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:1.0:1788880889.470533:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000094565713. 00000100:00000001:3.0:1788880889.470533:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880889.470535:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880889.470536:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:1.0:1788880889.470537:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880889.470540:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880889.470541:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:3.0:1788880889.470542:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880889.470543:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880889.470547:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880889.470548:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:2.0:1788880889.470551:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880889.470552:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880889.470553:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880889.470560:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880889.470560:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880889.470561:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880889.470563:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880889.470565:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880889.470569:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880889.470572:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880889.470573:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880889.470573:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880889.470576:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880889.470579:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880889.470583:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000094565713] to pc [ptlrpcd_00_01+1] req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880889.470583:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780694144:192.168.204.111@tcp:400: 00000100:00100000:3.0:1788880889.470585:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780694272:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880889.470596:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880889.470597:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880889.470601:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880889.470602:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880889.470604:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880889.470607:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880889.470611:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880889.470611:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:3.0:1788880889.470614:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880889.470615:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:3.0:1788880889.470618:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000001be0e14c. 00000100:00000040:1.0:1788880889.470619:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000010:2.0:1788880889.470619:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008d432156. 02000000:00000001:3.0:1788880889.470623:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880889.470625:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880889.470628:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11858) 00000400:00000010:3.0:1788880889.470629:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004092e92f. 00000400:00000010:2.0:1788880889.470632:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000deb55c04. 00000100:00000001:0.0:1788880889.470639:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:2.0:1788880889.470639:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000044b1d7fc. 00000100:00000001:0.0:1788880889.470643:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880889.470646:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:2.0:1788880889.470647:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780694144, portal 25 00000100:00000001:2.0:1788880889.470652:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880889.470653:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880889.470654:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000040:2.0:1788880889.470665:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880905 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880889.470666:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880889.470669:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880889.470673:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880889.470676:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880889.470681:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880889.470682:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880889.470688:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780694400:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880889.470688:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880889.470693:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c0c07443. 00000100:00000001:0.0:1788880889.470694:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:3.0:1788880889.470697:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005b101ac6. 02000000:00000001:0.0:1788880889.470699:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000200:2.0:1788880889.470699:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780694144, offset 0 02000000:00000001:0.0:1788880889.470703:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880889.470704:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780694272, portal 10 02000000:00000001:0.0:1788880889.470709:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880889.470709:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788880889.470713:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880889.470713:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938056832 : -126823771494784 : ffff8ca78a26a680) 02000000:00000010:0.0:1788880889.470715:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000072dc05f6. 02000000:00000001:0.0:1788880889.470719:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880889.470724:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880889.470729:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:0.0:1788880889.470730:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000dd8b4e58. 00000400:00000010:0.0:1788880889.470737:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000400:00000200:2.0:1788880889.470740:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:0.0:1788880889.470743:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780694400, portal 10 00000100:00000001:3.0:1788880889.470744:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:2.0:1788880889.470747:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880889.470748:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:3.0:1788880889.470751:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880889.470752:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055040 : -126823771496576 : ffff8ca78a269f80) 00000400:00000200:2.0:1788880889.470755:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:3.0:1788880889.470755:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000100:00000200:3.0:1788880889.470761:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780694272, offset 0 00000400:00000200:2.0:1788880889.470762:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:0.0:1788880889.470764:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880889.470768:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880889.470778:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:2.0:1788880889.470778:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [501:501:256:4294967295] 192.168.204.111@tcp LPNI seq info [501:501:8:4294967295] 00000100:00000040:0.0:1788880889.470785:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880889.470789:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000100:00000200:0.0:1788880889.470794:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780694400, offset 0 00000400:00000200:2.0:1788880889.470798:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:0.0:1788880889.470804:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880889.470808:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880889.470809:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880889.470816:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880889.470818:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880889.470820:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880889.470822:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000010:2.0:1788880889.470823:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e53a8f34. 00000400:00000200:3.0:1788880889.470826:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [502:502:255:4294967295] 192.168.204.111@tcp LPNI seq info [502:502:7:4294967295] 00000400:00000200:3.0:1788880889.470836:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880889.470841:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880889.470846:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880889.470849:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:3.0:1788880889.470853:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000800:00000200:2.0:1788880889.470854:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880889.470856:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:3.0:1788880889.470861:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880889.470862:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e53a8f34 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880889.470865:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880889.470869:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880889.470872:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788880889.470873:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880889.470878:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880889.470882:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [503:503:254:4294967295] 192.168.204.111@tcp LPNI seq info [503:503:6:4294967295] 00000100:00000001:2.0:1788880889.470885:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880889.470886:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880889.470893:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 176171 usecs (177 jiffies) 00000100:00000001:2.0:1788880889.470894:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306525647 00000400:00000200:0.0:1788880889.470897:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880889.470900:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306520753 00000100:00000010:2.0:1788880889.470902:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005fb7b615. 00000800:00000200:0.0:1788880889.470907:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880889.470909:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000010:0.0:1788880889.470912:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ae4abc27. 00000100:00000001:2.0:1788880889.470916:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306525647 00000100:00000001:3.0:1788880889.470916:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306525647 00000800:00000200:0.0:1788880889.470920:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:2.0:1788880889.470922:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045d7c07c. 00000800:00000200:1.0:1788880889.470926:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880889.470926:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306520753 00000100:00000001:2.0:1788880889.470928:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000010:1.0:1788880889.470930:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e53a8f34. 00000800:00000200:0.0:1788880889.470931:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788880889.470935:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306525647 00000100:00000001:3.0:1788880889.470935:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306525647 00000400:00000200:1.0:1788880889.470936:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:0.0:1788880889.470937:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ae4abc27 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880889.470942:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306520753 00000100:00000010:2.0:1788880889.470943:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008b80730b. 00000100:00000001:0.0:1788880889.470945:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880889.470947:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880889.470950:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000040:0.0:1788880889.470951:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12770 usecs (13 jiffies) 00000400:00000200:1.0:1788880889.470952:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c0c07443 00000100:00000001:2.0:1788880889.470956:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306525647 00000400:00000010:1.0:1788880889.470957:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c0c07443. 00000100:00000001:3.0:1788880889.470957:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306525647 00000100:00000001:0.0:1788880889.470958:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306520589 00000100:00000001:1.0:1788880889.470963:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880889.470963:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306520753 00000100:00000001:0.0:1788880889.470964:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306525647 00000100:00000010:2.0:1788880889.470964:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fa7e66e6. 00000100:00000200:1.0:1788880889.470970:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880905 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880889.470971:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880889.470972:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306520589 00000100:00000001:2.0:1788880889.470977:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880889.470977:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306525647 00000100:00000001:0.0:1788880889.470979:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306525647 00000100:00000001:3.0:1788880889.470983:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880889.470985:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880889.470989:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880889.470992:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306520589 00000100:00000001:2.0:1788880889.470992:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880889.470993:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880889.470999:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880889.470999:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880889.471001:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880889.471005:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306525647 00000100:00000040:1.0:1788880889.471008:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880905 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880889.471010:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306520589 00000100:00000001:2.0:1788880889.471012:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880889.471012:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880889.471016:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306525647 00000100:00000001:2.0:1788880889.471017:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880889.471017:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880889.471023:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880889.471024:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880889.471025:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880889.471025:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880889.471028:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880889.471029:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880889.471030:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880889.471030:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880889.471032:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880889.471034:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880889.471036:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880889.471037:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880889.471039:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880889.471041:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:1.0:1788880889.471043:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880889.471046:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:0.0:1788880889.471047:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788880889.471048:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880889.471050:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:1.0:1788880889.471054:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880889.471055:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880889.471058:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880889.471059:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000100:00000001:0.0:1788880889.471061:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880889.471061:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000100:00000001:1.0:1788880889.471066:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880889.471067:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880889.471070:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880889.471076:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880889.471077:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880889.471089:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880889.471090:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880889.471115:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880889.471117:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ae4abc27. 00000400:00000200:1.0:1788880889.471119:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880889.471121:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880889.471122:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000400:00000010:1.0:1788880889.471123:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:1.0:1788880889.471125:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880889.471126:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880889.471130:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880889.471131:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880889.471135:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880889.471136:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880889.472517:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880889.472523:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880889.472525:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880889.472527:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880889.472532:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880889.472539:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0f280 00000400:00000200:2.0:1788880889.472550:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1071 [1] + 224 00000800:00000001:2.0:1788880889.472554:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880889.472556:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880889.472558:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880889.472560:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880889.472564:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880889.472567:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880889.472574:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880889.472585:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880889.472591:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880889.472594:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880889.472595:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880889.472596:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880889.472601:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880889.472602:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880889.472605:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880889.472608:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f300 00000400:00000001:3.0:1788880889.472611:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880889.472613:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880889.472614:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000deb55c04. 00000400:00000200:3.0:1788880889.472617:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000044b1d7fc 00000400:00000010:3.0:1788880889.472618:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000044b1d7fc. 00000100:00000001:3.0:1788880889.472621:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880889.472624:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1079 [1] + 224 00000100:00000200:3.0:1788880889.472627:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880889.472632:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880889.472638:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880889.472641:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880889.472643:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880889.472652:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880889.472655:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880889.472657:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880889.472659:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880889.472663:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000200:2.0:1788880889.472664:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880889.472666:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880889.472670:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:2.0:1788880889.472675:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880889.472679:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880889.472684:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880889.472691:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=117 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880889.472695:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=117 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880889.472698:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880889.472699:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880889.472700:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:3.0:1788880889.472702:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880889.472705:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880889.472707:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880889.472708:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880889.472713:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880889.472714:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880889.472722:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:2.0:1788880889.472724:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880889.472726:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880889.472729:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880889.472730:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880889.472731:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880889.472734:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880889.472735:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880889.472741:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880889.472741:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:0.0:1788880889.472745:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880889.472749:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004092e92f. 00000400:00000200:0.0:1788880889.472754:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005b101ac6 00000100:00100000:3.0:1788880889.472755:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780694144:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880889.472756:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f380 00000400:00000010:0.0:1788880889.472758:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005b101ac6. 00000100:00000001:0.0:1788880889.472766:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880889.472766:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880889.472770:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x107d [1] + 224 00000100:00000001:3.0:1788880889.472771:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:0.0:1788880889.472773:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880889.472778:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780694144/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880889.472780:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880889.472786:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880889.472790:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:0.0:1788880889.472799:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880889.472799:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880889.472799:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880889.472803:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788880889.472807:0:201280:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000010:3.0:1788880889.472807:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008d432156. 00000100:00000001:0.0:1788880889.472811:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880889.472813:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880889.472813:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880889.472814:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880889.472817:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880889.472820:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880889.472824:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:2.0:1788880889.472824:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880889.472825:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:0.0:1788880889.472829:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000010:3.0:1788880889.472830:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 00000100:00000001:0.0:1788880889.472832:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880889.472835:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880889.472839:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880889.472839:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880889.472843:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00001000:0.0:1788880889.472844:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=107 v=5 (5 5 5 5) 00000100:00000010:3.0:1788880889.472845:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:2.0:1788880889.472847:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880889.472850:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000800:00000001:2.0:1788880889.472852:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880889.472853:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=107 v=5 (1 1 1 1) 00000800:00000001:2.0:1788880889.472855:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880889.472857:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880889.472858:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880889.472860:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880889.472861:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880889.472864:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880889.472866:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:0.0:1788880889.472867:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880889.472869:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880889.472869:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880889.472871:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:1.0:1788880889.472872:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880889.472874:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880889.472875:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000001:1.0:1788880889.472876:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880889.472879:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000001:1.0:1788880889.472880:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880889.472880:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880889.472881:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880889.472883:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000dd8b4e58. 00000400:00000200:1.0:1788880889.472887:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000100:00000040:0.0:1788880889.472888:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880889.472891:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:1.0:1788880889.472896:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880889.472897:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880889.472900:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:1.0:1788880889.472901:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880889.472902:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880889.472908:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880889.472925:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780694272:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880889.472925:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880889.472932:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880889.472935:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880889.472939:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c09231ec x1875776780694272/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880889.472940:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880889.472943:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880889.472946:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880889.472949:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880889.472952:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880889.472952:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880889.472955:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880889.472956:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880889.472959:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000001be0e14c. 02000000:00000001:1.0:1788880889.472959:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880889.472963:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00001000:1.0:1788880889.472964:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=117 v=5 (5 5 5 5) 00000020:00000001:0.0:1788880889.472965:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880889.472967:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00001000:1.0:1788880889.472969:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=51 v=5 (1 0 0 0) 00000020:00000001:0.0:1788880889.472970:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880889.472972:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000252f4482. 00000100:00000001:1.0:1788880889.472974:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880889.472976:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880889.472977:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880889.472979:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00010000:00000001:1.0:1788880889.472980:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000010:0.0:1788880889.472981:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c09231ec. 00000100:00000001:0.0:1788880889.472984:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00010000:00080000:1.0:1788880889.472984:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880889.472989:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880889.472994:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880889.472994:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880889.472998:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880889.472998:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880889.472999:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880889.473000:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880889.473002:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880889.473003:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880889.473004:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880889.473005:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880889.473008:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880889.473014:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880889.473015:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880889.473017:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880889.473018:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880889.473023:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780694400:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880889.473025:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880889.473026:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880889.473027:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000094565713 x1875776780694400/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880905 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880889.473030:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880889.473032:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880889.473033:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000072dc05f6. 02000000:00000001:1.0:1788880889.473035:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880889.473036:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880889.473037:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880889.473038:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880889.473039:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 02000000:00000001:1.0:1788880889.473040:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880889.473041:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880889.473042:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000094565713. 00000100:00000001:1.0:1788880889.473044:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880889.473045:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880889.473046:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880889.473049:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880889.473049:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880889.473050:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880889.473051:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880889.534314:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880889.534321:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880889.534327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880889.534329:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880889.534376:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880889.534380:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880889.534384:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880890.558320:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880890.558326:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880890.558331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880890.558332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880890.558379:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880890.558382:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880890.558384:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880891.582345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880891.582351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880891.582357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880891.582357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880891.582382:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880891.582386:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880891.582388:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880892.606406:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880892.606414:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880892.606420:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880892.606420:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880892.606477:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880892.606481:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880892.606483:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880893.630322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880893.630328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880893.630332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880893.630333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880893.630363:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880893.630366:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880893.630368:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880894.590355:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306525647, at 4306525696 00000100:00000040:3.0:1788880894.590355:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306525647, at 4306525696 00000100:00000040:1.0:1788880894.590362:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306525647, at 4306525696 00000100:00000010:3.0:1788880894.590372:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fa7e66e6. 00000100:00000010:2.0:1788880894.590373:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008b80730b. 00000100:00000010:1.0:1788880894.590376:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045d7c07c. 00000100:00000040:0.0:1788880894.590382:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306525647, at 4306525696 00000100:00000010:0.0:1788880894.590390:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005fb7b615. 00000100:00000040:1.0:1788880894.590408:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880894.590414:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880894.590422:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000094565713. 00000020:00000040:1.0:1788880894.590426:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880894.590429:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880894.590469:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880894.590481:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880894.590488:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:1.0:1788880894.590495:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880894.590504:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880894.590514:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880894.590517:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880894.590522:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000094565713] to pc [ptlrpcd_00_02+2] req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880894.590543:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880894.590546:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880894.590549:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880894.590552:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000020:00000040:1.0:1788880894.590555:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880894.590558:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880894.590559:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880894.590561:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880894.590564:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:1.0:1788880894.590567:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880894.590568:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880894.590572:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880894.590574:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880894.590578:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880894.590579:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880894.590584:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880894.590586:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880894.590589:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880894.590590:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880894.590597:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca5e4c87] to pc [ptlrpcd_00_03+3] req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880894.590610:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880894.590614:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880894.590620:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880894.590622:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880894.590623:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880894.590627:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880894.590629:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788880894.590635:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780694528:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880894.590635:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880894.590638:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c27d4622. 00000100:00000001:0.0:1788880894.590643:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000020:00000040:1.0:1788880894.590644:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880894.590644:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880894.590649:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880894.590650:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880894.590650:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880894.590651:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880894.590655:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880894.590656:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880894.590656:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880894.590661:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880894.590664:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000252f4482. 00000100:00000040:3.0:1788880894.590664:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788880894.590668:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000fc6edce. 00000100:00000001:1.0:1788880894.590669:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880894.590674:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880894.590680:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880894.590681:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880894.590684:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880894.590685:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000fe5bffa. 02000000:00000001:3.0:1788880894.590689:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880894.590693:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880894.590693:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880894.590696:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880894.590697:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880894.590702:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c27d4622] to pc [ptlrpcd_00_00+0] req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880894.590706:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780694656:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880894.590715:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880894.590719:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880894.590721:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880894.590724:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880894.590727:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005578e134. 02000000:00000001:3.0:1788880894.590729:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880894.590732:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 00000400:00000010:3.0:1788880894.590734:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c54f86e6. 00000100:00000001:1.0:1788880894.590739:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880894.590741:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780694528, portal 25 00000100:00000040:1.0:1788880894.590742:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880894.590742:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 00000100:00000001:0.0:1788880894.590747:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000200:3.0:1788880894.590748:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780694656, portal 10 00000100:00000001:0.0:1788880894.590750:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055040 : -126823771496576 : ffff8ca78a269f80) 00000100:00000040:1.0:1788880894.590750:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880894.590754:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880894.590757:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000040:1.0:1788880894.590760:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11863) 00000100:00000001:2.0:1788880894.590761:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880894.590762:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880910 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880894.590766:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880894.590768:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880894.590770:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880894.590773:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880894.590777:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880894.590778:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880894.590793:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880894.590798:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:0.0:1788880894.590800:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880894.590802:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880894.590806:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000033f4d237. 00000100:00000001:3.0:1788880894.590809:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:0.0:1788880894.590811:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780694528, offset 0 02000000:00000001:2.0:1788880894.590814:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1788880894.590817:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880894.590818:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880894.590822:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880894.590823:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880894.590823:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005f2b805f. 00000100:00100000:2.0:1788880894.590828:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780694784:192.168.204.111@tcp:400: 00000100:00000200:3.0:1788880894.590830:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780694656, offset 0 00000100:00000001:2.0:1788880894.590832:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880894.590835:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880894.590837:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880894.590838:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880894.590841:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880894.590842:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000010:2.0:1788880894.590843:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008d264e32. 02000000:00000001:2.0:1788880894.590847:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880894.590850:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:2.0:1788880894.590851:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000011bf8993. 00000400:00000010:2.0:1788880894.590856:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cc94f399. 00000400:00000200:0.0:1788880894.590859:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:2.0:1788880894.590860:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780694784, portal 10 00000100:00000001:2.0:1788880894.590863:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880894.590866:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880894.590866:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938049664 : -126823771501952 : ffff8ca78a268a80) 00000400:00000200:0.0:1788880894.590873:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:2.0:1788880894.590874:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880894.590882:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [504:504:256:4294967295] 192.168.204.111@tcp LPNI seq info [504:504:8:4294967295] 00000100:00000001:2.0:1788880894.590885:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880894.590890:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880894.590892:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000084794ee4. 00000100:00000200:2.0:1788880894.590893:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780694784, offset 0 00000400:00000200:2.0:1788880894.590900:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880894.590905:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880894.590912:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880894.590916:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788880894.590918:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000005f53e16f. 00000400:00000200:3.0:1788880894.590920:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880894.590926:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788880894.590930:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880894.590933:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880894.590936:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880894.590939:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [505:505:255:4294967295] 192.168.204.111@tcp LPNI seq info [505:505:7:4294967295] 00000800:00000200:0.0:1788880894.590943:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880894.590948:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000005f53e16f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880894.590954:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880894.590960:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880894.590963:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000400:00000200:2.0:1788880894.590965:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788880894.590970:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880894.590971:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880894.590972:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880894.590979:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306530767 00000400:00000200:2.0:1788880894.590981:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880894.590982:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880894.590987:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000010:0.0:1788880894.590988:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006bd48548. 00000400:00000200:2.0:1788880894.590988:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880894.590993:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880894.590995:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880894.590996:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000040:3.0:1788880894.590999:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 175001 usecs (176 jiffies) 00000100:00000001:0.0:1788880894.591003:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306530767 00000400:00000200:2.0:1788880894.591004:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [506:506:254:4294967295] 192.168.204.111@tcp LPNI seq info [506:506:6:4294967295] 00000100:00000001:3.0:1788880894.591004:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306525872 00000100:00000010:0.0:1788880894.591009:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c0bfe00. 00000100:00000001:3.0:1788880894.591010:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306530767 00000100:00000001:3.0:1788880894.591015:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306525872 00000100:00000001:0.0:1788880894.591016:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:2.0:1788880894.591021:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880894.591028:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306530767 00000100:00000001:3.0:1788880894.591028:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306530767 00000800:00000200:2.0:1788880894.591033:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880894.591035:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306525872 00000100:00000010:0.0:1788880894.591036:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000029f028ab. 00000800:00000010:2.0:1788880894.591039:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e53a8f34. 00000100:00000001:0.0:1788880894.591042:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:2.0:1788880894.591047:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880894.591048:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306530767 00000100:00000001:3.0:1788880894.591049:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306530767 00000100:00000001:3.0:1788880894.591054:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306525872 00000100:00000010:0.0:1788880894.591056:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005b8bebe2. 00000100:00000001:0.0:1788880894.591062:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:2.0:1788880894.591065:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880894.591068:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880894.591069:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306530767 00000800:00000200:2.0:1788880894.591070:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e53a8f34 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880894.591074:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880894.591076:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880894.591080:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880894.591081:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880894.591082:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12702 usecs (13 jiffies) 00000100:00000001:3.0:1788880894.591086:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880894.591087:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880894.591088:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306525709 00000100:00000001:3.0:1788880894.591090:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880894.591094:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880894.591094:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306530767 00000100:00000001:3.0:1788880894.591095:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880894.591097:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880894.591101:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:1.0:1788880894.591101:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000005f53e16f. 00000100:00000001:2.0:1788880894.591101:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306525709 00000100:00000001:3.0:1788880894.591103:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880894.591106:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880894.591107:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306530767 00000100:00000001:0.0:1788880894.591108:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880894.591109:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880894.591110:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880894.591112:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880894.591113:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306525709 00000100:00000001:3.0:1788880894.591114:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880894.591117:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880894.591118:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880894.591119:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306530767 00000400:00000200:1.0:1788880894.591122:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880894.591123:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880894.591124:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306525709 00000100:00000001:3.0:1788880894.591124:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880894.591130:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306530767 00000400:00000200:1.0:1788880894.591131:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000033f4d237 00000100:00000001:2.0:1788880894.591133:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788880894.591136:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000033f4d237. 00000100:00000001:2.0:1788880894.591137:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880894.591142:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880894.591142:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880894.591147:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880894.591150:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880910 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880894.591152:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880894.591155:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880894.591159:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880894.591162:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880894.591165:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880894.591167:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880894.591168:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880894.591173:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880910 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880894.591181:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880894.591183:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880894.591195:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880894.591197:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:1.0:1788880894.591200:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880894.591203:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880894.591205:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005f2b805f 00000400:00000010:1.0:1788880894.591207:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005f2b805f. 00000100:00000001:1.0:1788880894.591209:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880894.591211:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880894.591218:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880894.591220:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880894.591225:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880894.591227:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880894.591389:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880894.591392:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e53a8f34. 00000400:00000200:1.0:1788880894.591396:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880894.591399:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880894.591402:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000084794ee4 00000400:00000010:1.0:1788880894.591404:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000084794ee4. 00000100:00000001:1.0:1788880894.591407:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880894.591410:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880894.591417:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880894.591419:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880894.591424:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880894.591425:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880894.593189:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880894.593201:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880894.593205:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880894.593207:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880894.593215:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880894.593228:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0f400 00000400:00000200:1.0:1788880894.593257:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1089 [1] + 224 00000800:00000001:1.0:1788880894.593262:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880894.593280:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880894.593283:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880894.593287:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880894.593292:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880894.593297:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880894.593306:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880894.593324:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880894.593330:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880894.593333:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880894.593371:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880894.593376:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880894.593381:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880894.593383:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880894.593384:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000fe5bffa. 00000400:00000200:0.0:1788880894.593388:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000400:00000010:0.0:1788880894.593391:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000001:0.0:1788880894.593394:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880894.593398:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880894.593407:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880894.593414:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880894.593416:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880894.593418:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880894.593421:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880894.593423:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880894.593425:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880894.593428:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880894.593443:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=122 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880894.593448:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=122 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880894.593450:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880894.593452:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880894.593454:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880894.593456:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880894.593459:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880894.593464:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880894.593471:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880894.593474:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880894.593476:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880894.593476:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880894.593490:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880894.593491:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880894.593494:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880894.593497:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880894.593507:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00100000:0.0:1788880894.593511:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780694528:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880894.593517:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880894.593518:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f480 00000100:00000001:0.0:1788880894.593521:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788880894.593527:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x108d [1] + 224 00000100:00000040:0.0:1788880894.593528:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000094565713 x1875776780694528/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880894.593537:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880894.593540:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880894.593542:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880894.593544:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000fc6edce. 02000000:00000001:0.0:1788880894.593548:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880894.593551:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000800:00000001:2.0:1788880894.593552:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:0.0:1788880894.593553:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000200:2.0:1788880894.593556:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000020:00000001:0.0:1788880894.593558:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880894.593562:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 00000400:00000200:2.0:1788880894.593564:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:0.0:1788880894.593568:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788880894.593569:0:201280:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:0.0:1788880894.593573:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000200:2.0:1788880894.593576:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880894.593577:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000094565713. 00000100:00000001:0.0:1788880894.593584:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880894.593586:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880894.593590:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880894.593590:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880894.593605:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880894.593607:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880894.593609:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880894.593610:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880894.593613:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880894.593623:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880894.593628:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880894.593630:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880894.593633:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880894.593639:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880894.593639:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880894.593643:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880894.593646:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f500 00000400:00000001:3.0:1788880894.593647:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880894.593651:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880894.593653:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c54f86e6. 00000400:00000200:3.0:1788880894.593656:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000400:00000010:3.0:1788880894.593658:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:3.0:1788880894.593661:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880894.593667:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1099 [1] + 224 00000100:00000200:3.0:1788880894.593668:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880894.593674:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880894.593680:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880894.593683:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880894.593689:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880894.593693:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880894.593696:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880894.593698:0:201288:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:2.0:1788880894.593700:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:3.0:1788880894.593701:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880894.593704:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880894.593706:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880894.593706:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880894.593710:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880894.593712:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880894.593716:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=112 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880894.593723:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=112 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880894.593728:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:2.0:1788880894.593730:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880894.593730:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880894.593734:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880894.593738:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880894.593743:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788880894.593751:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788880894.593751:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880894.593755:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000800:00000001:2.0:1788880894.593756:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880894.593757:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880894.593760:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880894.593764:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880894.593772:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880894.593775:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880894.593777:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880894.593777:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880894.593780:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880894.593783:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880894.593784:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880894.593788:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880894.593790:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000011bf8993. 00000100:00100000:3.0:1788880894.593796:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780694656:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880894.593797:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cc94f399 00000400:00000010:0.0:1788880894.593800:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cc94f399. 00000100:00000001:3.0:1788880894.593805:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880894.593806:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880894.593809:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:0.0:1788880894.593813:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880894.593815:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780694656/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880894.593837:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880894.593840:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000200:0.0:1788880894.593842:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880894.593843:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005578e134. 02000000:00000001:3.0:1788880894.593848:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880894.593851:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880894.593853:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880894.593856:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:3.0:1788880894.593856:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880894.593859:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1788880894.593859:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 02000000:00000001:3.0:1788880894.593864:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880894.593865:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880894.593867:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:0.0:1788880894.593868:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880894.593871:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000010:3.0:1788880894.593871:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:0.0:1788880894.593872:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880894.593874:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880894.593878:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=122 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880894.593879:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=56 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880894.593881:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880894.593882:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880894.593883:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880894.593886:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00010000:00080000:0.0:1788880894.593888:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880894.593892:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880894.593896:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880894.593899:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880894.593901:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880894.593902:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880894.593904:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880894.593907:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880894.593910:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880894.593912:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880894.593914:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880894.593917:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880894.593919:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880894.593929:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880894.593932:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880894.593933:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880894.593936:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880894.593953:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780694784:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880894.593957:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880894.593962:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880894.593964:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c27d4622 x1875776780694784/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880910 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880894.593969:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880894.593970:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880894.593972:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008d264e32. 02000000:00000001:0.0:1788880894.593974:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880894.593975:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880894.593976:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880894.593978:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880894.593979:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000252f4482. 02000000:00000001:0.0:1788880894.593981:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880894.593982:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880894.593983:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c27d4622. 00000100:00000001:0.0:1788880894.593985:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880894.593986:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880894.593987:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880894.593991:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880894.593992:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880894.593993:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880894.593995:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880894.654345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880894.654352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880894.654358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880894.654359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880894.654406:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880894.654409:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880894.654411:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880895.678303:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880895.678309:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880895.678311:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880895.678313:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880895.678316:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880895.678318:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880895.678320:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880895.678321:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880895.678324:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880895.678325:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880895.678360:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880895.678365:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880895.678368:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880896.702370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880896.702377:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880896.702383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880896.702384:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880896.702436:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880896.702439:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880896.702441:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880897.726385:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880897.726392:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880897.726399:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880897.726403:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880897.726406:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880897.726409:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880897.726411:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880897.726414:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880897.726415:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880897.726417:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880897.726478:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880897.726482:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880897.726485:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880898.750352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880898.750365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880898.750372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880898.750373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880898.750396:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880898.750402:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880898.750405:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880899.710302:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306530767, at 4306530816 00000100:00000040:2.0:1788880899.710302:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306530767, at 4306530816 00000100:00000040:3.0:1788880899.710304:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880899.710314:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880899.710316:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c0bfe00. 00000100:00000010:2.0:1788880899.710321:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000029f028ab. 00000100:00000010:3.0:1788880899.710321:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000020:00000040:3.0:1788880899.710327:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880899.710333:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880899.710336:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880899.710340:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880899.710383:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306530767, at 4306530816 00000100:00000010:0.0:1788880899.710387:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006bd48548. 02000000:00000010:3.0:1788880899.710424:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:3.0:1788880899.710437:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880899.710450:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880899.710459:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880899.710462:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880899.710467:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_01+1] req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880899.710493:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880899.710496:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880899.710499:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880899.710503:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788880899.710505:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880899.710506:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880899.710509:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880899.710511:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880899.710512:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880899.710521:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880899.710522:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880899.710528:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:3.0:1788880899.710531:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880899.710534:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880899.710537:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880899.710560:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880899.710571:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880899.710572:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880899.710578:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880899.710589:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880899.710592:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880899.710594:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880899.710604:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880899.710604:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_02+2] req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880899.710616:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880899.710622:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780694912:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880899.710627:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880899.710632:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880899.710637:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880899.710639:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880899.710644:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004ffd965d. 02000000:00000001:1.0:1788880899.710645:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880899.710651:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000020fda082. 00000100:00000001:3.0:1788880899.710653:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880899.710656:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000100:00000040:3.0:1788880899.710656:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880899.710667:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000200:1.0:1788880899.710669:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780694912, portal 25 00000100:00000010:3.0:1788880899.710671:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000100:00000001:0.0:1788880899.710672:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880899.710673:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000020:00000040:3.0:1788880899.710675:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880899.710676:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000100:00000001:2.0:1788880899.710680:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880899.710680:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880899.710687:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880899.710687:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880899.710689:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880899.710692:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880899.710693:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880915 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880899.710695:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880899.710706:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000040:0.0:1788880899.710707:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880899.710718:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880899.710732:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880899.710757:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880899.710767:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880899.710772:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880899.710774:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880899.710780:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 02000000:00000001:0.0:1788880899.710781:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880899.710782:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788880899.710786:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880899.710788:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880899.710790:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780694912, offset 0 02000000:00000001:0.0:1788880899.710797:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880899.710800:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_03+3] req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880899.710805:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780695040:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788880899.710810:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880899.710821:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880899.710827:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880899.710829:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880899.710830:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880899.710833:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880899.710835:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880899.710840:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:0.0:1788880899.710845:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000033d66d82. 00000400:00000200:1.0:1788880899.710849:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880899.710850:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:0.0:1788880899.710851:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880899.710852:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880899.710861:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880899.710862:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788880899.710866:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11868) 00000100:00000001:2.0:1788880899.710867:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880899.710868:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000fe5bffa. 00000400:00000200:1.0:1788880899.710868:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880899.710881:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:0.0:1788880899.710884:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 00000100:00000040:2.0:1788880899.710885:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880899.710892:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [507:507:256:4294967295] 192.168.204.111@tcp LPNI seq info [507:507:8:4294967295] 00000100:00000200:0.0:1788880899.710896:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780695040, portal 10 00000100:00000040:3.0:1788880899.710901:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306530767, at 4306530816 00000100:00000001:0.0:1788880899.710908:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880899.710918:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 00000100:00000001:2.0:1788880899.710919:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000010:3.0:1788880899.710922:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005b8bebe2. 00000100:00000001:2.0:1788880899.710923:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880899.710924:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788880899.710936:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:0.0:1788880899.710940:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880899.710942:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880899.710944:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880899.710949:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880899.710956:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000100:00100000:2.0:1788880899.710962:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780695168:192.168.204.111@tcp:400: 00000800:00000200:1.0:1788880899.710975:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880899.710977:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880899.710986:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880899.710988:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:1.0:1788880899.710993:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:2.0:1788880899.710994:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880899.710997:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880899.711000:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788880899.711000:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880899.711007:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ca284ac2. 00000400:00000010:0.0:1788880899.711009:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009fe01c6d. 00000100:00000200:0.0:1788880899.711015:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780695040, offset 0 02000000:00000001:2.0:1788880899.711015:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880899.711018:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880899.711029:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306535887 00000400:00000010:2.0:1788880899.711030:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007593939c. 00000400:00000200:0.0:1788880899.711035:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880899.711041:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000400:00000010:2.0:1788880899.711046:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d208f26. 00000100:00000001:1.0:1788880899.711048:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880899.711053:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880899.711054:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306535887 00000100:00000200:2.0:1788880899.711054:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780695168, portal 10 00000100:00000001:2.0:1788880899.711067:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000010:1.0:1788880899.711069:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000100:00000001:2.0:1788880899.711070:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000400:00000200:0.0:1788880899.711071:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880899.711073:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880899.711081:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880899.711083:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306535887 00000100:00000040:2.0:1788880899.711088:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880899.711093:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000400:00000200:0.0:1788880899.711094:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880899.711103:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880899.711104:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:0.0:1788880899.711112:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [508:508:255:4294967295] 192.168.204.111@tcp LPNI seq info [508:508:7:4294967295] 00000100:00000001:1.0:1788880899.711116:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306535887 00000800:00000200:3.0:1788880899.711126:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:1.0:1788880899.711129:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000800:00000010:3.0:1788880899.711137:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000400:00000200:0.0:1788880899.711144:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880899.711144:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788880899.711148:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880899.711162:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880899.711166:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880899.711186:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880899.711190:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880899.711192:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000050c5542b. 00000100:00000001:1.0:1788880899.711206:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880899.711208:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000100:00000001:1.0:1788880899.711210:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880899.711210:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880899.711211:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:3.0:1788880899.711216:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000040:2.0:1788880899.711222:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880899.711226:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:2.0:1788880899.711229:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000032b7316b. 00000800:00000200:0.0:1788880899.711230:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000050c5542b type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880899.711271:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880899.711274:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880899.711279:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880899.711282:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880899.711286:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880899.711288:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880899.711289:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880899.711301:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 173852 usecs (174 jiffies) 00000100:00000001:0.0:1788880899.711306:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306530991 00000100:00000001:0.0:1788880899.711310:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306535887 00000100:00000001:0.0:1788880899.711312:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306530991 00000100:00000001:0.0:1788880899.711316:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306535887 00000100:00000001:0.0:1788880899.711319:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306530991 00000100:00000200:2.0:1788880899.711321:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780695168, offset 0 00000100:00000001:0.0:1788880899.711323:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306535887 00000100:00000001:3.0:1788880899.711323:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880899.711331:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306530991 00000400:00000200:2.0:1788880899.711341:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:3.0:1788880899.711341:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880915 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880899.711342:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306535887 00000100:00000001:0.0:1788880899.711350:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880899.711354:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880899.711360:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880899.711370:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880899.711372:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880899.711377:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880899.711385:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788880899.711386:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880899.711387:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880899.711396:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880899.711400:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788880899.711401:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880915 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880899.711406:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880899.711408:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [509:509:254:4294967295] 192.168.204.111@tcp LPNI seq info [509:509:6:4294967295] 00000100:00000001:0.0:1788880899.711416:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880899.711421:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880899.711428:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880899.711437:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880899.711447:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880899.711452:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880899.711457:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880899.711459:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000010:2.0:1788880899.711463:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000800:00000200:2.0:1788880899.711470:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:2.0:1788880899.711475:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880899.711477:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880899.711480:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880899.711484:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12642 usecs (13 jiffies) 00000100:00000001:2.0:1788880899.711488:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306530830 00000800:00000200:3.0:1788880899.711488:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880899.711495:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000050c5542b. 00000100:00000001:2.0:1788880899.711497:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306535887 00000100:00000001:2.0:1788880899.711503:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306530830 00000400:00000200:3.0:1788880899.711503:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880899.711510:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306535887 00000400:00000200:3.0:1788880899.711514:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880899.711516:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306530830 00000400:00000200:3.0:1788880899.711520:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009fe01c6d 00000100:00000001:2.0:1788880899.711521:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306535887 00000400:00000010:3.0:1788880899.711525:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009fe01c6d. 00000100:00000001:2.0:1788880899.711527:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306530830 00000100:00000001:3.0:1788880899.711530:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880899.711532:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306535887 00000100:00000200:3.0:1788880899.711536:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880899.711544:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880899.711552:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880899.711559:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880899.711566:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880899.711566:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880899.711578:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788880899.711578:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880899.711582:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880899.711589:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880899.711592:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880899.711595:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880899.711603:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880899.711612:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880899.711615:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880899.711657:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880899.711660:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000400:00000200:3.0:1788880899.711664:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880899.711667:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880899.711670:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000032b7316b 00000400:00000010:3.0:1788880899.711672:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000032b7316b. 00000100:00000001:3.0:1788880899.711675:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880899.711678:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880899.711686:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880899.711689:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880899.711702:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880899.711704:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880899.712355:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880899.712365:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880899.712368:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880899.712369:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880899.712374:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880899.712383:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0f580 00000400:00000200:1.0:1788880899.712389:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x10a1 [1] + 224 00000800:00000001:1.0:1788880899.712394:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880899.712408:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880899.712410:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880899.712412:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880899.712415:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880899.712417:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880899.712423:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880899.712441:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880899.712446:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880899.712448:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880899.712467:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880899.712470:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880899.712475:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880899.712477:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880899.712478:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000020fda082. 00000400:00000200:2.0:1788880899.712483:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000400:00000010:2.0:1788880899.712485:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:2.0:1788880899.712489:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880899.712494:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880899.712503:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880899.712510:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880899.712512:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880899.712514:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880899.712516:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880899.712519:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880899.712521:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880899.712523:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880899.712528:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=127 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880899.712532:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=127 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880899.712534:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880899.712536:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880899.712538:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880899.712540:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880899.712542:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880899.712553:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880899.712560:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880899.712562:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880899.712564:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880899.712566:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880899.712574:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780694912:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880899.712579:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880899.712581:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880899.712583:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780694912/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880899.712587:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880899.712588:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880899.712590:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004ffd965d. 02000000:00000001:2.0:1788880899.712592:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880899.712593:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880899.712595:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880899.712597:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880899.712598:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 02000000:00000001:2.0:1788880899.712600:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880899.712601:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880899.712602:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:2.0:1788880899.712604:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880899.712605:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880899.712606:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880899.712609:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880899.712610:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880899.712612:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880899.712613:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880899.712669:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880899.712679:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880899.712681:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880899.712684:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880899.712693:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880899.712702:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f600 00000400:00000200:2.0:1788880899.712707:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x10a9 [1] + 224 00000800:00000001:2.0:1788880899.712713:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880899.712719:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880899.712722:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880899.712726:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880899.712732:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880899.712740:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880899.712749:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880899.712767:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880899.712773:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880899.712777:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880899.712801:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880899.712805:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880899.712809:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880899.712810:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880899.712812:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000fe5bffa. 00000400:00000200:0.0:1788880899.712815:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000400:00000010:0.0:1788880899.712817:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000001:0.0:1788880899.712819:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880899.712823:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880899.712832:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880899.712839:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880899.712840:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880899.712842:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880899.712844:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880899.712845:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880899.712846:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880899.712848:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880899.712853:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=117 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880899.712856:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=117 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880899.712858:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880899.712859:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880899.712860:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880899.712863:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880899.712866:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880899.712867:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880899.712869:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880899.712870:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880899.712872:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880899.712892:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880899.712893:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880899.712894:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880899.712897:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880899.712902:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780695040:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880899.712905:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880899.712906:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880899.712907:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780695040/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880899.712911:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880899.712912:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880899.712914:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000033d66d82. 02000000:00000001:0.0:1788880899.712916:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880899.712917:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880899.712917:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880899.712919:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880899.712920:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:0.0:1788880899.712921:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880899.712922:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880899.712923:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:0.0:1788880899.712927:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880899.712928:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880899.712929:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880899.712932:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880899.712933:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880899.712935:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880899.712938:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880899.712948:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880899.712955:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880899.712958:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880899.712992:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880899.712999:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880899.713008:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f680 00000400:00000200:3.0:1788880899.713014:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x10b1 [1] + 224 00000800:00000001:3.0:1788880899.713019:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880899.713024:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880899.713026:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880899.713029:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880899.713033:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880899.713036:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880899.713045:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880899.713060:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880899.713065:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880899.713067:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880899.713092:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880899.713094:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880899.713097:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880899.713098:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880899.713099:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007593939c. 00000400:00000200:2.0:1788880899.713102:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d208f26 00000400:00000010:2.0:1788880899.713102:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d208f26. 00000100:00000001:2.0:1788880899.713104:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880899.713108:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880899.713112:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880899.713117:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880899.713117:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880899.713118:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880899.713119:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880899.713120:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880899.713121:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880899.713126:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880899.713129:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=127 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880899.713131:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=61 v=5 (1 0 0 0) 00000100:00000001:2.0:1788880899.713133:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880899.713133:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880899.713134:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788880899.713136:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880899.713140:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880899.713142:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880899.713142:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880899.713143:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880899.713144:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880899.713145:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880899.713149:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880899.713150:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880899.713151:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880899.713152:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880899.713156:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780695168:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880899.713158:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880899.713159:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880899.713160:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780695168/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880915 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880899.713163:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880899.713164:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880899.713165:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ca284ac2. 02000000:00000001:2.0:1788880899.713167:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880899.713168:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880899.713180:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880899.713186:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880899.713187:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:2.0:1788880899.713189:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880899.713189:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880899.713190:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000001:2.0:1788880899.713192:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880899.713193:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880899.713194:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880899.713196:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880899.713197:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880899.713198:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880899.713199:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880899.774381:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880899.774390:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880899.774398:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880899.774400:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880899.774464:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880899.774471:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880899.774476:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880900.798327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880900.798334:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880900.798341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880900.798342:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880900.798370:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880900.798373:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880900.798376:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880901.822353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880901.822364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880901.822369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880901.822370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880901.822414:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880901.822431:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880901.822433:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880902.846338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880902.846344:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880902.846348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880902.846349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880902.846371:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880902.846374:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880902.846376:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880903.870391:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880903.870397:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880903.870400:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880903.870401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880903.870454:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880903.870457:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880903.870459:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880904.830285:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306535887, at 4306535936 00000100:00000040:1.0:1788880904.830289:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306535887, at 4306535936 00000100:00000040:2.0:1788880904.830292:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306535887, at 4306535936 00000100:00000010:0.0:1788880904.830298:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:1.0:1788880904.830300:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000010:2.0:1788880904.830302:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000040:3.0:1788880904.830305:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306535887, at 4306535936 00000100:00000010:3.0:1788880904.830312:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000040:0.0:1788880904.830328:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880904.830334:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880904.830380:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cdd8dec9. 00000020:00000040:0.0:1788880904.830385:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880904.830396:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880904.830399:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880904.830404:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880904.830411:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b7399401. 00000100:00000001:0.0:1788880904.830424:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880904.830434:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880904.830445:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880904.830448:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880904.830454:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cdd8dec9] to pc [ptlrpcd_00_00+0] req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.830473:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880904.830474:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880904.830478:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880904.830481:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c2a263b1. 00000020:00000040:0.0:1788880904.830483:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880904.830483:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880904.830487:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00100000:1.0:1788880904.830488:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 02000000:00000001:0.0:1788880904.830489:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880904.830490:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880904.830491:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880904.830492:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880904.830494:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880904.830497:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000f9f1055. 00000100:00000001:0.0:1788880904.830501:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880904.830510:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880904.830512:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880904.830515:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880904.830520:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880904.830520:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880904.830522:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880904.830523:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880904.830526:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880904.830527:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c2a263b1] to pc [ptlrpcd_00_01+1] req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880904.830532:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.830542:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880904.830546:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880904.830547:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880904.830549:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880904.830549:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880904.830552:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000363de9c9. 02000000:00000001:1.0:1788880904.830553:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000020:00000040:0.0:1788880904.830555:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880904.830557:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880904.830558:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880904.830558:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880904.830562:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880904.830562:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880904.830563:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 02000000:00000001:0.0:1788880904.830565:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880904.830566:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1788880904.830568:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cdd8dec9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780695296:192.168.204.111@tcp:400: 02000000:00000010:0.0:1788880904.830571:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000001c4b7888. 00000100:00000001:1.0:1788880904.830573:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880904.830573:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880904.830574:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880904.830579:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880904.830579:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880904.830581:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880904.830582:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880904.830586:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880904.830587:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880904.830589:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880904.830592:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000010:1.0:1788880904.830592:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000bd87bd8d. 00000100:00000001:2.0:1788880904.830593:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880904.830594:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880904.830598:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880904.830600:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880904.830602:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000363de9c9] to pc [ptlrpcd_00_02+2] req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880904.830605:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008388b815. 00000400:00000010:1.0:1788880904.830612:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000007206624. 00000100:00000200:1.0:1788880904.830619:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780695296, portal 25 00000100:00000001:2.0:1788880904.830620:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880904.830623:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880904.830624:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880904.830627:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849372160 : -126822860179456 : ffff8ca7c0783800) 02000000:00000001:2.0:1788880904.830627:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880904.830631:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880904.830633:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880904.830634:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880920 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.830637:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880904.830641:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880904.830645:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00100000:2.0:1788880904.830645:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780695424:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880904.830647:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880904.830648:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880904.830651:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880904.830653:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880904.830655:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880904.830655:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880904.830657:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11873) 02000000:00000001:2.0:1788880904.830657:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880904.830660:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880904.830661:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000994ad42e. 02000000:00000001:2.0:1788880904.830663:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880904.830663:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880904.830667:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780695296, offset 0 02000000:00000010:2.0:1788880904.830669:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e6ca1161. 02000000:00000001:2.0:1788880904.830673:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880904.830675:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:2.0:1788880904.830678:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007593939c. 00000100:00000001:0.0:1788880904.830680:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880904.830680:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1788880904.830683:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880904.830683:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880904.830690:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d208f26. 02000000:00000001:3.0:1788880904.830692:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:2.0:1788880904.830694:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780695424, portal 10 02000000:00000001:3.0:1788880904.830695:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880904.830699:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880904.830700:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880904.830700:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880904.830701:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364992 : -126822860186624 : ffff8ca7c0781c00) 00000100:00100000:3.0:1788880904.830706:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780695552:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880904.830709:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788880904.830712:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:2.0:1788880904.830712:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880904.830713:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880904.830716:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880904.830719:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880904.830720:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880904.830724:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000006010d39b. 00000100:00000001:2.0:1788880904.830725:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880904.830727:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880904.830729:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880904.830732:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:2.0:1788880904.830733:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880904.830735:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000c54f86e6. 00000400:00000010:2.0:1788880904.830737:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c013629f. 00000400:00000010:3.0:1788880904.830741:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 00000400:00000200:1.0:1788880904.830742:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [510:510:256:4294967295] 192.168.204.111@tcp LPNI seq info [510:510:8:4294967295] 00000100:00000200:2.0:1788880904.830742:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780695424, offset 0 00000100:00000200:3.0:1788880904.830746:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780695552, portal 10 00000400:00000200:2.0:1788880904.830751:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880904.830751:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880904.830754:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849365888 : -126822860185728 : ffff8ca7c0781f80) 00000100:00000040:3.0:1788880904.830759:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880904.830766:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880904.830768:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880904.830774:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880904.830777:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000400:00000200:2.0:1788880904.830780:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880904.830781:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880904.830784:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:3.0:1788880904.830784:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780695552, offset 0 00000400:00000200:2.0:1788880904.830789:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000010:1.0:1788880904.830790:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000400:00000200:2.0:1788880904.830792:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880904.830796:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880904.830797:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880904.830802:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880904.830804:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [511:511:255:4294967295] 192.168.204.111@tcp LPNI seq info [511:511:7:4294967295] 00000800:00000200:1.0:1788880904.830814:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880904.830815:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880904.830820:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000800:00000200:2.0:1788880904.830821:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880904.830826:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880904.830827:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000045330cf1. 00000800:00000200:2.0:1788880904.830834:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880904.830834:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880904.830836:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880904.830841:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880904.830843:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306541007 00000800:00000200:2.0:1788880904.830845:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880904.830846:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:1.0:1788880904.830849:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000800:00000200:2.0:1788880904.830851:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000045330cf1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880904.830851:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880904.830855:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:3.0:1788880904.830855:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [512:512:254:4294967295] 192.168.204.111@tcp LPNI seq info [512:512:6:4294967295] 00000100:00000001:1.0:1788880904.830861:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306541007 00000100:00000001:2.0:1788880904.830865:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788880904.830866:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000400:00000200:3.0:1788880904.830868:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880904.830871:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:2.0:1788880904.830873:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 172711 usecs (173 jiffies) 00000100:00000001:1.0:1788880904.830876:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306541007 00000800:00000200:3.0:1788880904.830877:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880904.830879:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306536109 00000800:00000200:0.0:1788880904.830880:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:1.0:1788880904.830881:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000800:00000010:3.0:1788880904.830884:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:1.0:1788880904.830885:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880904.830885:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306541007 00000100:00000001:2.0:1788880904.830891:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306536109 00000100:00000001:1.0:1788880904.830892:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306541007 00000800:00000010:0.0:1788880904.830893:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000100:00000001:2.0:1788880904.830897:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306541007 00000800:00000200:3.0:1788880904.830897:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:1.0:1788880904.830898:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000400:00000200:0.0:1788880904.830900:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880904.830901:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306536109 00000100:00000001:1.0:1788880904.830902:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880904.830907:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306541007 00000100:00000001:1.0:1788880904.830908:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880904.830912:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880904.830913:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880904.830914:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306536109 00000800:00000200:3.0:1788880904.830915:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880904.830919:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880904.830919:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306541007 00000400:00000200:0.0:1788880904.830920:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000994ad42e 00000800:00000200:3.0:1788880904.830920:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000400:00000010:0.0:1788880904.830924:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000994ad42e. 00000100:00000001:2.0:1788880904.830925:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880904.830926:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880904.830927:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880904.830930:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880904.830931:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880904.830931:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880904.830932:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788880904.830934:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12576 usecs (13 jiffies) 00000100:00000001:1.0:1788880904.830938:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880904.830938:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880904.830939:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306535949 00000100:00000200:0.0:1788880904.830940:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880920 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880904.830943:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880904.830945:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306541007 00000100:00000001:2.0:1788880904.830946:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880904.830951:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306535949 00000100:00000001:3.0:1788880904.830956:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306541007 00000100:00000001:3.0:1788880904.830961:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306535949 00000100:00000001:0.0:1788880904.830962:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880904.830963:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306541007 00000100:00000040:0.0:1788880904.830967:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880920 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880904.830967:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306535949 00000100:00000001:3.0:1788880904.830971:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306541007 00000100:00000001:3.0:1788880904.830976:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880904.830981:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880904.830985:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880904.830987:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880904.830989:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880904.830992:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880904.830994:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880904.830997:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880904.831000:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880904.831001:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880904.831002:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880904.831004:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880904.831008:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880904.831011:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000045330cf1. 00000400:00000200:0.0:1788880904.831014:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880904.831018:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880904.831020:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c013629f 00000400:00000010:0.0:1788880904.831022:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c013629f. 00000100:00000001:0.0:1788880904.831024:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880904.831027:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.831033:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880904.831035:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.831041:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880904.831043:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880904.831080:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880904.831082:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:0.0:1788880904.831085:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880904.831088:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880904.831091:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000010:0.0:1788880904.831093:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:0.0:1788880904.831100:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880904.831103:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.831110:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880904.831112:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.831118:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880904.831119:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880904.832399:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880904.832409:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880904.832412:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880904.832414:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880904.832421:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880904.832432:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0f700 00000400:00000200:2.0:1788880904.832438:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x10b9 [1] + 224 00000800:00000001:2.0:1788880904.832444:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880904.832450:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880904.832452:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880904.832455:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880904.832458:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880904.832462:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880904.832470:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880904.832491:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880904.832501:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880904.832506:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880904.832508:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880904.832511:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880904.832515:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880904.832516:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880904.832519:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880904.832524:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880904.832526:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f780 00000400:00000001:1.0:1788880904.832527:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880904.832530:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008388b815. 00000400:00000200:1.0:1788880904.832534:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000007206624 00000400:00000010:1.0:1788880904.832535:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000007206624. 00000100:00000001:1.0:1788880904.832537:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880904.832542:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880904.832542:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x10c1 [1] + 224 00000800:00000001:2.0:1788880904.832551:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788880904.832554:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880904.832560:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880904.832560:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880904.832561:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880904.832564:0:201286:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:2.0:1788880904.832564:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1788880904.832568:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880904.832571:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:2.0:1788880904.832572:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880904.832573:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880904.832577:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880904.832579:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:1.0:1788880904.832582:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=132 v=5 (5 5 5 5) 00000100:00000200:2.0:1788880904.832584:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880904.832586:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=132 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880904.832590:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880904.832592:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880904.832597:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880904.832600:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880904.832602:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880904.832603:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880904.832610:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880904.832621:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880904.832623:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880904.832624:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880904.832626:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880904.832627:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880904.832632:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:1.0:1788880904.832634:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cdd8dec9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780695296:192.168.204.111@tcp:400: 00000800:00000001:2.0:1788880904.832637:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880904.832639:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880904.832641:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880904.832644:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cdd8dec9 x1875776780695296/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.832645:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880904.832649:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880904.832653:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788880904.832655:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000001:0.0:1788880904.832656:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880904.832659:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007593939c. 02000000:00000001:1.0:1788880904.832659:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880904.832661:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000bd87bd8d. 00000400:00000200:0.0:1788880904.832663:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d208f26 02000000:00000001:1.0:1788880904.832664:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000010:0.0:1788880904.832665:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d208f26. 00000020:00000001:1.0:1788880904.832667:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880904.832669:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000040:1.0:1788880904.832669:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880904.832672:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000200:0.0:1788880904.832674:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880904.832674:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b7399401. 02000000:00000001:1.0:1788880904.832677:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880904.832679:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880904.832681:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cdd8dec9. 00000100:00000001:1.0:1788880904.832685:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000200:0.0:1788880904.832687:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880904.832688:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880904.832691:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880904.832696:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880904.832698:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880904.832698:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880904.832700:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880904.832701:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880904.832703:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880904.832704:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880904.832705:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880904.832707:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880904.832708:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880904.832709:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880904.832713:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=122 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880904.832715:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=122 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880904.832717:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880904.832718:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880904.832719:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880904.832721:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880904.832723:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880904.832724:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880904.832729:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880904.832730:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880904.832732:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.832737:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880904.832738:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880904.832739:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880904.832741:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880904.832746:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780695424:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880904.832748:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880904.832749:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880904.832750:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c2a263b1 x1875776780695424/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.832754:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880904.832756:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880904.832757:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e6ca1161. 02000000:00000001:0.0:1788880904.832760:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880904.832761:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880904.832762:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880904.832764:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880904.832765:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000f9f1055. 02000000:00000001:0.0:1788880904.832767:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880904.832768:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880904.832769:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c2a263b1. 00000100:00000001:0.0:1788880904.832771:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880904.832772:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880904.832773:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880904.832776:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000001:3.0:1788880904.832776:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880904.832777:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880904.832780:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880904.832782:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880904.832782:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880904.832787:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880904.832813:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880904.832818:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880904.832823:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f800 00000400:00000200:3.0:1788880904.832826:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x10c9 [1] + 224 00000800:00000001:3.0:1788880904.832829:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880904.832837:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880904.832838:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880904.832840:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880904.832842:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880904.832845:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880904.832851:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880904.832861:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788880904.832864:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880904.832866:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880904.832877:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880904.832885:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880904.832888:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880904.832889:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880904.832891:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000c54f86e6. 00000400:00000200:0.0:1788880904.832894:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000400:00000010:0.0:1788880904.832895:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:0.0:1788880904.832898:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880904.832901:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880904.832909:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.832915:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880904.832916:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880904.832918:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880904.832919:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880904.832920:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880904.832921:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880904.832923:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880904.832926:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=132 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880904.832929:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=66 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880904.832931:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880904.832932:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880904.832934:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880904.832936:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880904.832942:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880904.832944:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880904.832945:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880904.832947:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880904.832948:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880904.832950:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.832956:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880904.832958:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880904.832963:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880904.832966:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880904.832973:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780695552:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880904.832977:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880904.832979:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880904.832980:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000363de9c9 x1875776780695552/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880920 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880904.832985:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880904.832986:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880904.832987:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000006010d39b. 02000000:00000001:0.0:1788880904.832989:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880904.832989:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880904.832990:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880904.832991:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880904.832992:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000001c4b7888. 02000000:00000001:0.0:1788880904.832993:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880904.832994:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880904.832995:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000363de9c9. 00000100:00000001:0.0:1788880904.832996:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880904.832997:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880904.832998:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880904.833000:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880904.833001:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880904.833003:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880904.833004:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880904.894339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880904.894345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880904.894352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880904.894353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880904.894387:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880904.894391:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880904.894393:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880905.918316:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880905.918325:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880905.918327:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880905.918330:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880905.918334:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880905.918336:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880905.918338:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880905.918340:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880905.918344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880905.918345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880905.918405:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880905.918409:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880905.918412:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880906.942348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880906.942354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880906.942360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880906.942361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880906.942374:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880906.942377:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880906.942380:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880907.966319:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880907.966327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880907.966334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880907.966337:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880907.966339:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880907.966342:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880907.966345:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880907.966347:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880907.966349:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880907.966351:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880907.966395:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880907.966398:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880907.966400:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880908.990331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880908.990336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880908.990340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880908.990341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880908.990371:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880908.990374:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880908.990377:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880909.950306:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306541007, at 4306541056 00000100:00000040:2.0:1788880909.950306:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306541007, at 4306541056 00000100:00000010:2.0:1788880909.950321:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000010:1.0:1788880909.950323:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000040:0.0:1788880909.950324:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306541007, at 4306541056 00000100:00000010:0.0:1788880909.950333:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000040:1.0:1788880909.950353:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880909.950358:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880909.950364:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 00000020:00000040:1.0:1788880909.950369:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880909.950374:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880909.950378:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:3.0:1788880909.950380:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306541007, at 4306541056 02000000:00000001:1.0:1788880909.950384:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880909.950388:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 02000000:00000010:1.0:1788880909.950393:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000749ed09e. 00000100:00000001:1.0:1788880909.950402:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880909.950412:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880909.950422:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880909.950425:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880909.950430:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_00_03+3] req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880909.950454:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880909.950456:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880909.950460:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880909.950464:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000663051e5. 00000020:00000040:1.0:1788880909.950467:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880909.950469:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880909.950471:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880909.950471:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880909.950475:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880909.950477:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:1.0:1788880909.950479:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5b428a0. 00000100:00000001:0.0:1788880909.950481:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880909.950484:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880909.950489:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880909.950490:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880909.950515:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880909.950517:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880909.950517:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880909.950520:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880909.950522:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000663051e5] to pc [ptlrpcd_00_00+0] req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880909.950527:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880909.950530:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880909.950534:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880909.950537:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788880909.950539:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780695680:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880909.950542:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880909.950542:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880909.950544:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880909.950546:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880909.950551:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880909.950551:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880909.950554:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002b387587. 02000000:00000001:0.0:1788880909.950555:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880909.950555:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:1.0:1788880909.950559:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880909.950559:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880909.950560:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880909.950562:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880909.950562:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880909.950566:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008f22742c. 00000100:00000001:1.0:1788880909.950566:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880909.950567:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880909.950569:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880909.950570:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880909.950571:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880909.950575:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880909.950580:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000fe5bffa. 02000000:00000010:1.0:1788880909.950585:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000000deab64f. 00000100:00000001:2.0:1788880909.950585:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880909.950587:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880909.950589:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 02000000:00000001:2.0:1788880909.950591:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880909.950592:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880909.950593:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880909.950597:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780695680, portal 25 02000000:00000001:2.0:1788880909.950597:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880909.950601:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788880909.950601:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780695808:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880909.950602:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880909.950606:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 00000100:00000001:2.0:1788880909.950607:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788880909.950611:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880909.950613:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880909.950617:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880925 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880909.950617:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880909.950618:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880909.950620:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880909.950621:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000044022300. 02000000:00000001:2.0:1788880909.950626:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880909.950628:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002b387587] to pc [ptlrpcd_00_01+1] req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880909.950631:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a55bc54d. 00000400:00000010:2.0:1788880909.950637:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009c31f261. 00000100:00000001:0.0:1788880909.950641:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:2.0:1788880909.950642:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780695808, portal 10 00000100:00000001:2.0:1788880909.950646:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:0.0:1788880909.950649:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880909.950649:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938053248 : -126823771498368 : ffff8ca78a269880) 00000400:00000010:0.0:1788880909.950655:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a08ac001. 00000100:00000001:1.0:1788880909.950657:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880909.950658:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880909.950662:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780695680, offset 0 00000100:00000040:1.0:1788880909.950663:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880909.950669:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880909.950672:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788880909.950674:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880909.950674:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880909.950678:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880909.950678:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880909.950679:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880909.950683:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11878) 00000400:00000010:2.0:1788880909.950684:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000170fdf47. 00000100:00000040:3.0:1788880909.950688:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880909.950689:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780695808, offset 0 00000400:00000200:0.0:1788880909.950696:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880909.950697:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880909.950706:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880909.950707:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880909.950710:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880909.950713:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880909.950714:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880909.950717:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880909.950719:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880909.950721:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880909.950724:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780695936:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880909.950730:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880909.950733:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880909.950734:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880909.950737:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880909.950740:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880909.950741:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [513:513:256:4294967295] 192.168.204.111@tcp LPNI seq info [513:513:8:4294967295] 02000000:00000010:3.0:1788880909.950745:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f8c34a6a. 02000000:00000001:3.0:1788880909.950755:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880909.950756:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:3.0:1788880909.950760:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000063398faa. 00000800:00000200:0.0:1788880909.950765:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880909.950767:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f73021f1. 00000400:00000200:2.0:1788880909.950768:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788880909.950772:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000001c4b7888. 00000100:00000200:3.0:1788880909.950773:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780695936, portal 10 00000400:00000200:2.0:1788880909.950774:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880909.950779:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788880909.950781:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880909.950782:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938057728 : -126823771493888 : ffff8ca78a26aa00) 00000800:00000200:0.0:1788880909.950785:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880909.950789:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880909.950792:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880909.950799:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880909.950802:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788880909.950807:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000001c4b7888 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880909.950808:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:2.0:1788880909.950812:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [514:514:255:4294967295] 192.168.204.111@tcp LPNI seq info [514:514:7:4294967295] 00000100:00000040:3.0:1788880909.950815:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880909.950819:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8336eb2. 00000100:00000200:3.0:1788880909.950823:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780695936, offset 0 00000100:00000001:0.0:1788880909.950824:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880909.950831:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306546127 00000400:00000200:2.0:1788880909.950833:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:3.0:1788880909.950834:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880909.950837:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006bd48548. 00000100:00000001:0.0:1788880909.950844:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:2.0:1788880909.950845:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880909.950850:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306546127 00000400:00000200:3.0:1788880909.950850:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880909.950852:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006aca3ff7. 00000100:00000010:0.0:1788880909.950857:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e64ede4. 00000400:00000200:3.0:1788880909.950857:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880909.950862:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880909.950863:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:3.0:1788880909.950864:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880909.950871:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880909.950872:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306546127 00000100:00000010:0.0:1788880909.950877:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014321469. 00000800:00000200:2.0:1788880909.950880:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880909.950882:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:3.0:1788880909.950885:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788880909.950887:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006aca3ff7 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880909.950888:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306546127 00000800:00000200:1.0:1788880909.950891:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880909.950893:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880909.950893:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [515:515:254:4294967295] 192.168.204.111@tcp LPNI seq info [515:515:6:4294967295] 00000100:00000010:0.0:1788880909.950895:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001a2ff840. 00000800:00000010:1.0:1788880909.950896:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000001c4b7888. 00000100:00000001:0.0:1788880909.950902:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000040:2.0:1788880909.950902:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 171586 usecs (172 jiffies) 00000400:00000200:1.0:1788880909.950904:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880909.950908:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306541228 00000100:00000001:0.0:1788880909.950909:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880909.950915:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880909.950915:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306546127 00000400:00000200:1.0:1788880909.950917:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880909.950918:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880909.950921:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306541228 00000100:00000001:0.0:1788880909.950923:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880909.950928:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306546127 00000100:00000001:0.0:1788880909.950929:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880909.950933:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306541228 00000800:00000200:3.0:1788880909.950933:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880909.950934:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a08ac001 00000100:00000001:2.0:1788880909.950938:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306546127 00000100:00000001:0.0:1788880909.950939:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:1.0:1788880909.950939:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a08ac001. 00000800:00000010:3.0:1788880909.950941:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:2.0:1788880909.950943:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306541228 00000100:00000001:1.0:1788880909.950947:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880909.950947:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306546127 00000100:00000001:0.0:1788880909.950949:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880909.950951:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788880909.950954:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880925 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880909.950956:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880909.950958:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880909.950961:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880909.950962:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880909.950965:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880909.950967:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880909.950970:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880909.950983:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880909.950985:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880909.950988:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:1.0:1788880909.950989:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880925 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880909.950994:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880909.950997:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880909.950998:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880909.951001:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880909.951002:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880909.951006:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880909.951007:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880909.951015:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880909.951019:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880909.951023:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880909.951026:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12511 usecs (13 jiffies) 00000100:00000001:3.0:1788880909.951030:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306541069 00000100:00000001:3.0:1788880909.951032:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306546127 00000100:00000001:3.0:1788880909.951034:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306541069 00000800:00000200:1.0:1788880909.951039:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880909.951044:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006aca3ff7. 00000100:00000001:3.0:1788880909.951044:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306546127 00000100:00000001:3.0:1788880909.951049:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306541069 00000400:00000200:1.0:1788880909.951050:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880909.951052:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306546127 00000100:00000001:3.0:1788880909.951056:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306541069 00000400:00000200:1.0:1788880909.951059:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880909.951060:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306546127 00000100:00000001:3.0:1788880909.951065:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880909.951066:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000170fdf47 00000100:00000001:3.0:1788880909.951067:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880909.951069:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000170fdf47. 00000100:00000001:3.0:1788880909.951071:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880909.951074:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880909.951075:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880909.951079:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:1.0:1788880909.951080:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880909.951082:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880909.951087:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880909.951089:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880909.951091:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880909.951095:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880909.951099:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880909.951101:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880909.951107:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880909.951109:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880909.951121:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880909.951122:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:1.0:1788880909.951126:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880909.951129:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880909.951131:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8336eb2 00000400:00000010:1.0:1788880909.951132:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8336eb2. 00000100:00000001:1.0:1788880909.951134:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880909.951137:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880909.951141:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880909.951143:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880909.951147:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880909.951148:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880909.952433:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880909.952463:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880909.952466:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880909.952467:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880909.952472:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880909.952478:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0f880 00000400:00000200:2.0:1788880909.952483:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x10d1 [1] + 224 00000800:00000001:2.0:1788880909.952487:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880909.952489:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880909.952490:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880909.952492:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880909.952496:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880909.952498:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880909.952504:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880909.952516:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880909.952521:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880909.952524:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880909.952525:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880909.952527:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880909.952530:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880909.952536:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f900 00000400:00000200:2.0:1788880909.952541:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x10d5 [1] + 224 00000800:00000001:2.0:1788880909.952544:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880909.952546:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880909.952547:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880909.952548:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880909.952549:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880909.952551:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:0.0:1788880909.952552:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880909.952554:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880909.952555:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000fe5bffa. 00000400:00000200:0.0:1788880909.952558:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000400:00000010:0.0:1788880909.952559:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000001:0.0:1788880909.952561:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880909.952562:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880909.952564:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880909.952564:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880909.952572:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880909.952572:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880909.952578:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880909.952579:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880909.952581:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880909.952584:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880909.952586:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880909.952588:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880909.952591:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880909.952591:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880909.952595:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880909.952596:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=137 v=5 (5 5 5 5) 00000800:00000001:2.0:1788880909.952598:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880909.952599:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=137 v=5 (1 1 1 1) 00000800:00000001:2.0:1788880909.952599:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880909.952601:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:2.0:1788880909.952601:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880909.952603:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880909.952605:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:2.0:1788880909.952605:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00010000:00000001:0.0:1788880909.952607:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880909.952610:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880909.952610:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0f980 00000400:00000200:2.0:1788880909.952613:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x10e1 [1] + 224 00000100:00000040:0.0:1788880909.952614:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880909.952617:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880909.952621:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:2.0:1788880909.952621:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880909.952623:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:2.0:1788880909.952623:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880909.952625:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880909.952626:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880909.952626:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880909.952630:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880909.952631:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880909.952631:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880909.952635:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:3.0:1788880909.952635:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880909.952639:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00100000:0.0:1788880909.952642:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780695680:192.168.204.111@tcp:400: 00000400:00000010:3.0:1788880909.952642:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a55bc54d. 00000100:00000040:2.0:1788880909.952647:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880909.952648:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009c31f261 00000400:00000010:3.0:1788880909.952651:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009c31f261. 00000100:00000001:0.0:1788880909.952654:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880909.952657:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880909.952658:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:0.0:1788880909.952661:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780695680/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880909.952665:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880909.952666:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880909.952670:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880909.952674:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880909.952677:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880909.952681:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880909.952685:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008f22742c. 00000100:00000200:3.0:1788880909.952687:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880909.952690:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880909.952694:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880909.952694:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880909.952697:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:0.0:1788880909.952698:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880909.952700:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000001:1.0:1788880909.952702:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000001:0.0:1788880909.952703:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880909.952704:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880909.952705:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000010:0.0:1788880909.952707:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000749ed09e. 00000100:00000001:3.0:1788880909.952708:0:201285:0:(client.c:1580:after_reply()) Process entered 00000400:00000010:1.0:1788880909.952709:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000063398faa. 02000000:00000001:0.0:1788880909.952712:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880909.952712:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000200:1.0:1788880909.952713:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f73021f1 00000100:00000001:3.0:1788880909.952716:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:0.0:1788880909.952717:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000010:1.0:1788880909.952717:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f73021f1. 00000100:00000001:3.0:1788880909.952719:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788880909.952721:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000001:1.0:1788880909.952722:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880909.952726:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:3.0:1788880909.952729:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880909.952730:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880909.952734:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880909.952734:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880909.952736:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=127 v=5 (5 5 5 5) 00000100:00000001:0.0:1788880909.952737:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880909.952740:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880909.952746:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:3.0:1788880909.952746:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=127 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880909.952749:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880909.952751:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880909.952752:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880909.952755:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880909.952759:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880909.952763:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880909.952766:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880909.952768:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:1.0:1788880909.952770:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880909.952771:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880909.952775:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880909.952775:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000001:1.0:1788880909.952779:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880909.952779:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880909.952783:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880909.952785:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880909.952786:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880909.952790:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880909.952796:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=137 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880909.952800:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00001000:1.0:1788880909.952801:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=71 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880909.952804:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880909.952805:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880909.952807:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880909.952808:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880909.952811:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:3.0:1788880909.952814:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00080000:1.0:1788880909.952817:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880909.952835:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880909.952837:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780695808:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880909.952838:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880909.952840:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880909.952844:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880909.952846:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880909.952846:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880909.952850:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880909.952850:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880909.952856:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000663051e5 x1875776780695808/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880909.952857:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880909.952862:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880909.952864:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880909.952869:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880909.952872:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880909.952875:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00100000:1.0:1788880909.952879:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780695936:192.168.204.111@tcp:400: 02000000:00000010:3.0:1788880909.952879:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000044022300. 02000000:00000001:3.0:1788880909.952884:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880909.952885:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880909.952887:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880909.952888:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000020:00000040:3.0:1788880909.952889:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000040:1.0:1788880909.952891:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002b387587 x1875776780695936/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880925 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880909.952894:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880909.952897:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5b428a0. 00000100:00000001:1.0:1788880909.952902:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880909.952903:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880909.952905:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880909.952906:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f8c34a6a. 02000000:00000001:1.0:1788880909.952908:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880909.952911:0:201286:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000001:3.0:1788880909.952911:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000020:00000040:1.0:1788880909.952915:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000010:3.0:1788880909.952915:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000663051e5. 00000020:00000001:1.0:1788880909.952919:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880909.952921:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000000deab64f. 00000100:00000001:3.0:1788880909.952922:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:1.0:1788880909.952924:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880909.952925:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880909.952926:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880909.952930:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002b387587. 00000100:00000001:3.0:1788880909.952930:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880909.952936:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880909.952937:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880909.952938:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880909.952938:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880909.952941:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880909.952943:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880909.952945:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880909.952945:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880909.952949:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880909.952950:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880909.952953:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880910.014346:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880910.014353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880910.014359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880910.014360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880910.014414:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880910.014418:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880910.014421:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880911.038355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880911.038362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880911.038367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880911.038368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880911.038391:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880911.038394:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880911.038397:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880912.062348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880912.062355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880912.062360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880912.062362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880912.062445:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880912.062449:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880912.062452:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880913.086340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880913.086346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880913.086352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880913.086353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880913.086387:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880913.086390:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880913.086393:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880914.110333:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880914.110346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880914.110351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880914.110352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880914.110399:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880914.110404:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880914.110407:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880915.070280:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306546127, at 4306546176 00000100:00000040:3.0:1788880915.070288:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306546127, at 4306546176 00000100:00000010:2.0:1788880915.070296:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014321469. 00000100:00000010:3.0:1788880915.070298:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001a2ff840. 00000100:00000040:0.0:1788880915.070306:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306546127, at 4306546176 00000100:00000010:0.0:1788880915.070315:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006bd48548. 00000100:00000040:3.0:1788880915.070315:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788880915.070316:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306546127, at 4306546176 00000100:00000001:3.0:1788880915.070325:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880915.070326:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e64ede4. 00000100:00000010:3.0:1788880915.070330:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788880915.070335:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880915.070339:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880915.070342:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880915.070346:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880915.070353:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:3.0:1788880915.070358:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880915.070367:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880915.070376:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880915.070378:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880915.070383:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_02+2] req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880915.070406:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880915.070408:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880915.070411:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880915.070414:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000020:00000040:3.0:1788880915.070416:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880915.070419:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880915.070420:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880915.070421:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788880915.070423:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880915.070424:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880915.070426:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880915.070431:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880915.070433:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:3.0:1788880915.070438:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000040:0.0:1788880915.070439:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880915.070444:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880915.070450:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880915.070452:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880915.070453:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880915.070455:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880915.070458:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880915.070462:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880915.070468:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780696064:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880915.070469:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880915.070472:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880915.070473:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880915.070479:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880915.070479:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_03+3] req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880915.070482:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880915.070486:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880915.070491:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e0fe5497. 02000000:00000001:0.0:1788880915.070494:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880915.070500:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000fe5bffa. 00000100:00000001:3.0:1788880915.070500:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880915.070505:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880915.070507:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788880915.070508:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 00000100:00000001:3.0:1788880915.070509:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880915.070513:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000100:00000001:2.0:1788880915.070515:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880915.070518:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:3.0:1788880915.070518:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880915.070522:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000200:0.0:1788880915.070524:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780696064, portal 25 00000100:00000040:2.0:1788880915.070524:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880915.070525:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880915.070528:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880915.070534:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000010:3.0:1788880915.070535:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:0.0:1788880915.070538:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000001:2.0:1788880915.070540:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880915.070541:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880915.070544:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880915.070549:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880931 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880915.070549:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788880915.070550:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880915.070553:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880915.070559:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880915.070567:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780696192:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880915.070577:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880915.070579:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880915.070579:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788880915.070582:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880915.070583:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880915.070584:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880915.070585:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880915.070588:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005c796512. 02000000:00000001:2.0:1788880915.070590:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880915.070592:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_00+0] req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880915.070594:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780696064, offset 0 02000000:00000010:2.0:1788880915.070596:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000c21a3ff8. 02000000:00000001:2.0:1788880915.070602:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880915.070605:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880915.070610:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006bdd9925. 00000400:00000010:2.0:1788880915.070617:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c56cc487. 00000400:00000200:0.0:1788880915.070620:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:2.0:1788880915.070622:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780696192, portal 10 00000400:00000200:0.0:1788880915.070625:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880915.070625:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880915.070628:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000400:00000200:0.0:1788880915.070631:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880915.070632:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880915.070636:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:0.0:1788880915.070637:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1788880915.070637:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880915.070646:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880915.070646:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788880915.070649:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788880915.070650:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880915.070650:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880915.070654:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880915.070655:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880915.070655:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11883) 00000400:00000200:0.0:1788880915.070659:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [516:516:256:4294967295] 192.168.204.111@tcp LPNI seq info [516:516:8:4294967295] 00000400:00000010:2.0:1788880915.070659:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000793778c8. 00000100:00000040:1.0:1788880915.070662:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880915.070663:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780696192, offset 0 00000400:00000200:2.0:1788880915.070673:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880915.070674:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880915.070679:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880915.070681:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880915.070685:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880915.070688:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:1.0:1788880915.070688:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880915.070692:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880915.070695:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880915.070698:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788880915.070699:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780696320:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880915.070699:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788880915.070706:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000f9f1055. 00000100:00000001:1.0:1788880915.070706:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788880915.070707:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880915.070710:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1788880915.070713:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788880915.070714:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880915.070718:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:1.0:1788880915.070719:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880915.070719:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880915.070724:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000010:1.0:1788880915.070725:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009e1ad299. 00000800:00000200:0.0:1788880915.070730:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:1.0:1788880915.070730:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880915.070736:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000f9f1055 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880915.070736:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [517:517:255:4294967295] 192.168.204.111@tcp LPNI seq info [517:517:7:4294967295] 00000400:00000010:1.0:1788880915.070737:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008388b815. 00000400:00000010:1.0:1788880915.070742:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000007206624. 00000100:00000200:1.0:1788880915.070746:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780696320, portal 10 00000100:00000001:1.0:1788880915.070749:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788880915.070750:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880915.070751:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880915.070752:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000100:00000001:0.0:1788880915.070757:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306551247 00000100:00000040:1.0:1788880915.070761:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880915.070761:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880915.070763:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006bd48548. 00000800:00000010:2.0:1788880915.070766:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003139f6ba. 00000100:00000001:0.0:1788880915.070770:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:2.0:1788880915.070776:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880915.070777:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306551247 00000100:00000001:1.0:1788880915.070777:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:0.0:1788880915.070782:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000085563e6f. 00000100:00000040:1.0:1788880915.070785:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880915.070787:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880915.070788:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000010:1.0:1788880915.070790:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009822ffbc. 00000100:00000001:0.0:1788880915.070792:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306551247 00000800:00000200:2.0:1788880915.070793:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003139f6ba type 1, nob 320 niov 1 nkiov 1 00000100:00000200:1.0:1788880915.070795:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780696320, offset 0 00000100:00000010:0.0:1788880915.070796:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009f22a9a0. 00000100:00000001:0.0:1788880915.070801:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880915.070801:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880915.070805:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306551247 00000400:00000200:1.0:1788880915.070808:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880915.070808:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 170475 usecs (171 jiffies) 00000100:00000010:0.0:1788880915.070810:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000bc13d1e4. 00000100:00000001:0.0:1788880915.070815:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880915.070815:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306546347 00000100:00000001:0.0:1788880915.070819:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880915.070820:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880915.070821:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306551247 00000100:00000001:0.0:1788880915.070822:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880915.070828:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306546347 00000800:00000200:3.0:1788880915.070828:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880915.070829:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880915.070830:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880915.070834:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880915.070835:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306551247 00000800:00000010:3.0:1788880915.070836:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000f9f1055. 00000400:00000200:1.0:1788880915.070838:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880915.070840:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880915.070841:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306546347 00000100:00000001:0.0:1788880915.070844:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:1.0:1788880915.070844:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880915.070846:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306551247 00000100:00000001:0.0:1788880915.070848:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880915.070849:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880915.070851:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880915.070851:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306546347 00000400:00000200:1.0:1788880915.070852:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880915.070855:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880915.070857:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306551247 00000100:00000001:0.0:1788880915.070859:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880915.070861:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880915.070864:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [518:518:254:4294967295] 192.168.204.111@tcp LPNI seq info [518:518:6:4294967295] 00000100:00000001:2.0:1788880915.070864:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880915.070868:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880915.070873:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880915.070884:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880915.070889:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880915.070891:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880915.070895:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880915.070898:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880915.070901:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880915.070901:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880915.070905:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880915.070910:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880915.070913:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000deab64f. 00000800:00000200:1.0:1788880915.070916:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788880915.070920:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880915.070922:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000deab64f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880915.070924:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880915.070926:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12446 usecs (13 jiffies) 00000100:00000001:1.0:1788880915.070928:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306546189 00000100:00000001:1.0:1788880915.070929:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306551247 00000100:00000001:1.0:1788880915.070931:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306546189 00000100:00000001:1.0:1788880915.070932:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306551247 00000100:00000001:1.0:1788880915.070933:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306546189 00000100:00000001:1.0:1788880915.070934:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306551247 00000100:00000001:1.0:1788880915.070935:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306546189 00000100:00000001:1.0:1788880915.070936:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306551247 00000100:00000001:1.0:1788880915.070937:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880915.070938:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880915.070940:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880915.070941:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880915.070942:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005c796512 00000100:00000001:1.0:1788880915.070944:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880915.070947:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000010:3.0:1788880915.070947:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005c796512. 00000100:00000001:1.0:1788880915.070952:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880915.070953:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880915.070954:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880915.070954:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880915.070958:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880915.070961:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880931 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880915.070970:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880915.070972:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880931 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880915.070978:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880915.070980:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880915.070994:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880915.070996:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003139f6ba. 00000400:00000200:3.0:1788880915.070999:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880915.071002:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880915.071004:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000793778c8 00000400:00000010:3.0:1788880915.071005:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000793778c8. 00000100:00000001:3.0:1788880915.071007:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880915.071010:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880915.071015:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880915.071017:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880915.071022:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880915.071023:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880915.071048:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880915.071050:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000deab64f. 00000400:00000200:3.0:1788880915.071053:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880915.071056:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880915.071064:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009822ffbc 00000400:00000010:3.0:1788880915.071065:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009822ffbc. 00000100:00000001:3.0:1788880915.071067:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880915.071070:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880915.071076:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880915.071078:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880915.071083:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880915.071084:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880915.072694:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880915.072700:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880915.072702:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880915.072703:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880915.072710:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880915.072720:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0fa00 00000400:00000200:1.0:1788880915.072726:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x10e9 [1] + 224 00000800:00000001:1.0:1788880915.072732:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880915.072735:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880915.072737:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880915.072741:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880915.072746:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880915.072750:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880915.072759:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880915.072775:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880915.072781:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880915.072784:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880915.072786:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880915.072787:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880915.072791:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880915.072798:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0fa80 00000400:00000200:1.0:1788880915.072803:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x10f1 [1] + 224 00000800:00000001:1.0:1788880915.072807:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880915.072810:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880915.072813:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:1.0:1788880915.072813:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880915.072817:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880915.072820:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880915.072826:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:0.0:1788880915.072827:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:1.0:1788880915.072830:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880915.072831:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880915.072834:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000fe5bffa. 00000400:00000200:0.0:1788880915.072839:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000400:00000010:0.0:1788880915.072842:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000040:1.0:1788880915.072843:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880915.072846:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880915.072853:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880915.072857:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880915.072863:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788880915.072866:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880915.072866:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880915.072888:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880915.072888:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880915.072892:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880915.072892:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880915.072897:0:201287:0:(client.c:1580:after_reply()) Process entered 00000400:00000001:2.0:1788880915.072900:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:0.0:1788880915.072901:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000001:2.0:1788880915.072902:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880915.072905:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006bdd9925. 00000100:00000001:0.0:1788880915.072906:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:2.0:1788880915.072909:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c56cc487 00000400:00000010:2.0:1788880915.072911:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c56cc487. 00000100:00000001:2.0:1788880915.072913:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880915.072918:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880915.072918:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880915.072924:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880915.072929:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880915.072933:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=143 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880915.072934:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880915.072937:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788880915.072937:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788880915.072940:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=143 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880915.072940:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880915.072943:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880915.072945:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880915.072946:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:3.0:1788880915.072946:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880915.072947:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880915.072950:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880915.072951:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788880915.072952:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880915.072957:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880915.072958:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00001000:2.0:1788880915.072958:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=133 v=5 (5 5 5 5) 00010000:00000001:0.0:1788880915.072962:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880915.072963:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=133 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880915.072967:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880915.072968:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880915.072969:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880915.072972:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880915.072974:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880915.072976:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788880915.072977:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880915.072979:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880915.072980:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880915.072984:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880915.072984:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880915.073004:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788880915.073007:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880915.073009:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880915.073013:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880915.073014:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880915.073018:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880915.073019:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:3.0:1788880915.073022:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0fb00 00000100:00000001:0.0:1788880915.073023:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880915.073032:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x10f9 [1] + 224 00000100:00000040:0.0:1788880915.073033:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788880915.073042:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788880915.073047:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880915.073051:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00100000:2.0:1788880915.073057:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780696192:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788880915.073059:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00100000:0.0:1788880915.073066:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780696064:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880915.073067:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880915.073067:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880915.073072:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:3.0:1788880915.073075:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880915.073077:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880915.073078:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780696192/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880915.073090:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880915.073096:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:3.0:1788880915.073096:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880915.073098:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780696064/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880915.073100:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880915.073102:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000c21a3ff8. 02000000:00000001:2.0:1788880915.073107:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880915.073109:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880915.073111:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880915.073117:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880915.073119:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 00000100:00000001:0.0:1788880915.073122:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880915.073123:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880915.073123:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788880915.073126:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:2.0:1788880915.073127:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000800:00000001:3.0:1788880915.073129:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000010:0.0:1788880915.073131:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e0fe5497. 00000100:00000010:2.0:1788880915.073131:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000800:00000001:3.0:1788880915.073133:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880915.073135:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880915.073137:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880915.073138:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788880915.073139:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:0.0:1788880915.073142:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:2.0:1788880915.073144:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000020:00000040:0.0:1788880915.073145:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880915.073146:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880915.073147:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:0.0:1788880915.073151:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880915.073152:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880915.073152:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880915.073156:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 00000100:00000001:2.0:1788880915.073156:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880915.073157:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880915.073160:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880915.073162:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008388b815. 02000000:00000001:0.0:1788880915.073163:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000200:1.0:1788880915.073165:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000007206624 02000000:00000001:0.0:1788880915.073166:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000010:1.0:1788880915.073169:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000007206624. 00000100:00000010:0.0:1788880915.073170:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:1.0:1788880915.073173:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880915.073176:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000200:1.0:1788880915.073178:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880915.073179:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880915.073183:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880915.073190:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:1.0:1788880915.073190:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880915.073193:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880915.073198:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880915.073201:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880915.073202:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880915.073203:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880915.073206:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880915.073208:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880915.073209:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880915.073209:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880915.073211:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880915.073213:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=143 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880915.073216:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=77 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880915.073217:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880915.073218:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880915.073220:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880915.073222:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880915.073226:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880915.073227:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880915.073228:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880915.073229:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880915.073230:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880915.073248:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880915.073252:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880915.073254:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880915.073255:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880915.073258:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880915.073266:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780696320:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880915.073270:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880915.073271:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880915.073272:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780696320/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880931 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880915.073276:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880915.073277:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880915.073278:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009e1ad299. 02000000:00000001:1.0:1788880915.073280:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880915.073281:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880915.073282:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880915.073283:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880915.073284:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 02000000:00000001:1.0:1788880915.073287:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880915.073288:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880915.073289:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:1.0:1788880915.073292:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880915.073292:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880915.073294:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880915.073297:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880915.073298:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880915.073300:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880915.073301:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880915.134302:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880915.134308:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880915.134312:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880915.134313:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880915.134353:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880915.134357:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880915.134361:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880916.158338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880916.158347:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880916.158350:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880916.158353:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880916.158358:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880916.158360:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880916.158362:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880916.158365:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880916.158368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880916.158369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880916.158433:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880916.158437:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880916.158440:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880917.182294:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880917.182300:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880917.182306:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880917.182307:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880917.182367:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880917.182371:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880917.182374:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880918.206329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880918.206339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880918.206347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880918.206351:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880918.206354:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880918.206358:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880918.206362:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880918.206365:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880918.206368:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880918.206370:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880918.206435:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880918.206441:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880918.206444:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880919.230338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880919.230344:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880919.230348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880919.230349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880919.230405:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880919.230408:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880919.230412:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880920.190284:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306551247, at 4306551296 00000100:00000040:1.0:1788880920.190290:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306551247, at 4306551296 00000100:00000010:0.0:1788880920.190301:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006bd48548. 00000100:00000010:1.0:1788880920.190303:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000085563e6f. 00000100:00000040:2.0:1788880920.190306:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306551247, at 4306551296 00000100:00000010:2.0:1788880920.190314:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009f22a9a0. 00000100:00000040:1.0:1788880920.190316:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880920.190321:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880920.190326:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002b387587. 00000020:00000040:1.0:1788880920.190329:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000040:3.0:1788880920.190330:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306551247, at 4306551296 00000100:00000001:1.0:1788880920.190332:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880920.190336:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000010:3.0:1788880920.190339:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000bc13d1e4. 02000000:00000001:1.0:1788880920.190340:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880920.190348:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:1.0:1788880920.190354:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880920.190362:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880920.190370:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880920.190373:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880920.190378:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002b387587] to pc [ptlrpcd_00_01+1] req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880920.190397:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880920.190399:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880920.190401:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880920.190403:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e2be23b2. 00000020:00000040:1.0:1788880920.190404:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880920.190406:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880920.190407:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880920.190408:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880920.190410:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000252f4482. 00000100:00000001:1.0:1788880920.190412:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880920.190415:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880920.190420:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880920.190420:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880920.190421:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880920.190426:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e2be23b2] to pc [ptlrpcd_00_02+2] req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880920.190426:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880920.190426:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880920.190431:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880920.190431:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880920.190440:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880920.190443:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880920.190446:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880920.190450:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880920.190453:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00100000:2.0:1788880920.190454:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000010:1.0:1788880920.190456:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002980f9ff. 00000100:00000001:2.0:1788880920.190458:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880920.190460:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880920.190461:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:1.0:1788880920.190461:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880920.190465:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880920.190465:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880920.190466:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880920.190468:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880920.190471:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880920.190473:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000010:1.0:1788880920.190474:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:2.0:1788880920.190474:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880920.190478:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880920.190479:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880920.190479:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880920.190482:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880920.190485:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880920.190485:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880920.190485:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880920.190488:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00100000:3.0:1788880920.190493:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780696448:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880920.190495:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880920.190503:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880920.190510:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880920.190514:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880920.190519:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880920.190521:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880920.190524:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880920.190524:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880920.190525:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008545eb31. 00000100:00000001:1.0:1788880920.190527:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880920.190529:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880920.190530:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880920.190533:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002980f9ff] to pc [ptlrpcd_00_03+3] req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880920.190534:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788880920.190538:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a31f011. 02000000:00000001:2.0:1788880920.190540:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880920.190547:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007fe20b6f. 00000100:00100000:2.0:1788880920.190548:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780696576:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880920.190553:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880920.190554:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880920.190556:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:3.0:1788880920.190556:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780696448, portal 25 02000000:00000001:2.0:1788880920.190558:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880920.190562:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880920.190563:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880920.190563:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:2.0:1788880920.190568:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880920.190568:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938057728 : -126823771493888 : ffff8ca78a26aa00) 00000100:00000040:1.0:1788880920.190569:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11888) 02000000:00000010:2.0:1788880920.190573:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b4722d49. 02000000:00000001:2.0:1788880920.190577:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880920.190578:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880920.190578:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880936 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880920.190581:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000400:00000010:2.0:1788880920.190583:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006bdd9925. 00000100:00000001:0.0:1788880920.190586:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880920.190591:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c56cc487. 00000100:00000001:0.0:1788880920.190593:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880920.190595:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880920.190598:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780696576, portal 10 00000100:00000001:3.0:1788880920.190598:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880920.190599:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880920.190603:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:3.0:1788880920.190604:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880920.190606:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880920.190607:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055936 : -126823771495680 : ffff8ca78a26a300) 00000100:00000001:0.0:1788880920.190609:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:3.0:1788880920.190609:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cd783db0. 00000100:00000040:0.0:1788880920.190615:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880920.190615:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780696448, offset 0 00000100:00000040:2.0:1788880920.190617:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880920.190636:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:3.0:1788880920.190636:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880920.190639:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880920.190642:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880920.190643:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880920.190646:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880920.190649:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880920.190650:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00100000:0.0:1788880920.190653:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780696704:192.168.204.111@tcp:400: 00000400:00000010:2.0:1788880920.190655:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000841b943f. 00000400:00000200:3.0:1788880920.190655:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880920.190659:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000200:2.0:1788880920.190662:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780696576, offset 0 02000000:00000001:0.0:1788880920.190663:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:3.0:1788880920.190665:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788880920.190667:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880920.190670:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788880920.190673:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880920.190673:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000010:0.0:1788880920.190674:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000016c67947. 02000000:00000001:0.0:1788880920.190678:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880920.190681:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000fe5bffa. 00000400:00000200:3.0:1788880920.190681:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:0.0:1788880920.190685:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 00000100:00000200:0.0:1788880920.190687:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780696704, portal 10 00000100:00000001:0.0:1788880920.190688:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880920.190689:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938061312 : -126823771490304 : ffff8ca78a26b800) 00000100:00000040:0.0:1788880920.190694:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880920.190700:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788880920.190702:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:0.0:1788880920.190704:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880920.190708:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e0ace8ca. 00000100:00000200:0.0:1788880920.190710:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780696704, offset 0 00000400:00000200:3.0:1788880920.190713:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [519:519:256:4294967295] 192.168.204.111@tcp LPNI seq info [519:519:8:4294967295] 00000400:00000200:0.0:1788880920.190717:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880920.190730:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880920.190737:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880920.190741:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:3.0:1788880920.190744:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000400:00000200:2.0:1788880920.190745:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:3.0:1788880920.190755:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880920.190758:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880920.190763:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788880920.190767:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880920.190769:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788880920.190773:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880920.190775:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [520:520:255:4294967295] 192.168.204.111@tcp LPNI seq info [520:520:7:4294967295] 00000100:00000001:3.0:1788880920.190799:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880920.190800:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880920.190806:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306556367 00000800:00000200:2.0:1788880920.190810:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880920.190813:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:3.0:1788880920.190814:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c7c25dfc. 00000800:00000010:2.0:1788880920.190816:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000781b5970. 00000400:00000200:0.0:1788880920.190820:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880920.190821:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788880920.190825:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788880920.190828:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880920.190830:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880920.190835:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306556367 00000400:00000200:0.0:1788880920.190836:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788880920.190840:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:3.0:1788880920.190842:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002aba003e. 00000800:00000200:2.0:1788880920.190845:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000781b5970 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880920.190848:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880920.190852:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [521:521:254:4294967295] 192.168.204.111@tcp LPNI seq info [521:521:6:4294967295] 00000100:00000001:2.0:1788880920.190852:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880920.190854:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306556367 00000100:00000040:2.0:1788880920.190860:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 169381 usecs (170 jiffies) 00000100:00000010:3.0:1788880920.190862:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b0b8cfdf. 00000100:00000001:2.0:1788880920.190865:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306551466 00000100:00000001:3.0:1788880920.190869:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880920.190870:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306556367 00000400:00000200:0.0:1788880920.190874:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880920.190876:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306556367 00000800:00000200:1.0:1788880920.190878:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:3.0:1788880920.190881:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d6fe56. 00000800:00000200:0.0:1788880920.190883:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880920.190884:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:2.0:1788880920.190884:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306551466 00000100:00000001:3.0:1788880920.190888:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000010:0.0:1788880920.190889:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000f9f1055. 00000400:00000200:1.0:1788880920.190890:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880920.190894:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306556367 00000100:00000001:3.0:1788880920.190896:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880920.190899:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880920.190901:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306551466 00000400:00000200:1.0:1788880920.190903:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880920.190903:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880920.190906:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306556367 00000100:00000001:3.0:1788880920.190910:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880920.190912:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880920.190912:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cd783db0 00000100:00000001:2.0:1788880920.190912:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306551466 00000400:00000010:1.0:1788880920.190916:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cd783db0. 00000100:00000001:2.0:1788880920.190918:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306556367 00000800:00000200:0.0:1788880920.190919:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000f9f1055 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880920.190922:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880920.190922:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880920.190923:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880920.190925:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880920.190929:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880936 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880920.190929:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880920.190930:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788880920.190931:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12385 usecs (13 jiffies) 00000100:00000001:2.0:1788880920.190935:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880920.190935:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880920.190938:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306551309 00000100:00000001:2.0:1788880920.190942:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880920.190942:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880920.190945:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306556367 00000100:00000001:3.0:1788880920.190946:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880920.190948:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880920.190951:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306551309 00000100:00000001:1.0:1788880920.190951:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880920.190951:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880920.190952:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880920.190956:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880936 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880920.190956:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880920.190957:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306556367 00000100:00000001:2.0:1788880920.190960:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880920.190962:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306551309 00000100:00000001:0.0:1788880920.190967:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306556367 00000100:00000001:0.0:1788880920.190970:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306551309 00000100:00000001:0.0:1788880920.190972:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306556367 00000100:00000001:1.0:1788880920.190972:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880920.190975:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880920.190976:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880920.190979:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880920.190981:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880920.190983:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880920.190988:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880920.190990:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000781b5970. 00000400:00000200:1.0:1788880920.190992:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880920.190995:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880920.190996:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000841b943f 00000400:00000010:1.0:1788880920.190998:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000841b943f. 00000100:00000001:1.0:1788880920.191000:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880920.191002:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880920.191007:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880920.191009:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880920.191012:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880920.191013:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880920.191033:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880920.191033:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880920.191037:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000f9f1055. 00000400:00000200:1.0:1788880920.191040:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880920.191042:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880920.191043:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e0ace8ca 00000400:00000010:1.0:1788880920.191044:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e0ace8ca. 00000100:00000001:1.0:1788880920.191046:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880920.191047:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880920.191052:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880920.191053:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880920.191057:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880920.191058:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880920.192625:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880920.192634:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880920.192636:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880920.192638:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880920.192643:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880920.192650:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0fc00 00000400:00000200:1.0:1788880920.192656:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1105 [1] + 224 00000800:00000001:1.0:1788880920.192661:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880920.192663:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880920.192665:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880920.192667:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880920.192670:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880920.192673:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880920.192680:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880920.192691:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880920.192695:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880920.192697:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880920.192698:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880920.192700:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880920.192702:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880920.192705:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0fb80 00000400:00000200:1.0:1788880920.192709:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1101 [1] + 224 00000800:00000001:1.0:1788880920.192712:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880920.192715:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880920.192716:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880920.192720:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880920.192723:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880920.192723:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:1.0:1788880920.192730:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880920.192732:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880920.192738:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880920.192741:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:1.0:1788880920.192743:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788880920.192744:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006bdd9925. 00000400:00000200:2.0:1788880920.192750:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c56cc487 00000400:00000010:2.0:1788880920.192753:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c56cc487. 00000100:00000001:2.0:1788880920.192759:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880920.192764:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880920.192765:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880920.192770:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880920.192775:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880920.192777:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:2.0:1788880920.192788:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880920.192793:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880920.192796:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880920.192798:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880920.192800:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880920.192802:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880920.192802:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880920.192803:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880920.192807:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000001:3.0:1788880920.192807:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880920.192809:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000001:3.0:1788880920.192810:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:1.0:1788880920.192813:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0fc80 00000100:00000001:2.0:1788880920.192814:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880920.192814:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a31f011. 02000000:00000001:2.0:1788880920.192820:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880920.192821:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007fe20b6f 00000400:00000010:3.0:1788880920.192823:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007fe20b6f. 00000100:00000001:3.0:1788880920.192826:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788880920.192828:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1111 [1] + 224 00000100:00001000:2.0:1788880920.192830:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=138 v=5 (5 5 5 5) 00000100:00000200:3.0:1788880920.192830:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880920.192833:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:2.0:1788880920.192839:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=138 v=5 (1 1 1 1) 00000800:00000001:1.0:1788880920.192841:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880920.192844:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880920.192846:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880920.192847:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880920.192850:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880920.192855:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00010000:00000001:2.0:1788880920.192856:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880920.192861:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880920.192862:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:2.0:1788880920.192862:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880920.192863:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880920.192867:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880920.192868:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788880920.192869:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:3.0:1788880920.192871:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:2.0:1788880920.192872:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880920.192873:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880920.192876:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880920.192877:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880920.192880:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880920.192881:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880920.192884:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880920.192885:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=148 v=5 (5 5 5 5) 00000100:00000040:2.0:1788880920.192886:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880920.192892:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=148 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880920.192900:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880920.192902:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880920.192906:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880920.192909:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880920.192915:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880920.192915:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880920.192920:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880920.192921:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880920.192923:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880920.192923:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880920.192927:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:2.0:1788880920.192930:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880920.192932:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880920.192933:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880920.192942:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880920.192943:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880920.192948:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880920.192958:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880920.192962:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00100000:2.0:1788880920.192964:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780696576:192.168.204.111@tcp:400: 00000400:00000001:0.0:1788880920.192968:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00100000:3.0:1788880920.192968:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780696448:192.168.204.111@tcp:400: 00000400:00000001:0.0:1788880920.192972:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788880920.192973:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880920.192975:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000fe5bffa. 00000100:00000001:3.0:1788880920.192975:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880920.192977:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1788880920.192980:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000100:00000001:3.0:1788880920.192980:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000010:0.0:1788880920.192984:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000040:2.0:1788880920.192984:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e2be23b2 x1875776780696576/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880920.192987:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002b387587 x1875776780696448/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880920.192989:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880920.192999:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880920.193011:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880920.193012:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880920.193014:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:3.0:1788880920.193016:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880920.193017:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b4722d49. 02000000:00000010:3.0:1788880920.193019:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008545eb31. 00000100:00000200:0.0:1788880920.193022:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880920.193022:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:3.0:1788880920.193023:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880920.193026:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000001:3.0:1788880920.193027:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880920.193029:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000040:3.0:1788880920.193030:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880920.193035:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880920.193036:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:3.0:1788880920.193036:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880920.193040:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:2.0:1788880920.193040:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000252f4482. 02000000:00000010:3.0:1788880920.193041:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 00000100:00000001:0.0:1788880920.193045:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880920.193047:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:3.0:1788880920.193048:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880920.193049:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:2.0:1788880920.193049:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880920.193050:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880920.193052:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880920.193053:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880920.193053:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002b387587. 00000100:00000010:2.0:1788880920.193056:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e2be23b2. 02000000:00000001:0.0:1788880920.193057:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880920.193058:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880920.193060:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00001000:0.0:1788880920.193061:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=148 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880920.193061:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880920.193063:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880920.193065:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880920.193068:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880920.193069:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=82 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880920.193072:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880920.193072:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880920.193073:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880920.193074:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880920.193076:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880920.193077:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880920.193077:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880920.193080:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880920.193082:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880920.193082:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880920.193086:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:0.0:1788880920.193087:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880920.193093:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880920.193095:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880920.193096:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880920.193097:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880920.193098:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880920.193099:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880920.193103:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880920.193104:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880920.193105:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880920.193106:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880920.193110:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780696704:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880920.193117:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880920.193118:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880920.193120:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002980f9ff x1875776780696704/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880936 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880920.193123:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880920.193124:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880920.193125:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000016c67947. 02000000:00000001:0.0:1788880920.193129:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880920.193129:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880920.193130:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880920.193131:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880920.193132:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 02000000:00000001:0.0:1788880920.193134:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880920.193135:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880920.193136:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002980f9ff. 00000100:00000001:0.0:1788880920.193137:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880920.193138:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880920.193139:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880920.193141:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880920.193142:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880920.193143:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880920.193144:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880920.254326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880920.254332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880920.254339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880920.254340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880920.254395:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880920.254398:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880920.254400:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880921.278334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880921.278339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880921.278343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880921.278343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880921.278406:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880921.278410:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880921.278413:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880922.302328:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880922.302335:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880922.302342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880922.302343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880922.302389:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880922.302392:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880922.302394:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880923.326352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880923.326358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880923.326365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880923.326367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880923.326446:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880923.326453:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880923.326458:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880924.350335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880924.350341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880924.350346:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880924.350347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880924.350397:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880924.350401:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880924.350403:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788880925.310279:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306556367, at 4306556416 00000100:00000040:0.0:1788880925.310283:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306556367, at 4306556416 00000100:00000040:2.0:1788880925.310290:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306556367, at 4306556416 00000100:00000010:3.0:1788880925.310295:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d6fe56. 00000100:00000010:0.0:1788880925.310296:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c7c25dfc. 00000100:00000010:2.0:1788880925.310298:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b0b8cfdf. 00000100:00000040:1.0:1788880925.310316:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306556367, at 4306556416 00000100:00000010:1.0:1788880925.310325:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002aba003e. 00000100:00000040:3.0:1788880925.310349:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880925.310354:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880925.310359:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788880925.310363:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788880925.310368:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880925.310370:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788880925.310374:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880925.310381:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:3.0:1788880925.310388:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880925.310398:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880925.310408:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880925.310411:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880925.310418:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_00+0] req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880925.310438:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880925.310441:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880925.310445:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880925.310448:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000020:00000040:3.0:1788880925.310450:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880925.310453:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880925.310453:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880925.310456:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00100000:1.0:1788880925.310459:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 02000000:00000001:3.0:1788880925.310460:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880925.310465:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1788880925.310467:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:3.0:1788880925.310471:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880925.310474:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788880925.310475:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880925.310478:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880925.310483:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880925.310486:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880925.310488:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880925.310490:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788880925.310491:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880925.310499:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880925.310499:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_01+1] req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880925.310528:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880925.310531:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880925.310537:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880925.310541:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880925.310545:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880925.310546:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880925.310550:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788880925.310554:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780696832:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880925.310556:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880925.310560:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000100:00000001:1.0:1788880925.310561:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000020:00000040:3.0:1788880925.310565:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788880925.310569:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880925.310569:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788880925.310573:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880925.310574:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880925.310576:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880925.310580:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880925.310581:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 02000000:00000010:1.0:1788880925.310586:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b476d059. 00000100:00000001:3.0:1788880925.310586:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880925.310591:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880925.310591:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:1.0:1788880925.310599:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000008388b815. 00000100:00000001:0.0:1788880925.310600:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880925.310602:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:0.0:1788880925.310604:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:3.0:1788880925.310605:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880925.310608:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000007206624. 00000100:00000001:0.0:1788880925.310614:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880925.310616:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780696832, portal 25 00000100:00000001:0.0:1788880925.310619:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880925.310620:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880925.310623:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880925.310623:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000001:0.0:1788880925.310628:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880925.310630:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880925.310632:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880941 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880925.310633:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880925.310640:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880925.310649:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880925.310654:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880925.310657:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880925.310658:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788880925.310660:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:1.0:1788880925.310662:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a131edae. 02000000:00000001:0.0:1788880925.310663:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788880925.310666:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780696832, offset 0 02000000:00000001:0.0:1788880925.310673:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880925.310675:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:0.0:1788880925.310680:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780696960:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788880925.310693:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_02+2] req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880925.310696:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788880925.310700:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788880925.310701:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880925.310705:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880925.310709:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788880925.310713:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:0.0:1788880925.310714:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d9c9c3cb. 02000000:00000001:0.0:1788880925.310718:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880925.310719:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:0.0:1788880925.310725:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000fe5bffa. 00000400:00000200:1.0:1788880925.310725:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880925.310726:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880925.310731:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:1.0:1788880925.310732:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:0.0:1788880925.310738:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 00000100:00000040:3.0:1788880925.310738:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:1.0:1788880925.310739:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [522:522:256:4294967295] 192.168.204.111@tcp LPNI seq info [522:522:8:4294967295] 00000100:00000200:0.0:1788880925.310745:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780696960, portal 10 00000100:00000040:3.0:1788880925.310747:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11894) 00000100:00000001:0.0:1788880925.310749:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880925.310752:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000400:00000200:1.0:1788880925.310758:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:0.0:1788880925.310763:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880925.310767:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:1.0:1788880925.310768:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880925.310771:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880925.310775:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000010:1.0:1788880925.310777:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000100:00000001:0.0:1788880925.310781:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880925.310785:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880925.310789:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 00000800:00000200:1.0:1788880925.310789:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:0.0:1788880925.310793:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780696960, offset 0 00000400:00000200:0.0:1788880925.310799:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880925.310803:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880925.310808:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880925.310809:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880925.310812:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880925.310819:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880925.310821:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880925.310823:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880925.310824:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880925.310828:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [523:523:255:4294967295] 192.168.204.111@tcp LPNI seq info [523:523:7:4294967295] 00000100:00000001:1.0:1788880925.310829:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306561487 00000400:00000200:0.0:1788880925.310836:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:1.0:1788880925.310836:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000800:00000200:0.0:1788880925.310843:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880925.310846:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000010:0.0:1788880925.310847:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000001c4b7888. 00000800:00000200:0.0:1788880925.310852:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880925.310853:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306561487 00000800:00000200:0.0:1788880925.310858:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:1.0:1788880925.310860:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000800:00000200:0.0:1788880925.310861:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000001c4b7888 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880925.310866:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880925.310867:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:0.0:1788880925.310869:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 168300 usecs (169 jiffies) 00000100:00000001:1.0:1788880925.310872:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306561487 00000100:00000001:0.0:1788880925.310873:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306556585 00000100:00000001:0.0:1788880925.310877:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306561487 00000100:00000010:1.0:1788880925.310879:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000100:00000001:0.0:1788880925.310880:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306556585 00000100:00000001:0.0:1788880925.310885:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306561487 00000100:00000040:3.0:1788880925.310885:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880925.310886:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880925.310890:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306556585 00000100:00000001:0.0:1788880925.310894:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306561487 00000100:00000001:1.0:1788880925.310894:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306561487 00000100:00000001:0.0:1788880925.310900:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306556585 00000100:00000010:1.0:1788880925.310902:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000100:00000001:1.0:1788880925.310908:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880925.310908:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880925.310913:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880925.310914:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306561487 00000100:00000001:0.0:1788880925.310918:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880925.310919:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880925.310920:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880925.310925:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880925.310926:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880925.310927:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880925.310930:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880925.310934:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880925.310936:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880925.310938:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880925.310940:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880925.310945:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880925.310949:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880925.310951:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880925.310959:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 02000000:00000001:3.0:1788880925.310961:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880925.310963:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000100:00000001:1.0:1788880925.310964:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880925.310968:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880925.310969:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880925.310969:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780697088:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880925.310975:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880925.310978:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880925.310981:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a131edae 02000000:00000001:3.0:1788880925.310983:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880925.310985:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880925.310989:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a131edae. 02000000:00000001:3.0:1788880925.310989:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880925.310994:0:201280:0:(events.c:34:request_out_callback()) Process entered 02000000:00000010:3.0:1788880925.310995:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000001363d535. 00000100:00000200:0.0:1788880925.310998:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880941 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880925.311000:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880925.311004:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005a31f011. 00000400:00000010:3.0:1788880925.311008:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007fe20b6f. 00000100:00000200:3.0:1788880925.311012:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780697088, portal 10 00000100:00000001:3.0:1788880925.311018:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880925.311019:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880925.311019:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 00000100:00000040:0.0:1788880925.311024:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880941 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880925.311028:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880925.311033:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880925.311037:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880925.311042:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880925.311045:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880925.311048:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007839347f. 00000100:00000200:3.0:1788880925.311050:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780697088, offset 0 00000400:00000200:3.0:1788880925.311056:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880925.311061:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880925.311065:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1788880925.311066:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880925.311069:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000001c4b7888. 00000400:00000200:3.0:1788880925.311072:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880925.311073:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880925.311078:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880925.311079:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880925.311086:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880925.311089:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [524:524:255:4294967295] 192.168.204.111@tcp LPNI seq info [524:524:7:4294967295] 00000400:00000200:3.0:1788880925.311096:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:0.0:1788880925.311101:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8884ddc 00000400:00000010:0.0:1788880925.311103:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8884ddc. 00000100:00000001:0.0:1788880925.311105:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880925.311107:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880925.311111:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880925.311113:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880925.311116:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880925.311117:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880925.311139:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880925.311143:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000800:00000200:3.0:1788880925.311149:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880925.311153:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880925.311161:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880925.311174:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880925.311177:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12323 usecs (13 jiffies) 00000100:00000001:3.0:1788880925.311180:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306556429 00000100:00000001:3.0:1788880925.311183:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306561487 00000100:00000001:3.0:1788880925.311185:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306556429 00000100:00000001:3.0:1788880925.311187:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306561487 00000100:00000001:3.0:1788880925.311189:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306556429 00000100:00000001:3.0:1788880925.311191:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306561487 00000100:00000001:3.0:1788880925.311194:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306556429 00000100:00000001:3.0:1788880925.311195:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306561487 00000100:00000001:3.0:1788880925.311197:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880925.311199:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880925.311202:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880925.311204:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880925.311208:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880925.311209:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880925.311211:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880925.311213:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880925.311214:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880925.311216:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880925.311261:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880925.311265:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:1.0:1788880925.311270:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880925.311276:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880925.311280:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007839347f 00000400:00000010:1.0:1788880925.311282:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007839347f. 00000100:00000001:1.0:1788880925.311286:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880925.311290:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880925.311309:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880925.311313:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880925.311320:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880925.311322:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880925.312713:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880925.312729:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880925.312732:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880925.312734:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880925.312740:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880925.312750:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0fd00 00000400:00000200:1.0:1788880925.312757:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1119 [1] + 224 00000800:00000001:1.0:1788880925.312763:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880925.312768:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880925.312770:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880925.312774:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880925.312779:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880925.312783:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880925.312791:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880925.312812:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880925.312818:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880925.312821:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880925.312825:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880925.312827:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880925.312831:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880925.312833:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880925.312834:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000008388b815. 00000400:00000200:3.0:1788880925.312837:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000007206624 00000400:00000010:3.0:1788880925.312838:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000007206624. 00000100:00000001:3.0:1788880925.312840:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880925.312843:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880925.312925:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880925.312935:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880925.312940:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880925.312942:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880925.312943:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880925.312947:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880925.312948:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880925.312952:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:0.0:1788880925.312953:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880925.312961:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880925.312968:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0fe00 00000400:00000200:0.0:1788880925.312973:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1129 [1] + 224 00000800:00000001:0.0:1788880925.312977:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880925.312981:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880925.312983:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880925.312993:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880925.312996:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880925.312999:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880925.313005:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880925.313017:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880925.313021:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880925.313023:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880925.313046:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880925.313050:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880925.313053:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880925.313054:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880925.313056:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005a31f011. 00000100:00000001:3.0:1788880925.313056:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880925.313061:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880925.313062:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007fe20b6f 00000400:00000010:1.0:1788880925.313066:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007fe20b6f. 02000000:00000001:3.0:1788880925.313066:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880925.313072:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880925.313073:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:3.0:1788880925.313076:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=153 v=5 (5 5 5 5) 00000800:00000001:0.0:1788880925.313079:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880925.313080:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880925.313083:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=153 v=5 (1 1 1 1) 00000800:00000001:0.0:1788880925.313084:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880925.313088:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880925.313091:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000200:1.0:1788880925.313101:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880925.313102:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880925.313113:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:0.0:1788880925.313116:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0fd80 00000100:00000001:1.0:1788880925.313116:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880925.313123:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1121 [1] + 224 00000100:00000001:1.0:1788880925.313126:0:201287:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:0.0:1788880925.313128:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880925.313128:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880925.313130:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00010000:00000001:3.0:1788880925.313133:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:0.0:1788880925.313134:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788880925.313136:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880925.313137:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880925.313140:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880925.313141:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880925.313144:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880925.313144:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880925.313150:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880925.313150:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880925.313151:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880925.313157:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880925.313159:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=153 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880925.313167:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=87 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880925.313174:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880925.313176:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880925.313180:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880925.313180:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000040:0.0:1788880925.313181:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880925.313183:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880925.313185:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:3.0:1788880925.313193:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00080000:1.0:1788880925.313194:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880925.313195:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880925.313199:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880925.313202:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880925.313212:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880925.313215:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880925.313220:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00100000:3.0:1788880925.313220:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780696832:192.168.204.111@tcp:400: 00010000:00000001:1.0:1788880925.313221:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880925.313224:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880925.313227:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880925.313228:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880925.313231:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000010:0.0:1788880925.313253:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000fe5bffa. 00000100:00000001:3.0:1788880925.313258:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1788880925.313259:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000100:00000001:1.0:1788880925.313260:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000010:0.0:1788880925.313263:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000001:1.0:1788880925.313264:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880925.313264:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780696832/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880925.313269:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:1.0:1788880925.313274:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880925.313276:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880925.313281:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880925.313285:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880925.313289:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b476d059. 02000000:00000001:3.0:1788880925.313295:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880925.313298:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788880925.313303:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000020:00000040:3.0:1788880925.313303:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000200:0.0:1788880925.313306:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880925.313314:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880925.313315:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880925.313315:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000010:3.0:1788880925.313317:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 00000100:00000001:1.0:1788880925.313319:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880925.313320:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880925.313322:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880925.313324:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880925.313326:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:3.0:1788880925.313326:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880925.313329:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:0.0:1788880925.313330:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:1.0:1788880925.313330:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880925.313333:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880925.313335:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:0.0:1788880925.313337:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880925.313339:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880925.313342:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=143 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880925.313343:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880925.313348:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=143 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880925.313354:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00100000:1.0:1788880925.313354:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780697088:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880925.313357:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880925.313357:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880925.313361:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880925.313362:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788880925.313363:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880925.313365:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:0.0:1788880925.313367:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880925.313367:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880925.313368:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880925.313373:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880925.313373:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780697088/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880925.313377:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880925.313381:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880925.313383:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880925.313386:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880925.313389:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:0.0:1788880925.313390:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880925.313392:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000001363d535. 02000000:00000001:1.0:1788880925.313398:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880925.313399:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000020:00000001:1.0:1788880925.313400:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880925.313402:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000020:00000040:1.0:1788880925.313403:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880925.313404:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880925.313409:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788880925.313409:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880925.313412:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:1.0:1788880925.313415:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880925.313416:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880925.313418:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:1.0:1788880925.313422:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880925.313424:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880925.313428:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:0.0:1788880925.313430:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780696960:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880925.313433:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880925.313435:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880925.313437:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880925.313439:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880925.313444:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880925.313447:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880925.313451:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780696960/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880941 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880925.313458:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880925.313460:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880925.313461:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d9c9c3cb. 02000000:00000001:0.0:1788880925.313463:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880925.313464:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880925.313465:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880925.313466:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880925.313467:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 02000000:00000001:0.0:1788880925.313468:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880925.313469:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880925.313469:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:0.0:1788880925.313471:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880925.313472:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880925.313473:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880925.313476:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880925.313476:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880925.313477:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880925.313478:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880925.374325:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880925.374334:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880925.374342:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880925.374348:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880925.374352:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880925.374355:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880925.374356:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880925.374359:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880925.374362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880925.374364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880925.374412:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880925.374415:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880925.374417:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880926.398371:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880926.398379:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880926.398385:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880926.398386:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880926.398446:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880926.398453:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880926.398457:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880927.422321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880927.422330:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880927.422337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880927.422338:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880927.422399:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880927.422403:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880927.422406:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880928.446323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880928.446329:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880928.446334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880928.446336:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880928.446338:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880928.446340:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880928.446342:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880928.446344:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880928.446345:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880928.446347:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880928.446411:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880928.446416:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880928.446420:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880929.470321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880929.470328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880929.470335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880929.470336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880929.470390:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880929.470394:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880929.470398:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880930.430354:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306561487, at 4306561536 00000100:00000040:2.0:1788880930.430354:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306561487, at 4306561536 00000100:00000040:3.0:1788880930.430369:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306561487, at 4306561536 00000100:00000010:1.0:1788880930.430375:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:2.0:1788880930.430377:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:3.0:1788880930.430380:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000040:2.0:1788880930.430424:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880930.430431:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880930.430444:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 00000100:00000040:0.0:1788880930.430450:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306561487, at 4306561536 00000020:00000040:2.0:1788880930.430450:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:0.0:1788880930.430469:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000001:2.0:1788880930.430479:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880930.430485:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880930.430492:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880930.430502:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 00000100:00000001:2.0:1788880930.430511:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880930.430535:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880930.430547:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880930.430551:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880930.430558:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_03+3] req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880930.430606:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880930.430609:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880930.430613:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880930.430616:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:2.0:1788880930.430618:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000005a8bb1de. 00000100:00000001:1.0:1788880930.430621:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:2.0:1788880930.430622:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880930.430624:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880930.430628:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:1.0:1788880930.430631:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880930.430632:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880930.430636:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880930.430641:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 00000100:00000001:3.0:1788880930.430646:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880930.430647:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880930.430654:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880930.430654:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880930.430670:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880930.430672:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880930.430674:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880930.430675:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880930.430681:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880930.430682:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a8bb1de] to pc [ptlrpcd_00_00+0] req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880930.430685:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880930.430691:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880930.430698:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780697216:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880930.430704:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880930.430712:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880930.430714:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880930.430716:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880930.430718:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880930.430722:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:1.0:1788880930.430725:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008b528dd3. 00000100:00000001:2.0:1788880930.430728:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1788880930.430730:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880930.430739:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000400:00000010:1.0:1788880930.430740:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b43de041. 00000020:00000040:2.0:1788880930.430745:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880930.430745:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1788880930.430751:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005377d7ad. 00000100:00000001:2.0:1788880930.430752:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00100000:3.0:1788880930.430752:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788880930.430753:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880930.430756:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880930.430761:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880930.430762:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880930.430762:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880930.430764:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780697216, portal 25 02000000:00000010:2.0:1788880930.430772:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008eccda10. 00000100:00000001:1.0:1788880930.430773:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880930.430773:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880930.430779:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000100:00000001:3.0:1788880930.430779:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880930.430781:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880930.430790:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:1.0:1788880930.430793:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880946 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880930.430793:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880930.430800:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880930.430803:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880930.430809:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880930.430819:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880930.430823:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880930.430827:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880930.430832:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_01+1] req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880930.430836:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880930.430837:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880930.430842:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880930.430845:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880930.430846:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880930.430851:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880930.430857:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000007206624. 00000100:00100000:3.0:1788880930.430857:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780697344:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880930.430863:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780697216, offset 0 00000100:00000001:3.0:1788880930.430864:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880930.430867:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880930.430869:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880930.430873:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880930.430873:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880930.430877:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880930.430878:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880930.430881:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000010:3.0:1788880930.430885:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000063883087. 02000000:00000001:3.0:1788880930.430890:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880930.430893:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11899) 00000400:00000010:3.0:1788880930.430895:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000592dd309. 00000400:00000200:1.0:1788880930.430900:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880930.430901:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880930.430903:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000100:00100000:0.0:1788880930.430913:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000200:3.0:1788880930.430913:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780697344, portal 10 00000400:00000200:1.0:1788880930.430914:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880930.430922:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880930.430924:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880930.430925:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880930.430928:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938427392 : -126823771124224 : ffff8ca78a2c4e00) 00000400:00000200:1.0:1788880930.430941:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880930.430950:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:3.0:1788880930.430950:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880930.430953:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880930.430963:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880930.430964:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [525:525:256:4294967295] 192.168.204.111@tcp LPNI seq info [525:525:8:4294967295] 00000100:00000001:3.0:1788880930.430973:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880930.430976:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788880930.430984:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880930.430985:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:3.0:1788880930.430989:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eab8d071. 00000100:00000001:0.0:1788880930.430991:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:3.0:1788880930.430996:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780697344, offset 0 00000400:00000200:1.0:1788880930.430998:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:3.0:1788880930.431007:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880930.431015:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880930.431016:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880930.431021:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:1.0:1788880930.431029:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d7d1e0d5. 00000400:00000200:3.0:1788880930.431032:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:1.0:1788880930.431045:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880930.431054:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880930.431064:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880930.431065:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000200:1.0:1788880930.431065:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880930.431073:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880930.431074:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788880930.431076:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d7d1e0d5 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880930.431085:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [526:526:255:4294967295] 192.168.204.111@tcp LPNI seq info [526:526:7:4294967295] 02000000:00000001:0.0:1788880930.431086:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880930.431092:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880930.431100:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880930.431101:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880930.431106:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880930.431116:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306566607 00000100:00100000:0.0:1788880930.431121:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780697472:192.168.204.111@tcp:400: 00000800:00000200:3.0:1788880930.431121:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788880930.431126:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000800:00000010:3.0:1788880930.431130:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000001:0.0:1788880930.431135:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880930.431136:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1788880930.431144:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306566607 02000000:00000001:0.0:1788880930.431147:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000800:00000200:3.0:1788880930.431147:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:1.0:1788880930.431151:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 02000000:00000001:0.0:1788880930.431152:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880930.431158:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 02000000:00000001:0.0:1788880930.431160:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:3.0:1788880930.431168:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880930.431169:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306566607 02000000:00000010:0.0:1788880930.431173:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009b8d16de. 00000800:00000200:3.0:1788880930.431173:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000010:1.0:1788880930.431176:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 02000000:00000001:0.0:1788880930.431184:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880930.431186:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880930.431187:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880930.431195:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306566607 00000100:00000040:3.0:1788880930.431199:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 167236 usecs (168 jiffies) 00000100:00000010:1.0:1788880930.431200:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000400:00000010:0.0:1788880930.431204:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000fe5bffa. 00000100:00000001:3.0:1788880930.431205:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306561704 00000100:00000001:1.0:1788880930.431207:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880930.431213:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306566607 00000100:00000001:1.0:1788880930.431215:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880930.431216:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306561704 00000400:00000010:0.0:1788880930.431218:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 00000100:00000001:1.0:1788880930.431222:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880930.431223:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306566607 00000100:00000001:3.0:1788880930.431229:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306561704 00000100:00000001:1.0:1788880930.431257:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880930.431262:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880930.431267:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880930.431270:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880930.431274:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:0.0:1788880930.431275:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780697472, portal 10 00000100:00000001:1.0:1788880930.431276:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880930.431276:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306566607 00000100:00000001:1.0:1788880930.431281:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880930.431285:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306561705 00000100:00000001:0.0:1788880930.431287:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880930.431288:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880930.431289:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306566607 00000800:00000200:2.0:1788880930.431291:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880930.431294:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 00000100:00000001:3.0:1788880930.431295:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:2.0:1788880930.431300:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d7d1e0d5. 00000100:00000001:3.0:1788880930.431301:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880930.431308:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880930.431310:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880930.431311:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880930.431318:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880930.431318:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880930.431321:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880930.431323:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880930.431327:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880930.431332:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000007206624 00000100:00000001:3.0:1788880930.431334:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880930.431338:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000007206624. 00000100:00000001:2.0:1788880930.431346:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880930.431351:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880946 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880930.431356:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880930.431364:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880930.431365:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880930.431369:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880946 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880930.431372:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 00000100:00000200:0.0:1788880930.431386:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780697472, offset 0 00000100:00000001:2.0:1788880930.431387:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880930.431393:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:0.0:1788880930.431405:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880930.431416:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880930.431419:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:0.0:1788880930.431424:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788880930.431424:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880930.431433:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880930.431439:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880930.431456:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880930.431463:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880930.431468:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880930.431472:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [527:527:255:4294967295] 192.168.204.111@tcp LPNI seq info [527:527:7:4294967295] 00000400:00000200:0.0:1788880930.431484:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:2.0:1788880930.431491:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eab8d071 00000800:00000200:0.0:1788880930.431492:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880930.431497:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eab8d071. 00000800:00000010:0.0:1788880930.431501:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000001c4b7888. 00000100:00000001:2.0:1788880930.431503:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880930.431510:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788880930.431515:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880930.431529:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880930.431540:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000001c4b7888 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880930.431543:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880930.431548:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880930.431555:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12263 usecs (13 jiffies) 00000100:00000040:2.0:1788880930.431558:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880930.431562:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306561550 00000100:00000001:2.0:1788880930.431573:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880930.431575:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880930.431582:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306566607 00000100:00000001:0.0:1788880930.431598:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306561550 00000100:00000001:0.0:1788880930.431601:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306566607 00000100:00000001:0.0:1788880930.431603:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306561550 00000100:00000001:0.0:1788880930.431606:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306566607 00000100:00000001:0.0:1788880930.431608:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306561550 00000100:00000001:0.0:1788880930.431610:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306566607 00000100:00000001:0.0:1788880930.431612:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880930.431615:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880930.431618:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880930.431621:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880930.431637:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880930.431640:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000001c4b7888. 00000400:00000200:2.0:1788880930.431644:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880930.431647:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880930.431649:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8884ddc 00000400:00000010:2.0:1788880930.431651:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8884ddc. 00000100:00000001:2.0:1788880930.431654:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880930.431657:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880930.431664:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880930.431666:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880930.431672:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880930.431674:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880930.432872:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880930.432900:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880930.432905:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880930.432908:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880930.432917:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880930.432940:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b0fe80 00000400:00000200:0.0:1788880930.432949:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1131 [1] + 224 00000800:00000001:0.0:1788880930.432955:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880930.432979:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880930.432982:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880930.432988:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880930.432993:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880930.432998:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880930.433010:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880930.433027:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880930.433039:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880930.433044:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880930.433046:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880930.433048:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880930.433053:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880930.433060:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ff00 00000400:00000200:0.0:1788880930.433065:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1139 [1] + 224 00000100:00000001:1.0:1788880930.433065:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880930.433070:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:0.0:1788880930.433072:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880930.433077:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880930.433078:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:0.0:1788880930.433079:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:1.0:1788880930.433082:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:0.0:1788880930.433086:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:1.0:1788880930.433087:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b43de041. 00000400:00000200:1.0:1788880930.433093:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005377d7ad 00000400:00000010:1.0:1788880930.433096:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005377d7ad. 00000100:00000001:1.0:1788880930.433099:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880930.433101:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880930.433104:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880930.433113:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880930.433119:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880930.433121:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880930.433123:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880930.433126:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880930.433128:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880930.433131:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880930.433133:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880930.433136:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880930.433144:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788880930.433146:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=8 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880930.433154:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=8 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880930.433159:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880930.433160:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880930.433161:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880930.433166:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:1.0:1788880930.433167:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880930.433170:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880930.433176:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788880930.433177:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880930.433182:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788880930.433185:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880930.433187:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880930.433194:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880930.433202:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880930.433206:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880930.433207:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000001:3.0:1788880930.433211:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880930.433214:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788880930.433216:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000010:3.0:1788880930.433217:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000592dd309. 00000100:00000001:1.0:1788880930.433221:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880930.433222:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b0ff80 00000400:00000200:3.0:1788880930.433231:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000400:00000010:3.0:1788880930.433268:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000100:00000040:1.0:1788880930.433275:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880930.433276:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:0.0:1788880930.433279:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1141 [1] + 224 00000800:00000001:0.0:1788880930.433290:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1788880930.433291:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:0.0:1788880930.433302:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1788880930.433306:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780697216:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880930.433310:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880930.433323:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880930.433328:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880930.433333:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880930.433342:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:1.0:1788880930.433342:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780697216/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880930.433350:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880930.433390:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880930.433398:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880930.433404:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008b528dd3. 00000100:00000040:0.0:1788880930.433414:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880930.433422:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880930.433429:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000200:3.0:1788880930.433429:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:1.0:1788880930.433433:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880930.433434:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880930.433440:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000001:1.0:1788880930.433441:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000800:00000001:0.0:1788880930.433445:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880930.433447:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 02000000:00000001:1.0:1788880930.433456:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880930.433462:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788880930.433463:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000010:1.0:1788880930.433464:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 00000100:00000001:2.0:1788880930.433466:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880930.433468:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880930.433471:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880930.433471:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880930.433474:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880930.433476:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880930.433478:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:3.0:1788880930.433480:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880930.433481:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788880930.433482:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880930.433488:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000fe5bffa. 00000100:00000001:3.0:1788880930.433489:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880930.433494:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880930.433495:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000100:00000001:1.0:1788880930.433498:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:3.0:1788880930.433500:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880930.433502:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880930.433508:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000001:1.0:1788880930.433509:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:3.0:1788880930.433511:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=148 v=5 (5 5 5 5) 00000100:00000001:1.0:1788880930.433514:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880930.433515:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:3.0:1788880930.433519:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=148 v=5 (1 1 1 1) 00000100:00000200:2.0:1788880930.433524:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880930.433528:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880930.433532:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880930.433539:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000200:2.0:1788880930.433545:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880930.433548:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880930.433556:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880930.433564:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880930.433565:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880930.433568:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880930.433569:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880930.433574:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880930.433574:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880930.433581:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880930.433594:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:3.0:1788880930.433595:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880930.433598:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880930.433605:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880930.433615:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=8 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880930.433618:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00001000:2.0:1788880930.433622:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=92 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880930.433625:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880930.433628:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880930.433629:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880930.433632:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880930.433638:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788880930.433644:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00100000:3.0:1788880930.433652:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780697344:192.168.204.111@tcp:400: 00010000:00080000:2.0:1788880930.433655:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880930.433665:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788880930.433670:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880930.433673:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880930.433676:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880930.433681:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000005a8bb1de x1875776780697344/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880930.433684:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788880930.433689:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788880930.433692:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880930.433701:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880930.433722:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880930.433729:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880930.433732:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880930.433742:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880930.433745:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880930.433752:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880930.433756:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000063883087. 02000000:00000001:3.0:1788880930.433760:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880930.433764:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880930.433767:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880930.433777:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880930.433782:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 00000100:00100000:2.0:1788880930.433786:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780697472:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880930.433789:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880930.433794:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788880930.433797:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:3.0:1788880930.433798:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000005a8bb1de. 00000100:00000001:2.0:1788880930.433803:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880930.433806:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:2.0:1788880930.433809:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780697472/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880946 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880930.433811:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880930.433820:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880930.433830:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880930.433833:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880930.433834:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000010:2.0:1788880930.433836:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009b8d16de. 00000100:00000001:3.0:1788880930.433839:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880930.433843:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880930.433845:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:2.0:1788880930.433846:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880930.433850:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880930.433851:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880930.433854:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880930.433860:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008eccda10. 02000000:00000001:2.0:1788880930.433865:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880930.433867:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880930.433869:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000100:00000001:2.0:1788880930.433874:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880930.433876:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880930.433879:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880930.433885:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880930.433886:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880930.433889:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880930.433891:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880930.494378:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880930.494385:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880930.494392:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880930.494394:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880930.494449:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880930.494454:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880930.494458:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880931.518313:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880931.518319:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880931.518324:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880931.518324:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880931.518372:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880931.518375:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880931.518377:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880932.542312:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880932.542319:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880932.542325:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880932.542326:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880932.542367:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880932.542371:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880932.542374:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880933.566347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880933.566354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880933.566360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880933.566361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880933.566413:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880933.566416:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880933.566419:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880934.590314:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880934.590320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880934.590325:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880934.590328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880934.590401:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880934.590408:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880934.590420:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880935.550386:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306566607, at 4306566656 00000100:00000040:3.0:1788880935.550386:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306566607, at 4306566656 00000100:00000040:1.0:1788880935.550388:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306566607, at 4306566656 00000100:00000040:2.0:1788880935.550389:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880935.550410:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880935.550411:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:3.0:1788880935.550414:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000010:1.0:1788880935.550421:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000010:2.0:1788880935.550421:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000020:00000040:2.0:1788880935.550431:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880935.550437:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880935.550441:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880935.550448:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880935.550456:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008eccda10. 00000100:00000001:2.0:1788880935.550463:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880935.550508:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880935.550519:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880935.550521:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880935.550528:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_02+2] req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880935.550553:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880935.550555:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880935.550560:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880935.550564:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cc13c26b. 00000020:00000040:2.0:1788880935.550568:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880935.550570:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880935.550573:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880935.550575:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880935.550576:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880935.550580:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5d1e8ed. 00000100:00000001:3.0:1788880935.550582:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880935.550583:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880935.550587:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880935.550589:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880935.550589:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880935.550597:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880935.550602:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880935.550622:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880935.550625:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880935.550635:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880935.550636:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880935.550637:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880935.550642:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cc13c26b] to pc [ptlrpcd_00_03+3] req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880935.550653:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880935.550657:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880935.550661:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780697600:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880935.550667:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880935.550673:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880935.550676:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880935.550678:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880935.550681:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.550682:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788880935.550686:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788880935.550690:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880935.550694:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008b528dd3. 02000000:00000001:3.0:1788880935.550701:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.550713:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:3.0:1788880935.550714:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000592dd309. 00000100:00000001:1.0:1788880935.550716:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880935.550721:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:3.0:1788880935.550722:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 02000000:00000001:1.0:1788880935.550724:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880935.550728:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880935.550730:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780697600, portal 25 00000100:00000001:2.0:1788880935.550732:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880935.550735:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780697728:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880935.550737:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880935.550741:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 00000100:00000040:2.0:1788880935.550742:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880935.550750:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880935.550751:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880951 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880935.550755:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:1.0:1788880935.550758:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880935.550764:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880935.550766:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000910cb22c. 00000100:00000001:3.0:1788880935.550775:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788880935.550776:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788880935.550782:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000020:00000040:2.0:1788880935.550783:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000010:1.0:1788880935.550786:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005578e134. 00000400:00000010:3.0:1788880935.550786:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000097c86a00. 00000100:00000001:2.0:1788880935.550789:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000200:3.0:1788880935.550792:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780697600, offset 0 02000000:00000001:2.0:1788880935.550793:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880935.550794:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880935.550799:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880935.550800:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880935.550804:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b43de041. 02000000:00000010:2.0:1788880935.550813:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000400:00000200:3.0:1788880935.550816:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:1.0:1788880935.550817:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005377d7ad. 00000100:00000001:2.0:1788880935.550820:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880935.550824:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:1.0:1788880935.550828:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780697728, portal 10 00000400:00000200:3.0:1788880935.550831:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:2.0:1788880935.550837:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880935.550838:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788880935.550840:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880935.550843:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938433664 : -126823771117952 : ffff8ca78a2c6680) 00000400:00000200:3.0:1788880935.550848:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880935.550861:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [528:528:256:4294967295] 192.168.204.111@tcp LPNI seq info [528:528:8:4294967295] 00000100:00000040:1.0:1788880935.550862:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880935.550884:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880935.550890:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880935.550899:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:2.0:1788880935.550906:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000910cb22c] to pc [ptlrpcd_00_00+0] req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880935.550913:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:3.0:1788880935.550918:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880935.550929:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880935.550933:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000400:00000010:1.0:1788880935.550935:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a131edae. 00000100:00000200:1.0:1788880935.550941:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780697728, offset 0 00000800:00000200:3.0:1788880935.550948:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880935.550956:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880935.550956:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880935.550961:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000200:3.0:1788880935.550966:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:2.0:1788880935.550969:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:1.0:1788880935.550972:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:2.0:1788880935.550976:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11904) 00000800:00000200:3.0:1788880935.550987:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880935.550988:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880935.550996:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880935.551000:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880935.551002:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880935.551002:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306566607, at 4306566656 00000100:00000001:0.0:1788880935.551005:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788880935.551006:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880935.551006:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880935.551012:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000040:0.0:1788880935.551015:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880935.551017:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880935.551023:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [529:529:255:4294967295] 192.168.204.111@tcp LPNI seq info [529:529:7:4294967295] 00000100:00000001:3.0:1788880935.551023:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306571727 00000100:00000010:3.0:1788880935.551037:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002ad0ec0e. 00000100:00000001:0.0:1788880935.551040:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880935.551040:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880935.551044:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880935.551046:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880935.551047:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 02000000:00000001:0.0:1788880935.551050:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788880935.551054:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788880935.551055:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880935.551057:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306571727 02000000:00000001:0.0:1788880935.551062:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880935.551064:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000025d9193b. 00000800:00000200:1.0:1788880935.551069:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00100000:0.0:1788880935.551071:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780697856:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880935.551075:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880935.551078:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000010:1.0:1788880935.551078:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000009908d3b1. 02000000:00000001:0.0:1788880935.551083:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880935.551084:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306571727 02000000:00000001:0.0:1788880935.551087:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880935.551091:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:3.0:1788880935.551092:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a5ad6403. 02000000:00000001:0.0:1788880935.551093:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880935.551100:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 02000000:00000010:0.0:1788880935.551103:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d9c9c3cb. 02000000:00000001:0.0:1788880935.551108:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880935.551109:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306571727 00000800:00000200:1.0:1788880935.551112:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000010:3.0:1788880935.551119:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d6fe56. 00000100:00000001:3.0:1788880935.551123:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788880935.551128:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880935.551129:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000bcc6d2e. 00000800:00000200:1.0:1788880935.551133:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000009908d3b1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880935.551137:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880935.551141:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000001:3.0:1788880935.551142:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.551143:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880935.551149:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880935.551151:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780697856, portal 10 00000100:00000040:1.0:1788880935.551153:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 166186 usecs (167 jiffies) 00000100:00000001:0.0:1788880935.551156:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:2.0:1788880935.551157:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880935.551158:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306566823 00000100:00000001:3.0:1788880935.551158:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880935.551159:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938437248 : -126823771114368 : ffff8ca78a2c7480) 00000800:00000010:2.0:1788880935.551164:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000100:00000001:3.0:1788880935.551165:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880935.551174:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880935.551175:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306571727 00000400:00000200:2.0:1788880935.551177:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880935.551178:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788880935.551179:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880935.551182:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.551184:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306566823 00000100:00000001:3.0:1788880935.551191:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.551193:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306571727 00000400:00000200:2.0:1788880935.551196:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880935.551201:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306566823 00000100:00000001:1.0:1788880935.551207:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306571727 00000400:00000200:2.0:1788880935.551208:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000097c86a00 00000100:00000001:0.0:1788880935.551211:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880935.551213:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306566823 00000400:00000010:2.0:1788880935.551213:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000097c86a00. 00000100:00000040:0.0:1788880935.551217:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880935.551220:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306571727 00000400:00000010:0.0:1788880935.551221:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e9812fb0. 00000100:00000001:2.0:1788880935.551222:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880935.551224:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880935.551229:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780697856, offset 0 00000100:00000001:1.0:1788880935.551229:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880935.551230:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880951 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880935.551259:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.551263:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880935.551272:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880935.551273:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880935.551274:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880935.551277:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788880935.551281:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880935.551282:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880935.551282:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880951 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880935.551285:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788880935.551288:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880935.551288:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.551293:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880935.551295:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880935.551301:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880935.551302:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880935.551305:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880935.551305:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:0.0:1788880935.551309:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [530:530:254:4294967295] 192.168.204.111@tcp LPNI seq info [530:530:6:4294967295] 00000400:00000200:0.0:1788880935.551322:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880935.551333:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880935.551337:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b7399401. 00000800:00000200:0.0:1788880935.551342:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788880935.551347:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880935.551350:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b7399401 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880935.551353:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880935.551356:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12205 usecs (13 jiffies) 00000100:00000001:0.0:1788880935.551359:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306566670 00000100:00000001:0.0:1788880935.551361:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306571727 00000100:00000001:0.0:1788880935.551363:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306566670 00000100:00000001:0.0:1788880935.551364:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306571727 00000100:00000001:0.0:1788880935.551366:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306566670 00000100:00000001:0.0:1788880935.551368:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306571727 00000100:00000001:0.0:1788880935.551370:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306566670 00000100:00000001:0.0:1788880935.551372:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306571727 00000100:00000001:0.0:1788880935.551374:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788880935.551374:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880935.551378:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000009908d3b1. 00000100:00000001:0.0:1788880935.551379:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880935.551383:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880935.551384:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880935.551388:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880935.551391:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880935.551394:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880935.551395:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a131edae 00000100:00000001:0.0:1788880935.551398:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000010:2.0:1788880935.551398:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a131edae. 00000100:00000001:2.0:1788880935.551403:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880935.551404:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880935.551408:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880935.551416:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880935.551417:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880935.551420:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880935.551422:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880935.551424:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880935.551433:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880935.551435:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880935.551446:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880935.551448:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b7399401. 00000400:00000200:2.0:1788880935.551450:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880935.551453:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880935.551456:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e9812fb0 00000400:00000010:2.0:1788880935.551458:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e9812fb0. 00000100:00000001:2.0:1788880935.551460:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880935.551463:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880935.551467:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880935.551469:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880935.551473:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880935.551474:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880935.552905:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880935.552917:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880935.552920:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880935.552923:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880935.552929:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880935.552946:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b10000 00000400:00000200:2.0:1788880935.552959:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1149 [1] + 224 00000800:00000001:2.0:1788880935.552965:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880935.552968:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880935.552969:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880935.552972:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880935.552974:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880935.552977:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880935.552984:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880935.552995:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880935.553002:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880935.553005:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880935.553006:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880935.553008:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880935.553012:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880935.553017:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880935.553019:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10080 00000400:00000200:2.0:1788880935.553025:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1151 [1] + 224 00000100:00000001:3.0:1788880935.553026:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880935.553032:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1788880935.553033:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880935.553036:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880935.553037:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788880935.553040:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:3.0:1788880935.553041:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000592dd309. 00000400:00000200:2.0:1788880935.553047:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880935.553048:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000400:00000010:3.0:1788880935.553051:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000100:00000001:3.0:1788880935.553055:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880935.553059:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880935.553060:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880935.553064:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880935.553081:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880935.553111:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880935.553115:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880935.553122:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880935.553122:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880935.553126:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880935.553128:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880935.553131:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880935.553134:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880935.553137:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880935.553138:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880935.553141:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880935.553144:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880935.553146:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.553153:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880935.553153:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:3.0:1788880935.553153:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=13 v=5 (5 5 5 5) 00000100:00000001:1.0:1788880935.553158:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:3.0:1788880935.553162:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=13 v=5 (1 1 1 1) 00000400:00000001:1.0:1788880935.553164:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880935.553168:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788880935.553168:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10100 00000100:00000001:3.0:1788880935.553170:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000010:1.0:1788880935.553171:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b43de041. 00000100:00000001:3.0:1788880935.553174:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880935.553177:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005377d7ad 00010000:00000001:3.0:1788880935.553178:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000010:1.0:1788880935.553181:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005377d7ad. 00010000:00000001:3.0:1788880935.553182:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.553186:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880935.553187:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880935.553191:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1159 [1] + 224 00000100:00000200:1.0:1788880935.553195:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880935.553196:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880935.553197:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880935.553203:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880935.553208:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880935.553216:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880935.553224:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880935.553226:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880935.553231:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880935.553255:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880935.553258:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880935.553259:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880935.553261:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880935.553263:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880935.553264:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880935.553266:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880935.553271:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=153 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880935.553277:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00001000:1.0:1788880935.553281:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=153 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880935.553283:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880935.553286:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880935.553288:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880935.553291:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880935.553296:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:3.0:1788880935.553296:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880935.553298:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880935.553301:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.553307:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880935.553310:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880935.553315:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880935.553318:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880935.553326:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880935.553329:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780697600:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880935.553333:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880935.553341:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880935.553342:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880935.553347:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:2.0:1788880935.553348:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880935.553350:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880935.553354:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780697600/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880935.553355:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880935.553357:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880935.553366:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880935.553367:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880935.553371:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880935.553374:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008b528dd3. 00000100:00100000:1.0:1788880935.553377:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780697728:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880935.553384:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880935.553385:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880935.553388:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000020:00000001:3.0:1788880935.553389:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880935.553392:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000040:1.0:1788880935.553394:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cc13c26b x1875776780697728/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788880935.553396:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880935.553400:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008eccda10. 00000100:00000001:1.0:1788880935.553407:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880935.553407:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880935.553411:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:3.0:1788880935.553411:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000010:1.0:1788880935.553415:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005578e134. 00000100:00000001:0.0:1788880935.553416:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:3.0:1788880935.553416:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000100:00000001:0.0:1788880935.553420:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880935.553421:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880935.553422:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000020:00000001:1.0:1788880935.553424:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000001:0.0:1788880935.553425:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880935.553426:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:1.0:1788880935.553427:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000001:0.0:1788880935.553428:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788880935.553430:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788880935.553431:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000bcc6d2e. 00000020:00000001:1.0:1788880935.553432:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000200:0.0:1788880935.553437:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 02000000:00000010:1.0:1788880935.553437:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5d1e8ed. 02000000:00000001:1.0:1788880935.553441:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880935.553443:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000010:0.0:1788880935.553444:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:3.0:1788880935.553446:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000010:1.0:1788880935.553447:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cc13c26b. 00000100:00000001:0.0:1788880935.553448:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880935.553450:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880935.553454:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880935.553454:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880935.553456:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880935.553457:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880935.553460:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.553462:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880935.553469:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:0.0:1788880935.553470:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880935.553473:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880935.553479:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880935.553480:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880935.553483:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880935.553483:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880935.553499:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880935.553502:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880935.553503:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880935.553504:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880935.553508:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880935.553511:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=13 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880935.553513:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=97 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880935.553515:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880935.553516:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880935.553517:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880935.553519:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880935.553523:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880935.553524:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880935.553525:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880935.553526:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880935.553526:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880935.553528:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880935.553532:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880935.553534:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880935.553536:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880935.553538:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880935.553555:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780697856:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880935.553559:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880935.553561:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880935.553563:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000910cb22c x1875776780697856/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880951 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880935.553568:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880935.553570:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880935.553571:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d9c9c3cb. 02000000:00000001:0.0:1788880935.553574:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880935.553575:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880935.553576:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880935.553578:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880935.553579:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 02000000:00000001:0.0:1788880935.553582:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880935.553583:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880935.553584:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000910cb22c. 00000100:00000001:0.0:1788880935.553586:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880935.553587:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880935.553589:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880935.553592:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880935.553593:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880935.553595:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880935.553596:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880935.614344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880935.614352:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880935.614354:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880935.614357:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880935.614364:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880935.614366:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880935.614368:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880935.614371:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880935.614375:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880935.614376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880935.614432:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880935.614436:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880935.614439:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880936.638323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880936.638328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880936.638332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880936.638332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880936.638386:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880936.638391:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880936.638394:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880937.662357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880937.662366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880937.662374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880937.662375:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880937.662431:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880937.662436:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880937.662439:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880938.686369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880938.686377:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880938.686383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880938.686387:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880938.686389:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880938.686392:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880938.686395:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880938.686398:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880938.686399:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880938.686401:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880938.686466:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880938.686471:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880938.686474:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880939.710320:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880939.710325:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880939.710331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880939.710332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880939.710379:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880939.710383:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880939.710384:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880940.670331:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306571727, at 4306571776 00000100:00000040:1.0:1788880940.670331:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306571727, at 4306571776 00000100:00000040:3.0:1788880940.670331:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306571727, at 4306571776 00000100:00000040:2.0:1788880940.670332:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306571727, at 4306571776 00000100:00000010:3.0:1788880940.670346:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d6fe56. 00000100:00000010:0.0:1788880940.670348:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002ad0ec0e. 00000100:00000010:1.0:1788880940.670351:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000025d9193b. 00000100:00000010:2.0:1788880940.670358:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a5ad6403. 00000100:00000040:0.0:1788880940.670377:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880940.670408:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880940.670412:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000363de9c9. 00000020:00000040:0.0:1788880940.670415:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880940.670418:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880940.670420:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880940.670424:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880940.670429:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000ae4abc27. 00000100:00000001:0.0:1788880940.670435:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880940.670443:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880940.670450:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880940.670451:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880940.670455:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000363de9c9] to pc [ptlrpcd_00_01+1] req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880940.670473:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880940.670474:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880940.670477:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880940.670479:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c2a263b1. 00000020:00000040:0.0:1788880940.670481:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880940.670482:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880940.670483:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788880940.670485:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880940.670487:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000bd68c00a. 00000100:00000001:0.0:1788880940.670488:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880940.670489:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880940.670492:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880940.670494:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880940.670496:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880940.670500:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880940.670502:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880940.670503:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880940.670507:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c2a263b1] to pc [ptlrpcd_00_02+2] req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880940.670509:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880940.670515:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880940.670524:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880940.670526:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880940.670529:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880940.670531:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880940.670533:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880940.670535:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780697984:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880940.670537:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880940.670539:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:0.0:1788880940.670546:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880940.670548:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880940.670550:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880940.670551:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00100000:1.0:1788880940.670552:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 02000000:00000001:2.0:1788880940.670553:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880940.670555:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:0.0:1788880940.670556:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000fd20c80. 00000100:00000001:1.0:1788880940.670558:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880940.670559:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788880940.670559:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880940.670561:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880940.670565:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000010:2.0:1788880940.670566:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000c9708566. 02000000:00000001:0.0:1788880940.670567:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880940.670567:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788880940.670571:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880940.670572:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880940.670572:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880940.670578:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880940.670581:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000087d24527. 00000400:00000010:2.0:1788880940.670581:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006bdd9925. 00000100:00000001:1.0:1788880940.670582:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880940.670585:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788880940.670588:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880940.670591:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c56cc487. 00000100:00000040:1.0:1788880940.670593:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880940.670597:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000200:2.0:1788880940.670601:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780697984, portal 25 00000100:00000001:2.0:1788880940.670607:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880940.670610:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849365888 : -126822860185728 : ffff8ca7c0781f80) 00000100:00000001:1.0:1788880940.670616:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880940.670620:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880940.670620:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880956 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880940.670621:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788880940.670623:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880940.670625:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880940.670629:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880940.670631:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000fd20c80] to pc [ptlrpcd_00_03+3] req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880940.670635:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880940.670636:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880940.670643:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788880940.670644:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780698112:192.168.204.111@tcp:400: 00000400:00000010:2.0:1788880940.670647:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008f790e4a. 00000100:00000200:2.0:1788880940.670652:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780697984, offset 0 00000100:00000001:1.0:1788880940.670653:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880940.670657:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880940.670657:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880940.670658:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880940.670660:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788880940.670662:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880940.670664:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788880940.670667:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000010:1.0:1788880940.670671:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b2a347c7. 00000100:00000040:0.0:1788880940.670672:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11909) 00000100:00000001:3.0:1788880940.670674:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880940.670675:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788880940.670676:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880940.670679:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000400:00000200:2.0:1788880940.670681:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:1.0:1788880940.670682:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b43de041. 00000100:00000001:3.0:1788880940.670682:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880940.670687:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880940.670688:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:1.0:1788880940.670690:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005377d7ad. 00000400:00000200:2.0:1788880940.670692:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880940.670692:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880940.670697:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780698112, portal 10 00000400:00000200:2.0:1788880940.670698:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880940.670698:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880940.670700:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880940.670702:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880940.670703:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880940.670706:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364992 : -126822860186624 : ffff8ca7c0781c00) 00000400:00000200:2.0:1788880940.670706:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [531:531:256:4294967295] 192.168.204.111@tcp LPNI seq info [531:531:8:4294967295] 00000100:00000040:3.0:1788880940.670710:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880940.670719:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880940.670725:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880940.670741:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880940.670745:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880940.670747:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880940.670748:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000011e1ab42. 02000000:00000001:3.0:1788880940.670751:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880940.670754:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880940.670755:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880940.670759:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880940.670760:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:1.0:1788880940.670763:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00100000:3.0:1788880940.670764:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780698240:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880940.670768:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:1.0:1788880940.670769:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009822ffbc. 00000800:00000200:2.0:1788880940.670771:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:3.0:1788880940.670771:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880940.670774:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880940.670775:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780698112, offset 0 00000800:00000200:2.0:1788880940.670776:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000011e1ab42 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:3.0:1788880940.670779:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880940.670784:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ca284ac2. 02000000:00000001:3.0:1788880940.670787:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880940.670788:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880940.670790:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000592dd309. 00000100:00000001:2.0:1788880940.670792:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880940.670796:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306576847 00000100:00000010:2.0:1788880940.670800:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e91f0731. 00000400:00000200:1.0:1788880940.670802:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:3.0:1788880940.670803:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000400:00000200:1.0:1788880940.670808:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880940.670810:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000200:3.0:1788880940.670810:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780698240, portal 10 00000400:00000200:1.0:1788880940.670815:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880940.670816:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306576847 00000100:00000001:3.0:1788880940.670816:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880940.670820:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849357824 : -126822860193792 : ffff8ca7c0780000) 00000400:00000200:1.0:1788880940.670821:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:2.0:1788880940.670821:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003dccfd54. 00000400:00000200:1.0:1788880940.670827:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788880940.670827:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:2.0:1788880940.670830:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306576847 00000100:00000040:3.0:1788880940.670833:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880940.670836:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [532:532:255:4294967295] 192.168.204.111@tcp LPNI seq info [532:532:7:4294967295] 00000100:00000010:2.0:1788880940.670836:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000247083fe. 00000100:00000001:2.0:1788880940.670839:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788880940.670844:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306576847 00000100:00000010:2.0:1788880940.670849:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000008b80730b. 00000100:00000001:2.0:1788880940.670854:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788880940.670856:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880940.670860:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880940.670860:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788880940.670865:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880940.670867:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880940.670868:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880940.670870:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880940.670871:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ad0499ab. 00000800:00000010:1.0:1788880940.670874:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 00000100:00000001:2.0:1788880940.670874:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880940.670875:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780698240, offset 0 00000100:00000001:2.0:1788880940.670880:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880940.670884:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:1.0:1788880940.670886:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880940.670886:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880940.670888:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880940.670891:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788880940.670894:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880940.670895:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880940.670897:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880940.670898:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880940.670902:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880940.670902:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:1.0:1788880940.670903:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000800:00000010:0.0:1788880940.670905:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000011e1ab42. 00000400:00000200:3.0:1788880940.670907:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880940.670912:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880940.670912:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880940.670913:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:1.0:1788880940.670918:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 165148 usecs (166 jiffies) 00000400:00000200:3.0:1788880940.670918:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880940.670924:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880940.670924:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [533:533:254:4294967295] 192.168.204.111@tcp LPNI seq info [533:533:6:4294967295] 00000100:00000001:1.0:1788880940.670925:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306571942 00000100:00000001:1.0:1788880940.670931:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306576847 00000100:00000001:1.0:1788880940.670936:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306571942 00000400:00000200:3.0:1788880940.670939:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880940.670940:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306576847 00000100:00000001:1.0:1788880940.670943:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306571942 00000100:00000001:1.0:1788880940.670945:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306576847 00000800:00000200:3.0:1788880940.670952:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880940.670953:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008f790e4a 00000100:00000001:1.0:1788880940.670958:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306571942 00000400:00000010:0.0:1788880940.670959:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008f790e4a. 00000100:00000001:1.0:1788880940.670961:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306576847 00000100:00000001:1.0:1788880940.670965:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880940.670966:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880940.670967:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880940.670971:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880940.670973:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880956 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880940.670975:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880940.670979:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880940.670981:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880940.670984:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880940.670989:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880940.670990:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880940.670994:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880956 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880940.671010:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880940.671012:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000010:3.0:1788880940.671020:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000800:00000200:3.0:1788880940.671028:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788880940.671033:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880940.671036:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880940.671040:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788880940.671042:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880940.671044:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000100:00000040:3.0:1788880940.671044:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12147 usecs (13 jiffies) 00000400:00000200:0.0:1788880940.671049:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880940.671050:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306571789 00000400:00000200:0.0:1788880940.671055:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880940.671055:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306576847 00000400:00000200:0.0:1788880940.671058:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009822ffbc 00000400:00000010:0.0:1788880940.671060:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009822ffbc. 00000100:00000001:3.0:1788880940.671060:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306571789 00000100:00000001:0.0:1788880940.671063:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880940.671065:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306576847 00000100:00000200:0.0:1788880940.671067:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880940.671070:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306571789 00000100:00000001:0.0:1788880940.671074:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880940.671075:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306576847 00000100:00000040:0.0:1788880940.671078:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880940.671083:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880940.671084:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788880940.671091:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788880940.671093:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:0.0:1788880940.671094:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880940.671096:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880940.671097:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ad0499ab 00000400:00000010:0.0:1788880940.671098:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ad0499ab. 00000100:00000001:0.0:1788880940.671099:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788880940.671101:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880940.671105:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880940.671106:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880940.671110:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880940.671111:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880940.671149:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306571789 00000100:00000001:3.0:1788880940.671153:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306576847 00000100:00000001:3.0:1788880940.671155:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880940.671157:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880940.671159:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880940.671161:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880940.672337:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880940.672346:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880940.672348:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880940.672349:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880940.672353:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880940.672361:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b10180 00000400:00000200:2.0:1788880940.672365:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1161 [1] + 224 00000800:00000001:2.0:1788880940.672369:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880940.672371:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880940.672372:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880940.672374:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880940.672377:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880940.672379:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880940.672402:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880940.672414:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880940.672419:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880940.672450:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880940.672453:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880940.672456:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880940.672460:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880940.672464:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880940.672467:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880940.672469:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10200 00000400:00000001:3.0:1788880940.672473:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880940.672475:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880940.672476:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006bdd9925. 00000400:00000200:3.0:1788880940.672479:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c56cc487 00000400:00000010:3.0:1788880940.672480:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c56cc487. 00000400:00000200:2.0:1788880940.672486:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1169 [1] + 224 00000800:00000001:2.0:1788880940.672490:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880940.672493:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880940.672495:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880940.672497:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880940.672499:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880940.672501:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880940.672506:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880940.672511:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880940.672516:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880940.672519:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880940.672524:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1788880940.672527:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880940.672529:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880940.672531:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880940.672532:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880940.672542:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788880940.672545:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880940.672548:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880940.672549:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10280 00000100:00000001:3.0:1788880940.672551:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880940.672553:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880940.672554:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000200:2.0:1788880940.672555:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1171 [1] + 224 00000100:00000001:1.0:1788880940.672557:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880940.672558:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880940.672561:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880940.672562:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788880940.672563:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880940.672564:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880940.672566:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880940.672568:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:3.0:1788880940.672571:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=18 v=5 (5 5 5 5) 00000400:00000010:1.0:1788880940.672572:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b43de041. 00000400:00000200:2.0:1788880940.672574:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880940.672580:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005377d7ad 00000400:00000200:2.0:1788880940.672581:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00001000:3.0:1788880940.672581:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=18 v=5 (1 1 1 1) 00000400:00000010:1.0:1788880940.672585:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005377d7ad. 00000100:00000001:3.0:1788880940.672587:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880940.672589:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880940.672591:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880940.672591:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:3.0:1788880940.672592:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880940.672595:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880940.672597:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880940.672600:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880940.672601:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880940.672604:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880940.672635:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880940.672638:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000040:2.0:1788880940.672639:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880940.672640:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880940.672643:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880940.672647:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880940.672675:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000363de9c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780697984:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880940.672676:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880940.672679:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880940.672680:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880940.672681:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880940.672683:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880940.672684:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:1.0:1788880940.672686:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788880940.672687:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:3.0:1788880940.672689:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000363de9c9 x1875776780697984/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880940.672690:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880940.672692:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880940.672693:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880940.672697:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880940.672698:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880940.672701:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880940.672704:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00001000:1.0:1788880940.672705:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=8 v=5 (5 5 5 5) 02000000:00000001:3.0:1788880940.672706:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000001:0.0:1788880940.672710:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000010:3.0:1788880940.672710:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000c9708566. 00000100:00001000:1.0:1788880940.672711:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=8 v=5 (1 1 1 1) 00000400:00000001:0.0:1788880940.672713:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:3.0:1788880940.672715:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000010:0.0:1788880940.672716:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000592dd309. 00000100:00000001:1.0:1788880940.672718:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000020:00000001:3.0:1788880940.672718:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000200:0.0:1788880940.672722:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000100:00000001:1.0:1788880940.672722:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880940.672723:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000010:0.0:1788880940.672725:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00010000:00000001:1.0:1788880940.672726:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880940.672728:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:1.0:1788880940.672732:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880940.672732:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000200:0.0:1788880940.672735:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880940.672736:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000ae4abc27. 00000100:00000001:1.0:1788880940.672738:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:3.0:1788880940.672742:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:1.0:1788880940.672743:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:3.0:1788880940.672745:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880940.672749:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000010:3.0:1788880940.672749:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000363de9c9. 00000100:00000200:0.0:1788880940.672750:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880940.672754:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880940.672755:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880940.672757:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880940.672761:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880940.672762:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880940.672765:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880940.672765:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880940.672766:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880940.672769:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880940.672770:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880940.672773:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880940.672774:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880940.672776:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880940.672777:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880940.672778:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880940.672782:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880940.672787:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=18 v=5 (5 5 5 5) 00000100:00000001:1.0:1788880940.672788:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00001000:0.0:1788880940.672792:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=102 v=5 (1 0 0 0) 00000100:00000001:1.0:1788880940.672793:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880940.672795:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880940.672797:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880940.672797:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880940.672802:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:1.0:1788880940.672805:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00080000:0.0:1788880940.672807:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880940.672832:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880940.672835:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00100000:1.0:1788880940.672836:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c2a263b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780698112:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788880940.672837:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880940.672841:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880940.672843:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880940.672847:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880940.672848:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880940.672849:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880940.672854:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c2a263b1 x1875776780698112/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880940.672859:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880940.672860:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880940.672862:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b2a347c7. 02000000:00000001:1.0:1788880940.672864:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880940.672865:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880940.672866:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880940.672867:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880940.672868:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000bd68c00a. 02000000:00000001:1.0:1788880940.672872:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880940.672873:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880940.672874:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880940.672876:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000010:1.0:1788880940.672877:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c2a263b1. 00000100:00000001:1.0:1788880940.672879:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880940.672881:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880940.672883:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880940.672884:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880940.672889:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880940.672889:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880940.672892:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880940.672895:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880940.672897:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880940.672904:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780698240:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880940.672908:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880940.672910:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880940.672912:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000fd20c80 x1875776780698240/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880956 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880940.672916:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880940.672917:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880940.672918:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ca284ac2. 02000000:00000001:0.0:1788880940.672921:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880940.672922:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880940.672922:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880940.672924:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880940.672925:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000087d24527. 02000000:00000001:0.0:1788880940.672926:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880940.672927:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880940.672927:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000fd20c80. 00000100:00000001:0.0:1788880940.672929:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880940.672930:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880940.672931:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880940.672933:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880940.672934:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880940.672935:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880940.672936:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880940.734341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880940.734349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880940.734353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880940.734354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880940.734415:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880940.734419:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880940.734422:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880941.758338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880941.758346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880941.758351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880941.758352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880941.758417:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880941.758421:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880941.758424:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880942.782338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880942.782343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880942.782359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880942.782360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880942.782408:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880942.782411:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880942.782414:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880943.806361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880943.806368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880943.806374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880943.806375:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880943.806436:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880943.806440:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880943.806443:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880944.830341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880944.830347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880944.830351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880944.830352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880944.830404:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880944.830408:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880944.830411:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880945.790342:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306576847, at 4306576896 00000100:00000040:1.0:1788880945.790342:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306576847, at 4306576896 00000100:00000040:2.0:1788880945.790342:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306576847, at 4306576896 00000100:00000040:3.0:1788880945.790345:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306576847, at 4306576896 00000100:00000010:1.0:1788880945.790357:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003dccfd54. 00000100:00000010:2.0:1788880945.790358:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000247083fe. 00000100:00000010:0.0:1788880945.790361:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e91f0731. 00000100:00000010:3.0:1788880945.790363:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000008b80730b. 00000100:00000040:1.0:1788880945.790374:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880945.790378:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880945.790384:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002980f9ff. 00000020:00000040:1.0:1788880945.790387:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880945.790392:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880945.790394:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880945.790398:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880945.790403:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:1.0:1788880945.790408:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880945.790416:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880945.790423:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880945.790425:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880945.790429:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002980f9ff] to pc [ptlrpcd_00_00+0] req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880945.790447:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880945.790449:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880945.790451:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880945.790454:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e2be23b2. 00000020:00000040:1.0:1788880945.790456:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880945.790457:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880945.790458:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880945.790460:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880945.790462:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880945.790462:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:0.0:1788880945.790466:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880945.790466:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880945.790469:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788880945.790472:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880945.790474:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880945.790476:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880945.790479:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880945.790487:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880945.790488:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880945.790494:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e2be23b2] to pc [ptlrpcd_00_01+1] req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880945.790496:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880945.790498:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880945.790501:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880945.790504:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880945.790507:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880945.790509:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780698368:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880945.790512:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880945.790514:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880945.790517:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:1.0:1788880945.790517:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880945.790521:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880945.790521:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:0.0:1788880945.790524:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000010:1.0:1788880945.790524:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002b387587. 00000020:00000040:1.0:1788880945.790528:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000010:0.0:1788880945.790530:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000fa0f7569. 00000100:00000001:1.0:1788880945.790530:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880945.790531:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880945.790531:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880945.790534:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880945.790534:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880945.790536:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:1.0:1788880945.790538:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:3.0:1788880945.790539:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880945.790542:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000000bcc6d2e. 00000100:00000001:1.0:1788880945.790542:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880945.790547:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:3.0:1788880945.790547:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880945.790551:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000200:0.0:1788880945.790560:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780698368, portal 25 00000100:00000001:0.0:1788880945.790565:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880945.790569:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938061312 : -126823771490304 : ffff8ca78a26b800) 00000100:00000001:1.0:1788880945.790576:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880945.790578:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880945.790578:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:0.0:1788880945.790580:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880961 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880945.790582:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880945.790584:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002b387587] to pc [ptlrpcd_00_02+2] req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880945.790588:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880945.790593:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880945.790598:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880945.790606:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780698496:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880945.790609:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880945.790612:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:0.0:1788880945.790616:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880945.790616:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:0.0:1788880945.790619:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f34d0caa. 00000100:00000001:1.0:1788880945.790619:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880945.790622:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:0.0:1788880945.790623:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780698368, offset 0 00000100:00000040:1.0:1788880945.790629:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:3.0:1788880945.790633:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880945.790634:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880945.790635:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11914) 02000000:00000001:3.0:1788880945.790638:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880945.790640:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880945.790644:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:3.0:1788880945.790644:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ca284ac2. 00000100:00000001:1.0:1788880945.790645:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788880945.790648:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880945.790648:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880945.790650:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880945.790650:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880945.790656:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:2.0:1788880945.790657:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880945.790659:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000008ce0b05. 00000400:00000010:3.0:1788880945.790667:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000400:00000200:0.0:1788880945.790669:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880945.790673:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000200:3.0:1788880945.790674:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780698496, portal 10 00000400:00000200:0.0:1788880945.790676:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880945.790677:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880945.790679:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788880945.790681:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880945.790682:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055936 : -126823771495680 : ffff8ca78a26a300) 02000000:00000001:2.0:1788880945.790683:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788880945.790686:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [534:534:256:4294967295] 192.168.204.111@tcp LPNI seq info [534:534:8:4294967295] 02000000:00000001:2.0:1788880945.790687:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880945.790691:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880945.790695:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780698624:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880945.790699:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880945.790703:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788880945.790703:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788880945.790704:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880945.790705:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880945.790709:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880945.790714:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000feb2a015. 00000800:00000200:0.0:1788880945.790716:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880945.790720:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880945.790723:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000044c69dd2. 00000100:00000001:3.0:1788880945.790726:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:2.0:1788880945.790728:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000003e3a0df8. 00000800:00000200:0.0:1788880945.790733:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:3.0:1788880945.790734:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880945.790735:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000005509e00. 00000400:00000010:3.0:1788880945.790740:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ad0499ab. 00000100:00000200:2.0:1788880945.790741:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780698624, portal 10 00000800:00000200:0.0:1788880945.790742:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880945.790745:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000044c69dd2 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880945.790752:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880945.790753:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938057728 : -126823771493888 : ffff8ca78a26aa00) 00000100:00000200:3.0:1788880945.790755:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780698496, offset 0 00000100:00000001:0.0:1788880945.790760:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880945.790761:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880945.790766:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880945.790767:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306581967 00000100:00000001:2.0:1788880945.790773:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:0.0:1788880945.790774:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000751d0eed. 00000100:00000040:2.0:1788880945.790777:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880945.790779:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880945.790780:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000010:2.0:1788880945.790782:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001c5b8d0f. 00000100:00000001:0.0:1788880945.790785:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306581967 00000100:00000200:2.0:1788880945.790787:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780698624, offset 0 00000400:00000200:3.0:1788880945.790788:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:0.0:1788880945.790789:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009a4dee8c. 00000100:00000001:0.0:1788880945.790794:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:2.0:1788880945.790797:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880945.790797:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880945.790798:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306581967 00000100:00000010:0.0:1788880945.790804:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000506ae1c0. 00000400:00000200:3.0:1788880945.790804:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880945.790806:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880945.790809:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306581967 00000400:00000200:3.0:1788880945.790810:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:0.0:1788880945.790812:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e602733. 00000100:00000001:0.0:1788880945.790816:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788880945.790817:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [535:535:255:4294967295] 192.168.204.111@tcp LPNI seq info [535:535:7:4294967295] 00000100:00000001:0.0:1788880945.790820:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880945.790823:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880945.790825:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880945.790828:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880945.790832:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880945.790835:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880945.790837:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:3.0:1788880945.790841:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880945.790842:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880945.790842:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880945.790849:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000010:3.0:1788880945.790849:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000400:00000200:2.0:1788880945.790850:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880945.790851:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880945.790855:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880945.790857:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880945.790859:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880945.790864:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788880945.790869:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880945.790870:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788880945.790875:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000800:00000200:1.0:1788880945.790877:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:2.0:1788880945.790877:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [536:536:254:4294967295] 192.168.204.111@tcp LPNI seq info [536:536:6:4294967295] 00000100:00000001:3.0:1788880945.790881:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880945.790882:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000044c69dd2. 00000100:00000040:3.0:1788880945.790886:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 164121 usecs (165 jiffies) 00000400:00000200:1.0:1788880945.790889:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880945.790892:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880945.790893:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306577061 00000800:00000200:2.0:1788880945.790899:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880945.790900:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880945.790905:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306581967 00000800:00000010:2.0:1788880945.790906:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f9812869. 00000400:00000200:1.0:1788880945.790908:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f34d0caa 00000100:00000001:3.0:1788880945.790909:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306577061 00000400:00000010:1.0:1788880945.790913:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f34d0caa. 00000800:00000200:2.0:1788880945.790914:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788880945.790916:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306581967 00000100:00000001:1.0:1788880945.790919:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880945.790920:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306577061 00000800:00000200:2.0:1788880945.790923:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000200:1.0:1788880945.790926:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880961 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880945.790926:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306581967 00000800:00000200:2.0:1788880945.790929:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f9812869 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880945.790932:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306577061 00000100:00000001:2.0:1788880945.790938:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880945.790938:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306581967 00000100:00000040:2.0:1788880945.790945:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12086 usecs (13 jiffies) 00000100:00000001:3.0:1788880945.790945:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880945.790946:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880945.790951:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880945.790952:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880961 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880945.790952:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306576909 00000100:00000001:2.0:1788880945.790958:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306581967 00000100:00000001:2.0:1788880945.790963:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306576909 00000100:00000001:2.0:1788880945.790967:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306581967 00000100:00000001:1.0:1788880945.790969:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880945.790972:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880945.790979:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306576909 00000100:00000001:2.0:1788880945.790981:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306581967 00000800:00000200:1.0:1788880945.790987:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788880945.790987:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306576909 00000800:00000010:1.0:1788880945.790991:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000100:00000001:2.0:1788880945.790992:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306581967 00000400:00000200:1.0:1788880945.790995:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880945.790995:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880945.791001:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880945.791005:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880945.791006:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880945.791009:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ad0499ab 00000100:00000001:2.0:1788880945.791010:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880945.791013:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ad0499ab. 00000100:00000001:2.0:1788880945.791017:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880945.791018:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788880945.791021:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880945.791024:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880945.791025:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880945.791027:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880945.791030:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880945.791033:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880945.791043:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880945.791045:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880945.791050:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880945.791055:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880945.791061:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880945.791063:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880945.791064:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880945.791066:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880945.791071:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880945.791072:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880945.791074:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880945.791075:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880945.791081:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880945.791084:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f9812869. 00000400:00000200:1.0:1788880945.791087:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880945.791090:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880945.791093:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001c5b8d0f 00000400:00000010:1.0:1788880945.791095:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001c5b8d0f. 00000100:00000001:1.0:1788880945.791098:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880945.791100:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880945.791107:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880945.791109:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880945.791115:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880945.791117:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788880945.792436:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880945.792449:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788880945.792452:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880945.792454:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880945.792460:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:0.0:1788880945.792470:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b10300 00000400:00000200:0.0:1788880945.792476:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1179 [1] + 224 00000800:00000001:0.0:1788880945.792480:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880945.792485:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788880945.792486:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788880945.792488:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880945.792491:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880945.792494:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880945.792503:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880945.792522:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788880945.792529:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788880945.792532:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880945.792548:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880945.792551:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880945.792555:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880945.792557:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880945.792559:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000000bcc6d2e. 00000400:00000200:1.0:1788880945.792563:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:1.0:1788880945.792565:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:1.0:1788880945.792567:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880945.792571:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880945.792579:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880945.792586:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880945.792587:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880945.792589:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880945.792591:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880945.792593:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880945.792595:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880945.792597:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880945.792602:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=23 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880945.792606:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=23 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880945.792609:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880945.792610:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880945.792612:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880945.792614:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880945.792617:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880945.792622:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880945.792628:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880945.792630:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880945.792632:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880945.792635:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880945.792642:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780698368:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880945.792646:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880945.792648:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880945.792650:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002980f9ff x1875776780698368/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880945.792656:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880945.792659:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880945.792661:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000fa0f7569. 02000000:00000001:1.0:1788880945.792664:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880945.792666:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880945.792667:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880945.792669:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880945.792671:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 02000000:00000001:1.0:1788880945.792676:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880945.792678:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880945.792680:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002980f9ff. 00000100:00000001:1.0:1788880945.792690:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880945.792692:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880945.792694:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880945.792697:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880945.792698:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880945.792700:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880945.792702:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880945.792723:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880945.792730:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880945.792732:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880945.792734:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880945.792739:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880945.792746:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10400 00000400:00000200:2.0:1788880945.792751:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1185 [1] + 224 00000800:00000001:2.0:1788880945.792754:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880945.792757:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880945.792759:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880945.792761:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880945.792763:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880945.792765:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880945.792771:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880945.792783:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880945.792787:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880945.792790:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880945.792807:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880945.792809:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880945.792812:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880945.792813:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880945.792815:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000003e3a0df8. 00000400:00000200:0.0:1788880945.792818:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000005509e00 00000400:00000010:0.0:1788880945.792820:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000005509e00. 00000100:00000001:0.0:1788880945.792822:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880945.792825:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880945.792829:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880945.792842:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880945.792843:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880945.792844:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880945.792845:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880945.792846:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880945.792847:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880945.792848:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880945.792851:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=23 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880945.792853:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=107 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880945.792854:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880945.792855:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880945.792856:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788880945.792858:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788880945.792864:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880945.792866:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880945.792867:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880945.792868:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880945.792869:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880945.792871:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880945.792875:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880945.792876:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880945.792876:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880945.792878:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880945.792881:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780698624:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880945.792884:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880945.792885:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880945.792886:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002b387587 x1875776780698624/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880945.792895:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880945.792896:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880945.792898:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000feb2a015. 02000000:00000001:0.0:1788880945.792899:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880945.792900:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880945.792901:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880945.792902:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880945.792903:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 02000000:00000001:0.0:1788880945.792905:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880945.792905:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880945.792906:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002b387587. 00000100:00000001:0.0:1788880945.792908:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880945.792908:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880945.792909:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880945.792912:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880945.792912:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880945.792913:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880945.792914:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880945.793059:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880945.793065:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880945.793067:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880945.793070:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880945.793075:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880945.793082:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10380 00000400:00000200:1.0:1788880945.793087:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1181 [1] + 224 00000800:00000001:1.0:1788880945.793096:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880945.793101:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880945.793103:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880945.793105:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880945.793108:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880945.793111:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880945.793120:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880945.793133:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880945.793137:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880945.793140:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880945.793147:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880945.793149:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880945.793152:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880945.793153:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880945.793155:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000008ce0b05. 00000400:00000200:3.0:1788880945.793158:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000400:00000010:3.0:1788880945.793159:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000100:00000001:3.0:1788880945.793160:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880945.793163:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880945.793169:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880945.793173:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880945.793174:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880945.793175:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880945.793180:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880945.793181:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880945.793182:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880945.793183:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880945.793186:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=13 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880945.793187:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=13 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880945.793189:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880945.793190:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880945.793191:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788880945.793192:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880945.793193:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880945.793194:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880945.793195:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880945.793196:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880945.793198:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880945.793202:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880945.793203:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880945.793204:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880945.793205:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880945.793209:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780698496:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880945.793211:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880945.793212:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880945.793213:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e2be23b2 x1875776780698496/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880961 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880945.793216:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880945.793217:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880945.793218:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ca284ac2. 02000000:00000001:3.0:1788880945.793220:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880945.793221:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880945.793222:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880945.793248:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880945.793252:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 02000000:00000001:3.0:1788880945.793257:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880945.793258:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880945.793259:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e2be23b2. 00000100:00000001:3.0:1788880945.793261:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880945.793261:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880945.793263:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880945.793265:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880945.793266:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880945.793267:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880945.793268:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880945.854364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880945.854373:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880945.854375:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880945.854378:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880945.854383:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880945.854386:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880945.854388:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880945.854390:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880945.854394:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880945.854395:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880945.854446:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880945.854450:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880945.854452:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880946.878324:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880946.878330:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880946.878337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880946.878338:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880946.878388:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880946.878392:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880946.878395:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880947.902333:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880947.902339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880947.902345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880947.902347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880947.902394:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880947.902397:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880947.902399:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880948.926310:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880948.926315:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880948.926320:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880948.926322:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880948.926324:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880948.926327:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880948.926329:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880948.926330:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880948.926332:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880948.926333:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880948.926367:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880948.926371:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880948.926373:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880949.950324:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880949.950330:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880949.950334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880949.950335:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880949.950375:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880949.950378:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880949.950381:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880950.910359:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306581967, at 4306582016 00000100:00000040:1.0:1788880950.910359:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306581967, at 4306582016 00000100:00000040:2.0:1788880950.910359:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:0.0:1788880950.910374:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000751d0eed. 00000100:00000010:1.0:1788880950.910376:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009a4dee8c. 00000100:00000001:2.0:1788880950.910384:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880950.910389:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000071512652. 00000020:00000040:2.0:1788880950.910395:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880950.910402:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880950.910406:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880950.910410:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880950.910418:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 00000100:00000001:2.0:1788880950.910425:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880950.910435:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:3.0:1788880950.910441:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306581967, at 4306582016 00000100:00000001:2.0:1788880950.910447:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880950.910450:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880950.910458:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e602733. 00000100:00000040:2.0:1788880950.910461:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000071512652] to pc [ptlrpcd_00_03+3] req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880950.910488:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880950.910491:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880950.910496:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880950.910499:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ea101a56. 00000020:00000040:2.0:1788880950.910502:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880950.910503:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880950.910507:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880950.910508:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880950.910510:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880950.910510:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788880950.910511:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788880950.910514:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880950.910515:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880950.910518:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788880950.910521:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000001:2.0:1788880950.910527:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880950.910528:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880950.910532:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880950.910535:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788880950.910537:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880950.910540:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880950.910545:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880950.910550:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000071512652 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780698752:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880950.910550:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880950.910553:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880950.910555:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880950.910561:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880950.910562:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ea101a56] to pc [ptlrpcd_00_00+0] req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880950.910563:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880950.910567:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880950.910571:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e28573ff. 02000000:00000001:1.0:1788880950.910574:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880950.910579:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b43de041. 00000400:00000010:1.0:1788880950.910584:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005377d7ad. 00000100:00000200:1.0:1788880950.910590:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780698752, portal 25 00000100:00000001:2.0:1788880950.910591:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880950.910593:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880950.910595:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430080 : -126823771121536 : ffff8ca78a2c5880) 00000100:00000001:0.0:1788880950.910599:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880950.910600:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880966 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880950.910601:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788880950.910603:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788880950.910607:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880950.910611:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788880950.910615:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880950.910617:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000010:2.0:1788880950.910617:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000910cb22c. 00000100:00000040:1.0:1788880950.910618:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880950.910621:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880950.910623:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000994ad42e. 00000020:00000040:2.0:1788880950.910624:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880950.910625:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:1.0:1788880950.910626:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780698752, offset 0 00000100:00000001:0.0:1788880950.910629:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880950.910631:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880950.910632:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788880950.910634:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880950.910634:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880950.910638:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880950.910645:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f47bc5fc. 00000100:00000040:0.0:1788880950.910647:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880950.910648:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880950.910650:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880950.910656:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:2.0:1788880950.910661:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:1.0:1788880950.910663:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880950.910669:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1788880950.910669:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880950.910673:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880950.910676:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788880950.910679:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880950.910683:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880950.910684:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [537:537:256:4294967295] 192.168.204.111@tcp LPNI seq info [537:537:8:4294967295] 00000100:00000001:2.0:1788880950.910686:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880950.910689:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880950.910691:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880950.910692:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880950.910703:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880950.910704:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000910cb22c] to pc [ptlrpcd_00_01+1] req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880950.910706:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00100000:0.0:1788880950.910712:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780698880:192.168.204.111@tcp:400: 00000800:00000200:1.0:1788880950.910717:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880950.910723:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000010:1.0:1788880950.910724:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000008c752d14. 02000000:00000001:0.0:1788880950.910728:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880950.910732:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880950.910732:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:0.0:1788880950.910737:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880950.910738:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880950.910741:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003a17641d. 00000800:00000200:1.0:1788880950.910742:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:2.0:1788880950.910743:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880950.910748:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880950.910748:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000008c752d14 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:2.0:1788880950.910749:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:0.0:1788880950.910755:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000100:00000001:3.0:1788880950.910758:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880950.910759:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11919) 00000400:00000010:0.0:1788880950.910761:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00100000:3.0:1788880950.910762:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:1.0:1788880950.910765:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880950.910770:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880950.910772:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780698880, portal 10 00000100:00000001:1.0:1788880950.910775:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306587087 00000100:00000040:2.0:1788880950.910778:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306581967, at 4306582016 00000100:00000001:0.0:1788880950.910779:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880950.910781:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938436352 : -126823771115264 : ffff8ca78a2c7100) 00000100:00000001:3.0:1788880950.910782:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000010:1.0:1788880950.910783:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000100:00000001:3.0:1788880950.910789:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880950.910792:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000040:0.0:1788880950.910794:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788880950.910794:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000506ae1c0. 00000100:00000001:3.0:1788880950.910796:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880950.910799:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306587087 00000100:00000001:3.0:1788880950.910801:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880950.910807:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000010:1.0:1788880950.910808:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000100:00000001:0.0:1788880950.910815:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880950.910816:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880950.910821:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306587087 00000100:00000040:3.0:1788880950.910821:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880950.910825:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880950.910833:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f34d0caa. 00000100:00000010:1.0:1788880950.910838:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000100:00000200:0.0:1788880950.910843:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780698880, offset 0 00000100:00000001:1.0:1788880950.910847:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788880950.910859:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880950.910861:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306587087 00000100:00000001:3.0:1788880950.910861:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880950.910871:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880950.910877:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000010:1.0:1788880950.910878:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000400:00000200:0.0:1788880950.910883:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880950.910884:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:0.0:1788880950.910888:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880950.910889:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880950.910898:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880950.910900:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880950.910900:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880950.910903:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880950.910910:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788880950.910910:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788880950.910915:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 02000000:00000001:3.0:1788880950.910915:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880950.910918:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788880950.910918:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:2.0:1788880950.910924:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000008c752d14. 00000100:00000001:1.0:1788880950.910925:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880950.910931:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00100000:3.0:1788880950.910931:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780699008:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880950.910932:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [538:538:255:4294967295] 192.168.204.111@tcp LPNI seq info [538:538:7:4294967295] 00000100:00000001:1.0:1788880950.910933:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880950.910938:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880950.910942:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880950.910951:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880950.910951:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880950.910954:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880950.910964:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880950.910972:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880950.910976:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880950.910978:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000994ad42e 00000800:00000010:0.0:1788880950.910979:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000087d24527. 00000400:00000010:2.0:1788880950.910983:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000994ad42e. 00000800:00000200:0.0:1788880950.910986:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880950.910990:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:0.0:1788880950.910991:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880950.910995:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000087d24527 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880950.910997:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880950.911000:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880966 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880950.911002:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 163110 usecs (164 jiffies) 02000000:00000001:3.0:1788880950.911005:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880950.911007:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306582180 02000000:00000001:3.0:1788880950.911014:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880950.911015:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306587087 00000100:00000001:0.0:1788880950.911021:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306582180 02000000:00000010:3.0:1788880950.911022:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000790fa4ee. 00000100:00000001:0.0:1788880950.911028:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306587087 02000000:00000001:3.0:1788880950.911029:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880950.911039:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306582180 00000400:00000010:3.0:1788880950.911041:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000008ce0b05. 00000100:00000001:0.0:1788880950.911043:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306587087 00000100:00000001:0.0:1788880950.911049:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306582180 00000400:00000010:3.0:1788880950.911051:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000100:00000001:0.0:1788880950.911052:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306587087 00000100:00000001:0.0:1788880950.911059:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880950.911059:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880950.911062:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880950.911062:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780699008, portal 10 00000100:00000040:2.0:1788880950.911067:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880966 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880950.911071:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880950.911073:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880950.911077:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938437248 : -126823771114368 : ffff8ca78a2c7480) 00000100:00000001:0.0:1788880950.911080:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880950.911086:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880950.911089:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880950.911091:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880950.911091:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880950.911094:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880950.911098:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880950.911099:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880950.911113:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880950.911116:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880950.911118:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000097c86a00. 00000100:00000200:3.0:1788880950.911122:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780699008, offset 0 00000400:00000200:3.0:1788880950.911128:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880950.911138:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880950.911143:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788880950.911145:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880950.911150:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000087d24527. 00000400:00000200:3.0:1788880950.911150:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880950.911157:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880950.911157:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880950.911167:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880950.911174:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880950.911181:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [539:539:255:4294967295] 192.168.204.111@tcp LPNI seq info [539:539:7:4294967295] 00000400:00000200:3.0:1788880950.911271:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:2.0:1788880950.911277:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f34d0caa 00000800:00000200:3.0:1788880950.911277:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788880950.911284:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000400:00000010:2.0:1788880950.911285:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f34d0caa. 00000100:00000001:2.0:1788880950.911295:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:3.0:1788880950.911295:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788880950.911308:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880950.911308:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880950.911323:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880950.911332:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880950.911334:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880950.911337:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 12027 usecs (13 jiffies) 00000100:00000040:2.0:1788880950.911343:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880950.911344:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306582030 00000100:00000001:3.0:1788880950.911350:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306587087 00000100:00000001:3.0:1788880950.911354:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306582030 00000100:00000001:3.0:1788880950.911359:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306587087 00000100:00000001:3.0:1788880950.911366:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306582030 00000100:00000001:3.0:1788880950.911373:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306587087 00000100:00000001:3.0:1788880950.911379:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306582030 00000100:00000001:3.0:1788880950.911386:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306587087 00000100:00000001:3.0:1788880950.911392:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880950.911393:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880950.911394:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880950.911400:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880950.911401:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880950.911409:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880950.911452:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880950.911455:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:2.0:1788880950.911458:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880950.911460:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880950.911463:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000097c86a00 00000400:00000010:2.0:1788880950.911464:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000097c86a00. 00000100:00000001:2.0:1788880950.911466:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880950.911468:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880950.911473:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880950.911474:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880950.911490:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880950.911492:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880950.912793:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880950.912808:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880950.912812:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880950.912814:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880950.912823:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880950.912837:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b10480 00000400:00000200:2.0:1788880950.912844:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1191 [1] + 224 00000800:00000001:2.0:1788880950.912851:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880950.912859:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880950.912862:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880950.912866:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880950.912872:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880950.912876:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880950.912887:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880950.912908:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880950.912916:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880950.912920:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880950.912921:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880950.912923:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880950.912927:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880950.912932:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10500 00000400:00000200:2.0:1788880950.912937:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1199 [1] + 224 00000800:00000001:2.0:1788880950.912941:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880950.912945:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880950.912946:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880950.912950:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880950.912952:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880950.912956:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880950.912960:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:2.0:1788880950.912965:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788880950.912968:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880950.912971:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880950.912974:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b43de041. 00000400:00000200:1.0:1788880950.912978:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005377d7ad 00000400:00000010:1.0:1788880950.912982:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005377d7ad. 00000100:00000040:2.0:1788880950.912982:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880950.912987:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880950.912994:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880950.913005:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880950.913008:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880950.913014:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880950.913016:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880950.913017:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880950.913021:0:201288:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788880950.913022:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880950.913024:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788880950.913026:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880950.913029:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788880950.913030:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880950.913033:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880950.913039:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880950.913039:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880950.913043:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880950.913048:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880950.913048:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10580 00000100:00001000:1.0:1788880950.913049:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=28 v=5 (5 5 5 5) 00000400:00000001:0.0:1788880950.913053:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00001000:1.0:1788880950.913057:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=28 v=5 (1 1 1 1) 00000400:00000001:0.0:1788880950.913058:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880950.913061:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000100:00000001:1.0:1788880950.913063:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:0.0:1788880950.913067:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000100:00000001:1.0:1788880950.913067:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880950.913070:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880950.913074:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880950.913078:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:1.0:1788880950.913079:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880950.913086:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:1.0:1788880950.913090:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880950.913091:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x11a1 [1] + 224 00000100:00000200:0.0:1788880950.913094:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880950.913101:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880950.913107:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880950.913111:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880950.913116:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880950.913120:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880950.913120:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880950.913123:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880950.913130:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:1.0:1788880950.913133:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880950.913139:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880950.913146:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:2.0:1788880950.913146:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880950.913150:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880950.913156:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00100000:1.0:1788880950.913158:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000071512652 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780698752:192.168.204.111@tcp:400: 02000000:00000001:0.0:1788880950.913159:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880950.913163:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880950.913167:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880950.913168:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880950.913168:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880950.913171:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880950.913174:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00001000:0.0:1788880950.913180:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=18 v=5 (5 5 5 5) 00000100:00000040:1.0:1788880950.913181:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000071512652 x1875776780698752/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880950.913189:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=18 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880950.913202:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880950.913207:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880950.913209:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880950.913210:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880950.913210:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:1.0:1788880950.913214:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00010000:00000001:0.0:1788880950.913215:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788880950.913216:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880950.913219:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e28573ff. 00010000:00000001:0.0:1788880950.913220:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880950.913225:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880950.913226:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880950.913228:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880950.913230:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880950.913251:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000001:3.0:1788880950.913252:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880950.913254:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880950.913255:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880950.913257:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000008ce0b05. 02000000:00000001:1.0:1788880950.913260:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000200:3.0:1788880950.913261:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000100:00000040:0.0:1788880950.913262:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880950.913264:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000020:00000001:1.0:1788880950.913265:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880950.913269:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000040:1.0:1788880950.913270:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880950.913276:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000200:3.0:1788880950.913276:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880950.913280:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 00000100:00000001:0.0:1788880950.913282:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880950.913288:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:1.0:1788880950.913288:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788880950.913292:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880950.913293:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000200:3.0:1788880950.913293:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:1.0:1788880950.913297:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000071512652. 00000100:00000040:0.0:1788880950.913302:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880950.913304:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880950.913307:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880950.913308:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880950.913309:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880950.913314:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880950.913314:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00100000:0.0:1788880950.913321:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780698880:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880950.913322:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880950.913323:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880950.913324:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880950.913327:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880950.913327:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880950.913329:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880950.913332:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880950.913333:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880950.913337:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880950.913337:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=28 v=5 (5 5 5 5) 00000100:00000001:0.0:1788880950.913339:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00001000:3.0:1788880950.913344:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=112 v=5 (1 0 0 0) 00000100:00000040:0.0:1788880950.913346:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ea101a56 x1875776780698880/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880950.913347:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880950.913349:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880950.913353:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880950.913358:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880950.913361:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880950.913364:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880950.913367:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003a17641d. 00010000:00000001:3.0:1788880950.913367:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880950.913371:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880950.913372:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:0.0:1788880950.913373:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788880950.913373:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880950.913376:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880950.913377:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880950.913380:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000040:3.0:1788880950.913380:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788880950.913386:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880950.913388:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 00000100:00000001:3.0:1788880950.913389:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880950.913392:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:0.0:1788880950.913393:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880950.913394:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880950.913396:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:3.0:1788880950.913397:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880950.913400:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ea101a56. 00000100:00000001:0.0:1788880950.913407:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00100000:3.0:1788880950.913407:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780699008:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880950.913411:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880950.913412:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880950.913416:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880950.913416:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880950.913419:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000910cb22c x1875776780699008/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880966 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880950.913422:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880950.913425:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880950.913426:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880950.913429:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880950.913430:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788880950.913432:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000790fa4ee. 00000100:00000001:0.0:1788880950.913433:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880950.913436:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880950.913443:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880950.913445:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880950.913448:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880950.913450:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f47bc5fc. 02000000:00000001:3.0:1788880950.913452:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880950.913453:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880950.913455:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000910cb22c. 00000100:00000001:3.0:1788880950.913457:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880950.913458:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880950.913460:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880950.913464:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880950.913465:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880950.913467:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880950.913468:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880950.974370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880950.974380:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880950.974389:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880950.974391:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880950.974454:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880950.974459:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880950.974463:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880951.998315:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880951.998320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880951.998325:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880951.998326:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880951.998377:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880951.998380:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880951.998382:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880953.022339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880953.022346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880953.022351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880953.022352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880953.022395:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880953.022398:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880953.022400:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880954.046310:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880954.046317:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880954.046322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880954.046323:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880954.046370:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880954.046373:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880954.046375:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880955.070352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880955.070359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880955.070366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880955.070367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880955.070414:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880955.070417:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880955.070420:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788880956.030307:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306587087, at 4306587136 00000100:00000040:1.0:1788880956.030311:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306587087, at 4306587136 00000100:00000040:3.0:1788880956.030317:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306587087, at 4306587136 00000100:00000010:1.0:1788880956.030323:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000010:3.0:1788880956.030326:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:2.0:1788880956.030328:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000040:1.0:1788880956.030351:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880956.030355:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000040:0.0:1788880956.030359:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306587087, at 4306587136 00000100:00000010:1.0:1788880956.030361:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002980f9ff. 00000020:00000040:1.0:1788880956.030367:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:0.0:1788880956.030368:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000001:1.0:1788880956.030371:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880956.030374:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880956.030378:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880956.030385:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:1.0:1788880956.030392:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880956.030400:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880956.030407:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880956.030409:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880956.030416:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002980f9ff] to pc [ptlrpcd_00_02+2] req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880956.030440:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880956.030441:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880956.030444:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880956.030446:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000663051e5. 00000020:00000040:1.0:1788880956.030449:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880956.030450:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880956.030452:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880956.030455:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00100000:2.0:1788880956.030455:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 02000000:00000001:1.0:1788880956.030458:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880956.030458:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880956.030460:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880956.030465:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880956.030465:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:2.0:1788880956.030467:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880956.030470:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880956.030471:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880956.030476:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880956.030484:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880956.030486:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880956.030489:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880956.030489:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880956.030492:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880956.030496:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880956.030500:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000663051e5] to pc [ptlrpcd_00_03+3] req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880956.030507:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880956.030510:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880956.030515:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880956.030517:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880956.030519:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880956.030521:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880956.030523:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780699136:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880956.030524:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880956.030529:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880956.030530:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880956.030533:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 00000020:00000040:1.0:1788880956.030535:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:2.0:1788880956.030535:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880956.030538:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880956.030539:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788880956.030540:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880956.030541:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880956.030544:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00100000:0.0:1788880956.030545:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 02000000:00000001:1.0:1788880956.030545:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880956.030549:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788880956.030550:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 02000000:00000010:2.0:1788880956.030550:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005309c375. 00000100:00000001:1.0:1788880956.030556:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880956.030556:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880956.030562:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788880956.030563:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:2.0:1788880956.030565:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000096e78f73. 00000100:00000001:0.0:1788880956.030566:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880956.030572:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880956.030575:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:2.0:1788880956.030575:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000001:1.0:1788880956.030582:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880956.030584:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880956.030584:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780699136, portal 25 00000100:00000001:2.0:1788880956.030589:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:1.0:1788880956.030590:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_00_00+0] req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880956.030591:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938061312 : -126823771490304 : ffff8ca78a26b800) 00000100:00000001:0.0:1788880956.030599:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880956.030599:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880972 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880956.030607:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880956.030608:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880956.030609:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880956.030613:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880956.030615:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788880956.030619:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c3a5d4ae. 00000100:00000040:1.0:1788880956.030622:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000200:2.0:1788880956.030624:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780699136, offset 0 00000100:00000001:3.0:1788880956.030629:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880956.030630:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880956.030630:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11924) 00000100:00000001:3.0:1788880956.030633:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880956.030634:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880956.030635:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880956.030636:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880956.030641:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880956.030644:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880956.030644:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880956.030649:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880956.030649:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880956.030654:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880956.030657:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:0.0:1788880956.030658:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780699264:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880956.030658:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880956.030664:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880956.030665:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880956.030667:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788880956.030669:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880956.030671:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1788880956.030672:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880956.030673:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880956.030674:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880956.030675:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880956.030679:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:3.0:1788880956.030680:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880956.030682:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000010:0.0:1788880956.030683:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a58a4768. 00000100:00100000:3.0:1788880956.030685:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780699392:192.168.204.111@tcp:400: 02000000:00000001:0.0:1788880956.030688:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880956.030690:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [540:540:256:4294967295] 192.168.204.111@tcp LPNI seq info [540:540:8:4294967295] 00000100:00000001:3.0:1788880956.030693:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:0.0:1788880956.030695:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 02000000:00000001:3.0:1788880956.030696:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:0.0:1788880956.030701:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000400:00000200:2.0:1788880956.030705:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:3.0:1788880956.030705:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880956.030707:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780699264, portal 10 02000000:00000001:3.0:1788880956.030709:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788880956.030713:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:2.0:1788880956.030714:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000010:3.0:1788880956.030714:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008d432156. 00000100:00000001:0.0:1788880956.030715:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938053248 : -126823771498368 : ffff8ca78a269880) 02000000:00000001:3.0:1788880956.030719:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788880956.030722:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000400:00000010:3.0:1788880956.030723:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000008ce0b05. 00000100:00000040:0.0:1788880956.030727:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880956.030730:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000800:00000200:2.0:1788880956.030731:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:3.0:1788880956.030737:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780699392, portal 10 00000800:00000200:2.0:1788880956.030742:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880956.030743:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880956.030743:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880956.030747:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 00000800:00000200:2.0:1788880956.030748:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000040:0.0:1788880956.030751:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880956.030758:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880956.030762:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880956.030768:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306592207 00000100:00000001:3.0:1788880956.030769:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:2.0:1788880956.030771:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ccadc63b. 00000400:00000010:0.0:1788880956.030774:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e9812fb0. 00000100:00000040:3.0:1788880956.030776:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880956.030777:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000200:0.0:1788880956.030779:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780699264, offset 0 00000100:00000001:2.0:1788880956.030782:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306592207 00000400:00000010:3.0:1788880956.030782:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eab8d071. 00000100:00000010:2.0:1788880956.030787:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f6d3b065. 00000400:00000200:0.0:1788880956.030788:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880956.030791:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:2.0:1788880956.030794:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306592207 00000100:00000200:3.0:1788880956.030795:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780699392, offset 0 00000100:00000010:2.0:1788880956.030796:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000076a63e17. 00000400:00000200:0.0:1788880956.030797:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880956.030801:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:0.0:1788880956.030806:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880956.030806:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306592207 00000400:00000200:3.0:1788880956.030807:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788880956.030811:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000247083fe. 00000100:00000001:2.0:1788880956.030815:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:0.0:1788880956.030817:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880956.030818:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880956.030821:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880956.030823:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880956.030826:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880956.030830:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [541:541:255:4294967295] 192.168.204.111@tcp LPNI seq info [541:541:7:4294967295] 00000100:00000001:2.0:1788880956.030831:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880956.030834:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880956.030837:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880956.030838:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880956.030839:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880956.030842:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880956.030847:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880956.030854:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880956.030859:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000800:00000200:0.0:1788880956.030860:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880956.030863:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788880956.030866:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880956.030869:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880956.030875:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788880956.030876:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880956.030880:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880956.030883:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880956.030886:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [542:542:254:4294967295] 192.168.204.111@tcp LPNI seq info [542:542:6:4294967295] 00000800:00000010:0.0:1788880956.030887:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000044c69dd2. 00000800:00000200:0.0:1788880956.030896:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880956.030901:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880956.030906:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880956.030910:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000044c69dd2 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788880956.030911:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c3a5d4ae 00000800:00000200:3.0:1788880956.030911:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880956.030916:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c3a5d4ae. 00000100:00000001:0.0:1788880956.030917:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880956.030919:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000040:0.0:1788880956.030923:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 162110 usecs (163 jiffies) 00000100:00000001:1.0:1788880956.030923:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:3.0:1788880956.030927:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880956.030929:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306587299 00000100:00000200:1.0:1788880956.030931:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880972 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880956.030934:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306592207 00000800:00000200:3.0:1788880956.030937:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880956.030939:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306587299 00000100:00000001:0.0:1788880956.030944:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306592207 00000100:00000001:0.0:1788880956.030948:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306587299 00000100:00000001:0.0:1788880956.030952:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306592207 00000100:00000001:1.0:1788880956.030952:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880956.030955:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306587299 00000100:00000001:0.0:1788880956.030958:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306592207 00000100:00000040:1.0:1788880956.030958:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880972 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880956.030960:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880956.030965:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880956.030969:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880956.030971:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880956.030973:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880956.030974:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880956.030991:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880956.030994:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880956.030996:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11972 usecs (12 jiffies) 00000100:00000001:3.0:1788880956.030997:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306587148 00000100:00000001:3.0:1788880956.031004:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306592207 00000100:00000001:3.0:1788880956.031006:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306587148 00000100:00000001:3.0:1788880956.031007:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306592207 00000100:00000001:3.0:1788880956.031008:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306587148 00000800:00000200:1.0:1788880956.031009:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880956.031010:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306592207 00000100:00000001:3.0:1788880956.031014:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306587148 00000800:00000010:1.0:1788880956.031015:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000044c69dd2. 00000100:00000001:3.0:1788880956.031015:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306592207 00000100:00000001:3.0:1788880956.031019:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880956.031021:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880956.031022:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880956.031023:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880956.031027:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880956.031031:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880956.031032:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880956.031034:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:1.0:1788880956.031037:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e9812fb0 00000100:00000001:3.0:1788880956.031037:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880956.031039:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:1.0:1788880956.031042:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e9812fb0. 00000100:00000001:3.0:1788880956.031042:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880956.031045:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880956.031048:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880956.031053:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880956.031066:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880956.031069:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880956.031075:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880956.031076:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880956.031121:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880956.031123:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:1.0:1788880956.031127:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880956.031130:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880956.031133:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eab8d071 00000400:00000010:1.0:1788880956.031135:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eab8d071. 00000100:00000001:1.0:1788880956.031137:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880956.031140:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880956.031147:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880956.031149:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880956.031154:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880956.031155:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880956.032658:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880956.032666:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880956.032672:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880956.032674:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880956.032678:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880956.032685:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b10600 00000400:00000200:2.0:1788880956.032689:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x11a9 [1] + 224 00000800:00000001:2.0:1788880956.032694:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880956.032696:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880956.032697:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880956.032699:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880956.032701:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880956.032704:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880956.032710:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880956.032721:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880956.032725:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880956.032727:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880956.032728:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880956.032730:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880956.032732:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880956.032736:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10680 00000400:00000200:2.0:1788880956.032739:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x11b1 [1] + 224 00000800:00000001:2.0:1788880956.032742:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880956.032749:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880956.032752:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880956.032754:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788880956.032757:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880956.032757:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:1.0:1788880956.032760:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788880956.032761:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:1.0:1788880956.032763:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000096e78f73. 00000400:00000200:1.0:1788880956.032766:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:1.0:1788880956.032767:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:1.0:1788880956.032770:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880956.032770:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880956.032774:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880956.032775:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880956.032805:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880956.032805:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880956.032827:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880956.032831:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880956.032836:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880956.032838:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880956.032840:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880956.032841:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880956.032843:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880956.032844:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880956.032848:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:1.0:1788880956.032849:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=34 v=5 (5 5 5 5) 00000800:00000001:2.0:1788880956.032852:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880956.032853:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=34 v=5 (1 1 1 1) 00000800:00000001:2.0:1788880956.032854:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880956.032858:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:2.0:1788880956.032859:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880956.032861:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880956.032865:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:2.0:1788880956.032866:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00010000:00000001:1.0:1788880956.032868:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880956.032870:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880956.032871:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880956.032873:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880956.032876:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10700 00000100:00000040:1.0:1788880956.032877:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880956.032879:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880956.032882:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880956.032885:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000100:00000001:1.0:1788880956.032887:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880956.032889:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880956.032890:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880956.032894:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880956.032897:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880956.032901:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00100000:1.0:1788880956.032903:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780699136:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880956.032907:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880956.032909:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880956.032912:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x11b5 [1] + 224 00000100:00000001:1.0:1788880956.032913:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:0.0:1788880956.032916:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880956.032918:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788880956.032919:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002980f9ff x1875776780699136/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880956.032925:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880956.032929:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880956.032937:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880956.032945:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:2.0:1788880956.032945:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880956.032948:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880956.032949:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000200:2.0:1788880956.032951:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788880956.032953:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005309c375. 02000000:00000001:1.0:1788880956.032958:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880956.032961:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880956.032964:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788880956.032969:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880956.032971:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 00000100:00000001:0.0:1788880956.032975:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788880956.032977:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:2.0:1788880956.032978:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880956.032979:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880956.032980:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880956.032983:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002980f9ff. 00000100:00000001:0.0:1788880956.032985:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880956.032988:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:0.0:1788880956.032990:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880956.032991:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880956.032994:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880956.032995:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880956.032997:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880956.032999:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1788880956.033001:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880956.033002:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880956.033004:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880956.033007:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:0.0:1788880956.033010:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=24 v=5 (5 5 5 5) 00000100:00000001:1.0:1788880956.033010:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880956.033011:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880956.033015:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880956.033016:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=24 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880956.033021:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880956.033023:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880956.033026:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880956.033029:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880956.033032:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880956.033034:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880956.033035:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880956.033038:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880956.033038:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880956.033042:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880956.033043:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880956.033046:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:0.0:1788880956.033047:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788880956.033048:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000008ce0b05. 00000400:00000200:3.0:1788880956.033052:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000400:00000010:3.0:1788880956.033054:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000100:00000001:3.0:1788880956.033058:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880956.033063:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:3.0:1788880956.033063:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880956.033069:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880956.033072:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880956.033076:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880956.033080:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880956.033083:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880956.033085:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880956.033088:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00100000:0.0:1788880956.033105:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780699264:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880956.033109:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880956.033111:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880956.033112:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880956.033114:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880956.033117:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=34 v=5 (5 5 5 5) 00000100:00000001:0.0:1788880956.033120:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788880956.033121:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=118 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880956.033125:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880956.033126:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880956.033129:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880956.033130:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:0.0:1788880956.033132:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000663051e5 x1875776780699264/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00080000:3.0:1788880956.033135:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880956.033149:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880956.033151:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880956.033151:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880956.033152:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880956.033153:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880956.033157:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880956.033161:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880956.033163:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880956.033166:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880956.033166:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880956.033169:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788880956.033170:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a58a4768. 00000100:00000040:3.0:1788880956.033173:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880956.033175:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880956.033178:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880956.033181:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00100000:3.0:1788880956.033182:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780699392:192.168.204.111@tcp:400: 00000020:00000001:0.0:1788880956.033185:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880956.033187:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788880956.033188:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 00000100:00000001:3.0:1788880956.033190:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:0.0:1788880956.033193:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:3.0:1788880956.033193:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780699392/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880972 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880956.033196:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880956.033199:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000663051e5. 00000100:00000001:3.0:1788880956.033201:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880956.033203:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788880956.033205:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000010:3.0:1788880956.033205:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008d432156. 00000100:00000001:0.0:1788880956.033207:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880956.033209:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788880956.033210:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880956.033213:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880956.033214:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880956.033216:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:3.0:1788880956.033217:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880956.033218:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788880956.033219:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880956.033223:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880956.033224:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 02000000:00000001:3.0:1788880956.033226:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880956.033228:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880956.033229:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000001:3.0:1788880956.033231:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880956.033232:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880956.033249:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880956.033252:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880956.033252:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880956.033253:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880956.033254:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880956.094310:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880956.094317:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880956.094319:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880956.094321:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880956.094325:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880956.094327:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880956.094328:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880956.094330:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880956.094332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880956.094333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880956.094382:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880956.094385:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880956.094387:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880957.118343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880957.118350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880957.118355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880957.118356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880957.118413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880957.118417:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880957.118419:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880958.142349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880958.142356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880958.142361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880958.142362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880958.142420:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880958.142425:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880958.142428:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880959.166347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880959.166354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880959.166360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880959.166363:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880959.166365:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880959.166368:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880959.166371:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880959.166374:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880959.166376:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880959.166378:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880959.166431:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880959.166435:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880959.166437:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880960.190349:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880960.190356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880960.190362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880960.190364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880960.190400:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880960.190404:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880960.190407:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788880961.150321:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306592207, at 4306592256 00000100:00000040:2.0:1788880961.150321:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306592207, at 4306592256 00000100:00000040:3.0:1788880961.150326:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306592207, at 4306592256 00000100:00000010:1.0:1788880961.150339:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f6d3b065. 00000100:00000010:3.0:1788880961.150341:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000247083fe. 00000100:00000010:2.0:1788880961.150345:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000076a63e17. 00000100:00000040:1.0:1788880961.150362:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880961.150369:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880961.150374:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002980f9ff. 00000100:00000040:0.0:1788880961.150376:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306592207, at 4306592256 00000020:00000040:1.0:1788880961.150380:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:0.0:1788880961.150385:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ccadc63b. 00000100:00000001:1.0:1788880961.150387:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880961.150392:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880961.150396:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880961.150404:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:1.0:1788880961.150411:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880961.150420:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880961.150431:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880961.150434:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880961.150440:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002980f9ff] to pc [ptlrpcd_00_01+1] req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880961.150464:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880961.150466:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880961.150470:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880961.150472:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c27d4622. 00000020:00000040:1.0:1788880961.150475:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880961.150477:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880961.150480:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880961.150481:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880961.150485:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:1.0:1788880961.150488:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880961.150491:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880961.150496:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880961.150496:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880961.150498:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880961.150502:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788880961.150502:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c27d4622] to pc [ptlrpcd_00_02+2] req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880961.150507:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880961.150516:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880961.150530:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880961.150534:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880961.150540:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880961.150544:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000100:00000001:0.0:1788880961.150547:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000020:00000040:1.0:1788880961.150549:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880961.150549:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880961.150551:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880961.150552:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880961.150554:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880961.150556:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880961.150558:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880961.150558:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880961.150560:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788880961.150562:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880961.150563:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880961.150563:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880961.150565:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880961.150568:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 02000000:00000001:0.0:1788880961.150571:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880961.150574:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880961.150580:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780699520:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880961.150584:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880961.150586:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880961.150590:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880961.150596:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880961.150597:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788880961.150597:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880961.150602:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880961.150604:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880961.150604:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880961.150606:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788880961.150609:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880961.150610:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880961.150610:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880961.150615:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880961.150617:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009330bb87. 00000100:00000040:1.0:1788880961.150620:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca5e4c87] to pc [ptlrpcd_00_03+3] req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880961.150620:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780699648:192.168.204.111@tcp:400: 02000000:00000001:0.0:1788880961.150623:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880961.150630:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:0.0:1788880961.150633:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 02000000:00000001:2.0:1788880961.150635:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880961.150638:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880961.150640:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:0.0:1788880961.150641:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 02000000:00000010:2.0:1788880961.150644:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005701f3a3. 00000100:00000001:1.0:1788880961.150646:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880961.150649:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780699520, portal 25 02000000:00000001:2.0:1788880961.150649:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880961.150651:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880961.150656:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:1.0:1788880961.150657:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:2.0:1788880961.150658:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009252dbe7. 00000100:00000001:0.0:1788880961.150660:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938061312 : -126823771490304 : ffff8ca78a26b800) 00000100:00000040:1.0:1788880961.150665:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11929) 00000400:00000010:2.0:1788880961.150666:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001e963647. 00000100:00000040:0.0:1788880961.150670:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880977 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880961.150671:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788880961.150672:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780699648, portal 10 00000100:00000001:3.0:1788880961.150676:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880961.150678:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880961.150680:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880961.150682:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938049664 : -126823771501952 : ffff8ca78a268a80) 00000100:00000001:0.0:1788880961.150689:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880961.150692:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880961.150694:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880961.150698:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880961.150704:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f43e3cd0. 00000100:00000200:0.0:1788880961.150711:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780699520, offset 0 00000100:00000001:3.0:1788880961.150724:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880961.150725:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788880961.150728:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880961.150729:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880961.150734:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880961.150736:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:2.0:1788880961.150739:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cc594c. 02000000:00000001:3.0:1788880961.150740:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880961.150744:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:2.0:1788880961.150745:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780699648, offset 0 02000000:00000001:3.0:1788880961.150745:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880961.150752:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:3.0:1788880961.150754:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780699776:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788880961.150756:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880961.150759:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788880961.150767:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880961.150767:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880961.150772:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880961.150772:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880961.150775:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880961.150779:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880961.150780:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [543:543:256:4294967295] 192.168.204.111@tcp LPNI seq info [543:543:8:4294967295] 02000000:00000010:3.0:1788880961.150783:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000039bd304c. 02000000:00000001:3.0:1788880961.150787:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880961.150791:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000008ce0b05. 00000400:00000200:0.0:1788880961.150798:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:3.0:1788880961.150798:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000800:00000200:0.0:1788880961.150812:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880961.150816:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788880961.150818:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000bd68c00a. 00000400:00000200:2.0:1788880961.150819:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788880961.150824:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788880961.150826:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788880961.150830:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788880961.150832:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788880961.150835:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [544:544:255:4294967295] 192.168.204.111@tcp LPNI seq info [544:544:7:4294967295] 00000800:00000200:0.0:1788880961.150840:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880961.150845:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000bd68c00a type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788880961.150846:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788880961.150853:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788880961.150857:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e5360691. 00000100:00000001:0.0:1788880961.150863:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880961.150868:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880961.150870:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306597327 00000100:00000200:3.0:1788880961.150872:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780699776, portal 10 00000100:00000010:0.0:1788880961.150875:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000751d0eed. 00000100:00000001:3.0:1788880961.150876:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880961.150879:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000001:0.0:1788880961.150881:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:2.0:1788880961.150883:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880961.150887:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306597327 00000800:00000200:2.0:1788880961.150889:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e5360691 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:3.0:1788880961.150891:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880961.150892:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000455e6647. 00000100:00000001:2.0:1788880961.150896:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880961.150897:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880961.150902:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306597327 00000100:00000040:2.0:1788880961.150904:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 161122 usecs (162 jiffies) 00000100:00000010:0.0:1788880961.150909:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e245be24. 00000100:00000001:2.0:1788880961.150912:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306592418 00000100:00000001:3.0:1788880961.150912:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880961.150913:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880961.150918:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306597327 00000100:00000001:2.0:1788880961.150919:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306597327 00000100:00000040:3.0:1788880961.150919:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788880961.150921:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e6f9f09c. 00000800:00000200:1.0:1788880961.150922:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000010:3.0:1788880961.150925:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007fe20b6f. 00000100:00000001:0.0:1788880961.150927:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880961.150927:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306592418 00000800:00000010:1.0:1788880961.150929:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000bd68c00a. 00000100:00000200:3.0:1788880961.150931:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780699776, offset 0 00000100:00000001:0.0:1788880961.150933:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880961.150933:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306597327 00000100:00000001:0.0:1788880961.150937:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880961.150937:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880961.150940:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306592418 00000400:00000200:3.0:1788880961.150943:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880961.150945:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880961.150949:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880961.150950:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306597327 00000400:00000200:1.0:1788880961.150951:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880961.150954:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788880961.150958:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880961.150958:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306592418 00000100:00000001:0.0:1788880961.150961:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:3.0:1788880961.150961:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788880961.150964:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306597327 00000100:00000001:0.0:1788880961.150967:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788880961.150969:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880961.150970:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880961.150970:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880961.150974:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880961.150974:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880961.150975:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880961.150981:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880961.150982:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880961.150983:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880961.150988:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [545:545:254:4294967295] 192.168.204.111@tcp LPNI seq info [545:545:6:4294967295] 00000100:00000001:2.0:1788880961.150989:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880961.150996:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880961.151011:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:3.0:1788880961.151012:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880961.151015:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880961.151018:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788880961.151021:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f43e3cd0 00000100:00000001:2.0:1788880961.151022:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880961.151022:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880961.151026:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f43e3cd0. 00000100:00000001:2.0:1788880961.151029:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880961.151030:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000001:1.0:1788880961.151033:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:3.0:1788880961.151038:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:1.0:1788880961.151041:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880977 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788880961.151048:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788880961.151054:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880961.151061:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880961.151062:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880961.151068:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880977 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788880961.151071:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11916 usecs (12 jiffies) 00000100:00000001:3.0:1788880961.151077:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306592268 00000100:00000001:3.0:1788880961.151082:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306597327 00000100:00000001:1.0:1788880961.151084:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880961.151086:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306592268 00000100:00000001:1.0:1788880961.151088:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880961.151089:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306597327 00000100:00000001:3.0:1788880961.151092:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306592268 00000100:00000001:3.0:1788880961.151095:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306597327 00000100:00000001:3.0:1788880961.151097:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306592268 00000100:00000001:3.0:1788880961.151099:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306597327 00000100:00000001:3.0:1788880961.151101:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788880961.151104:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880961.151104:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880961.151107:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880961.151109:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e5360691. 00000100:00000001:3.0:1788880961.151111:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880961.151114:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880961.151117:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788880961.151120:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880961.151121:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:1.0:1788880961.151124:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cc594c 00000100:00000001:3.0:1788880961.151126:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1788880961.151127:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cc594c. 00000100:00000001:3.0:1788880961.151129:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880961.151133:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788880961.151133:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880961.151138:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880961.151139:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880961.151149:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880961.151151:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880961.151157:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880961.151159:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880961.151193:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880961.151195:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:1.0:1788880961.151198:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880961.151201:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880961.151204:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007fe20b6f 00000400:00000010:1.0:1788880961.151205:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007fe20b6f. 00000100:00000001:1.0:1788880961.151207:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880961.151210:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880961.151218:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880961.151220:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880961.151226:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880961.151227:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880961.152789:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880961.152800:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880961.152803:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880961.152805:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880961.152811:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880961.152822:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b10780 00000400:00000200:1.0:1788880961.152829:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x11c1 [1] + 224 00000800:00000001:1.0:1788880961.152834:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880961.152839:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880961.152841:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880961.152845:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880961.152850:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880961.152854:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880961.152864:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880961.152879:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880961.152885:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880961.152888:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.152915:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880961.152918:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880961.152922:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880961.152924:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880961.152932:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000400:00000200:0.0:1788880961.152935:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880961.152937:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788880961.152940:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880961.152944:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880961.152952:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880961.152958:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880961.152959:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.152961:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880961.152963:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880961.152966:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880961.152968:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880961.152970:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880961.152975:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=39 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880961.152979:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=39 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880961.152981:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880961.152983:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880961.152985:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880961.152987:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880961.152990:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880961.152993:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880961.153012:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880961.153015:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880961.153017:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880961.153020:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880961.153027:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780699520:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880961.153032:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.153034:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880961.153036:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002980f9ff x1875776780699520/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880961.153047:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880961.153050:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880961.153052:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009330bb87. 02000000:00000001:0.0:1788880961.153055:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880961.153056:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880961.153058:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880961.153060:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880961.153063:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 02000000:00000001:0.0:1788880961.153068:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880961.153069:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880961.153070:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002980f9ff. 00000100:00000001:0.0:1788880961.153074:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880961.153076:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.153077:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.153081:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880961.153083:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880961.153085:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880961.153087:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880961.153106:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880961.153111:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880961.153113:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880961.153114:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880961.153118:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880961.153123:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10800 00000400:00000200:2.0:1788880961.153126:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x11c5 [1] + 224 00000800:00000001:2.0:1788880961.153134:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880961.153138:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880961.153139:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880961.153141:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880961.153143:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880961.153146:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880961.153151:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880961.153162:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880961.153165:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880961.153166:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.153190:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880961.153193:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880961.153196:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880961.153197:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880961.153199:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009252dbe7. 00000400:00000200:0.0:1788880961.153202:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001e963647 00000400:00000010:0.0:1788880961.153204:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001e963647. 00000100:00000001:0.0:1788880961.153206:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880961.153211:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880961.153219:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880961.153226:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880961.153227:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.153228:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880961.153230:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880961.153231:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:1.0:1788880961.153251:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880961.153257:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.153259:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880961.153262:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:0.0:1788880961.153263:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880961.153266:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880961.153269:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=29 v=5 (5 5 5 5) 00000400:00000200:1.0:1788880961.153275:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:0.0:1788880961.153277:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=29 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880961.153281:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880961.153283:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880961.153286:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10880 00010000:00000001:0.0:1788880961.153287:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880961.153292:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880961.153295:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x11d1 [1] + 224 00000100:00000001:0.0:1788880961.153298:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880961.153301:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:1.0:1788880961.153302:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880961.153305:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880961.153307:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880961.153309:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880961.153314:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880961.153314:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880961.153321:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880961.153324:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880961.153328:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880961.153342:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880961.153348:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000040:1.0:1788880961.153348:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880961.153351:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880961.153358:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880961.153379:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780699648:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880961.153383:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880961.153384:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.153388:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:1.0:1788880961.153390:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1788880961.153391:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c27d4622 x1875776780699648/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788880961.153395:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.153403:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880961.153405:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880961.153406:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:0.0:1788880961.153407:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005701f3a3. 00000100:00000001:3.0:1788880961.153410:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:0.0:1788880961.153412:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000001:3.0:1788880961.153415:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000001:0.0:1788880961.153416:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880961.153419:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000001:3.0:1788880961.153419:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880961.153423:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000008ce0b05. 00000020:00000001:0.0:1788880961.153424:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880961.153428:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 00000400:00000200:3.0:1788880961.153429:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 02000000:00000001:0.0:1788880961.153432:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000010:3.0:1788880961.153432:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000100:00000001:3.0:1788880961.153436:0:201288:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:0.0:1788880961.153439:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880961.153442:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c27d4622. 00000100:00000200:3.0:1788880961.153442:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880961.153453:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000200:3.0:1788880961.153455:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880961.153456:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.153460:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880961.153466:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880961.153467:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880961.153468:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880961.153471:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880961.153472:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880961.153476:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880961.153479:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880961.153481:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880961.153483:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880961.153485:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880961.153487:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880961.153491:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=39 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880961.153494:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=123 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880961.153496:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880961.153497:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880961.153499:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880961.153511:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880961.153519:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880961.153521:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880961.153523:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880961.153524:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880961.153525:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880961.153528:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880961.153534:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880961.153537:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880961.153538:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880961.153540:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880961.153548:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780699776:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880961.153553:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880961.153554:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880961.153557:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780699776/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880977 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880961.153563:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880961.153565:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880961.153570:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000039bd304c. 02000000:00000001:3.0:1788880961.153574:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880961.153576:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880961.153577:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880961.153579:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880961.153580:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 02000000:00000001:3.0:1788880961.153583:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880961.153584:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880961.153585:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00000001:3.0:1788880961.153588:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880961.153589:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880961.153591:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880961.153594:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880961.153596:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880961.153597:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880961.153599:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880961.214323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880961.214328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880961.214332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880961.214333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880961.214375:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880961.214378:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880961.214380:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880962.238370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880962.238376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880962.238380:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880962.238381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880962.238419:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880962.238426:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880962.238431:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880963.262345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880963.262351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880963.262354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880963.262355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880963.262406:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880963.262410:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880963.262413:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880964.286330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880964.286336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880964.286340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880964.286341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880964.286364:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880964.286368:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880964.286370:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880965.310384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880965.310390:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880965.310392:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880965.310394:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880965.310398:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880965.310400:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880965.310401:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880965.310403:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880965.310405:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880965.310406:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880965.310451:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880965.310455:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880965.310457:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880966.270389:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306597327, at 4306597376 00000100:00000040:3.0:1788880966.270389:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306597327, at 4306597376 00000100:00000040:1.0:1788880966.270390:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788880966.270398:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306597327, at 4306597376 00000100:00000001:1.0:1788880966.270402:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788880966.270405:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e6f9f09c. 00000100:00000010:0.0:1788880966.270407:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000751d0eed. 00000100:00000010:1.0:1788880966.270413:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000094565713. 00000100:00000010:2.0:1788880966.270419:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e245be24. 00000020:00000040:1.0:1788880966.270420:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880966.270428:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880966.270440:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880966.270445:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880966.270454:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000009908d3b1. 00000100:00000001:1.0:1788880966.270463:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880966.270476:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880966.270486:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880966.270489:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880966.270493:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000094565713] to pc [ptlrpcd_00_00+0] req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880966.270518:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880966.270521:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880966.270524:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880966.270526:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c09231ec. 00000020:00000040:1.0:1788880966.270528:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880966.270530:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880966.270531:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880966.270533:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880966.270536:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000001:1.0:1788880966.270539:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880966.270543:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880966.270550:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880966.270551:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880966.270556:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880966.270557:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880966.270559:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880966.270561:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788880966.270562:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880966.270566:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c09231ec] to pc [ptlrpcd_00_01+1] req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880966.270578:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880966.270578:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880966.270583:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880966.270589:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880966.270590:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880966.270593:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:1.0:1788880966.270595:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000100:00000001:0.0:1788880966.270597:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:1.0:1788880966.270600:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880966.270601:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880966.270606:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880966.270606:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880966.270607:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880966.270609:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880966.270613:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880966.270613:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880966.270617:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000010:1.0:1788880966.270619:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008c752d14. 00000100:00000001:0.0:1788880966.270623:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:3.0:1788880966.270623:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880966.270624:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880966.270626:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880966.270630:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880966.270633:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788880966.270633:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788880966.270636:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780699904:192.168.204.111@tcp:400: 02000000:00000001:0.0:1788880966.270638:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880966.270645:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00100000:0.0:1788880966.270647:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780700032:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880966.270650:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788880966.270653:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:3.0:1788880966.270654:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880966.270657:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880966.270657:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880966.270658:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:0.0:1788880966.270663:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000010:3.0:1788880966.270664:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000057e92cbc. 00000100:00000040:1.0:1788880966.270665:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000077a1dd13] to pc [ptlrpcd_00_02+2] req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880966.270668:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880966.270680:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880966.270683:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880966.270687:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003a17641d. 00000400:00000010:3.0:1788880966.270687:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000008ce0b05. 02000000:00000001:0.0:1788880966.270691:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880966.270691:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880966.270697:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788880966.270697:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d59e2cad. 00000400:00000010:0.0:1788880966.270700:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000100:00000040:1.0:1788880966.270703:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:0.0:1788880966.270710:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000040:1.0:1788880966.270711:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11935) 00000100:00000200:3.0:1788880966.270711:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780699904, portal 25 00000100:00000200:0.0:1788880966.270716:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780700032, portal 10 00000100:00000001:3.0:1788880966.270718:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880966.270720:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880966.270721:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055040 : -126823771496576 : ffff8ca78a269f80) 00000100:00000001:0.0:1788880966.270725:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938056832 : -126823771494784 : ffff8ca78a26a680) 00000100:00000001:2.0:1788880966.270725:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880966.270730:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306597327, at 4306597376 00000100:00000040:3.0:1788880966.270732:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880982 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880966.270734:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:1.0:1788880966.270739:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000455e6647. 00000100:00000001:2.0:1788880966.270739:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880966.270743:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880966.270755:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880966.270756:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880966.270762:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880966.270766:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788880966.270773:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007839347f. 00000100:00000040:0.0:1788880966.270775:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880966.270783:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8884ddc. 00000100:00000200:3.0:1788880966.270784:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780699904, offset 0 00000100:00000001:2.0:1788880966.270788:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880966.270790:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880966.270791:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780700032, offset 0 02000000:00000001:2.0:1788880966.270794:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:3.0:1788880966.270799:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880966.270800:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880966.270804:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880966.270810:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780700160:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880966.270818:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880966.270818:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880966.270828:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1788880966.270830:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880966.270834:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880966.270837:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880966.270839:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880966.270841:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [546:546:256:4294967295] 192.168.204.111@tcp LPNI seq info [546:546:8:4294967295] 02000000:00000001:2.0:1788880966.270851:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:3.0:1788880966.270855:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788880966.270857:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880966.270867:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:3.0:1788880966.270867:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880966.270868:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788880966.270877:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880966.270882:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000010:3.0:1788880966.270883:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000400:00000200:0.0:1788880966.270885:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788880966.270890:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000010:2.0:1788880966.270893:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000006b7f624f. 02000000:00000001:2.0:1788880966.270908:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880966.270908:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880966.270916:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [547:547:255:4294967295] 192.168.204.111@tcp LPNI seq info [547:547:7:4294967295] 00000400:00000010:2.0:1788880966.270920:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000009a2f7e83. 00000800:00000200:3.0:1788880966.270929:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:2.0:1788880966.270932:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000800:00000200:3.0:1788880966.270935:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000200:2.0:1788880966.270941:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780700160, portal 10 00000100:00000001:2.0:1788880966.270946:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880966.270949:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000400:00000200:0.0:1788880966.270950:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880966.270960:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880966.270961:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880966.270964:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:0.0:1788880966.270967:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ae4abc27. 00000100:00000001:3.0:1788880966.270969:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306602447 00000800:00000200:0.0:1788880966.270982:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:3.0:1788880966.270982:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d6fe56. 00000100:00000001:3.0:1788880966.270996:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:2.0:1788880966.270999:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880966.271000:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880966.271007:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ae4abc27 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880966.271007:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306602447 00000100:00000040:2.0:1788880966.271014:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880966.271017:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880966.271021:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880966.271021:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000002c2d3615. 00000400:00000010:2.0:1788880966.271022:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ac9b6c42. 00000100:00000001:1.0:1788880966.271023:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880966.271029:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 160148 usecs (161 jiffies) 00000100:00000001:3.0:1788880966.271029:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000200:2.0:1788880966.271031:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780700160, offset 0 00000100:00000001:0.0:1788880966.271034:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306597537 00000100:00000001:3.0:1788880966.271039:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306602447 00000400:00000200:2.0:1788880966.271047:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788880966.271049:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d5dcfb55. 00000100:00000001:0.0:1788880966.271054:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306602447 00000100:00000001:3.0:1788880966.271055:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788880966.271060:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306597537 00000100:00000001:3.0:1788880966.271063:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306602447 00000100:00000001:0.0:1788880966.271065:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306602447 00000400:00000200:2.0:1788880966.271065:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:3.0:1788880966.271068:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000016ab1a15. 00000100:00000001:3.0:1788880966.271071:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880966.271072:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306597537 00000800:00000200:1.0:1788880966.271075:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880966.271075:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880966.271076:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306602447 00000100:00000001:3.0:1788880966.271084:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788880966.271085:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000001:0.0:1788880966.271089:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306597537 00000100:00000001:3.0:1788880966.271090:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880966.271092:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788880966.271093:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880966.271098:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306602447 00000100:00000001:3.0:1788880966.271098:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880966.271104:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880966.271106:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788880966.271109:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880966.271114:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788880966.271114:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880966.271116:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880966.271124:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880966.271125:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880966.271125:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880966.271133:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [548:548:254:4294967295] 192.168.204.111@tcp LPNI seq info [548:548:6:4294967295] 00000100:00000001:0.0:1788880966.271135:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880966.271135:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880966.271143:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880966.271146:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880966.271147:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880966.271150:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880966.271152:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880966.271161:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880966.271161:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880966.271165:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880966.271171:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880966.271175:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880966.271177:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007839347f 00000800:00000200:2.0:1788880966.271181:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788880966.271182:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007839347f. 00000100:00000001:1.0:1788880966.271193:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000010:2.0:1788880966.271195:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000007cb951ea. 00000100:00000200:1.0:1788880966.271199:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880982 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880966.271210:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000200:2.0:1788880966.271210:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:1.0:1788880966.271218:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880982 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880966.271227:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880966.271230:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880966.271231:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880966.271271:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000007cb951ea type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880966.271277:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880966.271284:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11863 usecs (12 jiffies) 00000100:00000001:2.0:1788880966.271287:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306597389 00000800:00000200:1.0:1788880966.271288:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880966.271291:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ae4abc27. 00000400:00000200:1.0:1788880966.271294:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880966.271301:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880966.271303:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8884ddc 00000400:00000010:1.0:1788880966.271305:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8884ddc. 00000100:00000001:2.0:1788880966.271305:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306602447 00000100:00000001:1.0:1788880966.271308:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880966.271313:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880966.271313:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306597389 00000100:00000001:2.0:1788880966.271322:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306602447 00000100:00000001:1.0:1788880966.271324:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788880966.271328:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306597389 00000100:00000040:1.0:1788880966.271330:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880966.271336:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306602447 00000100:00000001:2.0:1788880966.271343:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306597389 00000100:00000001:2.0:1788880966.271352:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306602447 00000100:00000001:1.0:1788880966.271360:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880966.271361:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880966.271362:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788880966.271368:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880966.271372:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880966.271374:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880966.271385:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880966.271387:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880966.271390:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880966.271392:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880966.271394:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880966.271396:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880966.271417:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880966.271419:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000007cb951ea. 00000400:00000200:1.0:1788880966.271422:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880966.271424:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880966.271425:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ac9b6c42 00000400:00000010:1.0:1788880966.271427:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ac9b6c42. 00000100:00000001:1.0:1788880966.271428:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880966.271430:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880966.271434:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880966.271436:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880966.271439:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880966.271440:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880966.273457:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880966.273471:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880966.273474:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880966.273476:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880966.273493:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880966.273507:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10980 00000400:00000200:2.0:1788880966.273515:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x11dd [1] + 224 00000800:00000001:2.0:1788880966.273522:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880966.273527:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880966.273530:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880966.273542:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880966.273550:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880966.273556:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880966.273582:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880966.273603:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880966.273614:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880966.273618:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880966.273620:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880966.273622:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880966.273626:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880966.273632:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10a00 00000400:00000200:2.0:1788880966.273637:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x11e9 [1] + 224 00000800:00000001:2.0:1788880966.273641:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880966.273651:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880966.273654:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880966.273657:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880966.273657:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880966.273662:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880966.273663:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:0.0:1788880966.273670:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000200:2.0:1788880966.273671:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788880966.273675:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880966.273679:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000100:00000040:2.0:1788880966.273686:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880966.273689:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880966.273694:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788880966.273699:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880966.273705:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880966.273711:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:0.0:1788880966.273717:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880966.273719:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880966.273724:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880966.273728:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880966.273731:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880966.273733:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880966.273742:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880966.273744:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880966.273746:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880966.273749:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880966.273755:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:0.0:1788880966.273757:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=34 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880966.273759:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:0.0:1788880966.273762:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=34 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880966.273765:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000001:3.0:1788880966.273765:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880966.273768:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788880966.273769:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00010000:00000001:0.0:1788880966.273771:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880966.273776:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880966.273776:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000009a2f7e83. 00000100:00000001:0.0:1788880966.273779:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880966.273782:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788880966.273784:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000200:3.0:1788880966.273784:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000100:00000001:0.0:1788880966.273786:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788880966.273789:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000040:0.0:1788880966.273791:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880966.273797:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880966.273799:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880966.273800:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880966.273802:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880966.273806:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780700032:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880966.273810:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880966.273811:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880966.273813:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c09231ec x1875776780700032/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880966.273816:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880966.273818:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880966.273820:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003a17641d. 02000000:00000001:0.0:1788880966.273824:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880966.273826:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880966.273827:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880966.273829:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880966.273834:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 02000000:00000001:0.0:1788880966.273838:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880966.273840:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880966.273841:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c09231ec. 00000100:00000001:0.0:1788880966.273844:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880966.273845:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880966.273846:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880966.273851:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880966.273851:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880966.273854:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880966.273855:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880966.273914:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880966.273917:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880966.273925:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880966.273932:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880966.273934:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880966.273935:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880966.273938:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880966.273942:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880966.273944:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880966.273946:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880966.273948:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880966.273951:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880966.273955:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880966.273957:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880966.273967:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880966.273978:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b10900 00000400:00000200:1.0:1788880966.273987:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x11d9 [1] + 224 00000800:00000001:1.0:1788880966.273993:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:3.0:1788880966.273997:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=44 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880966.274000:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=128 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880966.274003:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:1.0:1788880966.274004:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880966.274004:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880966.274009:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:1.0:1788880966.274013:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00010000:00080000:3.0:1788880966.274014:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880966.274030:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880966.274031:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880966.274034:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880966.274035:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880966.274038:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880966.274039:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880966.274041:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1788880966.274043:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880966.274049:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880966.274054:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:1.0:1788880966.274055:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880966.274058:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880966.274063:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880966.274093:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780700160:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880966.274105:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880966.274113:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880966.274119:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780700160/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880966.274126:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880966.274127:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880966.274129:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880966.274133:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000006b7f624f. 02000000:00000001:3.0:1788880966.274140:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880966.274144:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880966.274147:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880966.274150:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880966.274155:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008c752d14. 02000000:00000001:3.0:1788880966.274158:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880966.274159:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880966.274159:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:3.0:1788880966.274161:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880966.274162:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880966.274164:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880966.274166:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880966.274168:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880966.274170:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880966.274172:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880966.274174:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880966.274177:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880966.274184:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880966.274200:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880966.274203:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880966.274207:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880966.274209:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788880966.274211:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000008ce0b05. 00000400:00000200:2.0:1788880966.274216:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d59e2cad 00000400:00000010:2.0:1788880966.274218:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d59e2cad. 00000100:00000001:2.0:1788880966.274222:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880966.274227:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880966.274273:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880966.274280:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880966.274282:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880966.274285:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788880966.274287:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880966.274288:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788880966.274290:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880966.274292:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880966.274294:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=44 v=5 (5 5 5 5) 00000100:00001000:2.0:1788880966.274297:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=44 v=5 (1 1 1 1) 00000100:00000001:2.0:1788880966.274299:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788880966.274300:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788880966.274302:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788880966.274304:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880966.274306:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880966.274309:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880966.274315:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880966.274317:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880966.274318:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880966.274320:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788880966.274327:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780699904:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880966.274332:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880966.274334:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880966.274335:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000094565713 x1875776780699904/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880982 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880966.274339:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788880966.274340:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880966.274341:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000057e92cbc. 02000000:00000001:2.0:1788880966.274348:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880966.274349:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788880966.274350:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788880966.274351:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788880966.274353:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000009908d3b1. 02000000:00000001:2.0:1788880966.274355:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880966.274356:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788880966.274357:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000094565713. 00000100:00000001:2.0:1788880966.274359:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880966.274360:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880966.274361:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880966.274364:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880966.274365:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880966.274366:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880966.274367:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880966.334318:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880966.334324:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880966.334328:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880966.334329:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880966.334377:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880966.334381:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880966.334384:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880967.358356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880967.358365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880967.358371:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880967.358372:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880967.358447:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880967.358453:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880967.358457:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880968.382341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880968.382348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880968.382354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880968.382357:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880968.382359:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880968.382362:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880968.382365:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880968.382367:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880968.382369:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880968.382371:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880968.382414:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880968.382418:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880968.382420:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880969.406330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880969.406336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880969.406341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880969.406342:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880969.406387:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880969.406391:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880969.406393:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880970.430388:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880970.430397:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880970.430404:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880970.430406:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880970.430478:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880970.430482:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880970.430486:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880971.390318:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306602447, at 4306602496 00000100:00000040:2.0:1788880971.390318:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306602447, at 4306602496 00000100:00000040:1.0:1788880971.390323:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880971.390331:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880971.390332:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d6fe56. 00000100:00000010:2.0:1788880971.390334:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d5dcfb55. 00000100:00000010:1.0:1788880971.390338:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b5bd1bf6. 00000020:00000040:1.0:1788880971.390343:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880971.390347:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880971.390350:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880971.390354:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880971.390362:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d7d1e0d5. 00000100:00000040:3.0:1788880971.390364:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306602447, at 4306602496 00000100:00000001:1.0:1788880971.390369:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880971.390371:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000016ab1a15. 00000100:00000040:1.0:1788880971.390381:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880971.390392:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880971.390394:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880971.390399:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b5bd1bf6] to pc [ptlrpcd_00_03+3] req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880971.390425:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880971.390427:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788880971.390431:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788880971.390434:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000003b122924. 00000020:00000040:1.0:1788880971.390436:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880971.390439:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788880971.390440:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880971.390443:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880971.390448:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000a371c7dc. 00000100:00000001:2.0:1788880971.390450:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880971.390451:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880971.390457:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000040:1.0:1788880971.390458:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880971.390461:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880971.390469:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880971.390470:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880971.390471:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880971.390474:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880971.390475:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003b122924] to pc [ptlrpcd_00_00+0] req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880971.390479:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880971.390482:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880971.390485:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880971.390492:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880971.390501:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880971.390503:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880971.390504:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880971.390507:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788880971.390508:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880971.390511:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880971.390512:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880971.390516:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880971.390517:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788880971.390518:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880971.390520:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788880971.390524:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880971.390524:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:2.0:1788880971.390525:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b5bd1bf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780700288:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880971.390526:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880971.390528:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880971.390530:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000010:1.0:1788880971.390531:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000dc99d6ba. 00000100:00000001:0.0:1788880971.390533:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:1.0:1788880971.390535:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788880971.390536:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880971.390540:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880971.390541:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788880971.390542:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880971.390544:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788880971.390544:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788880971.390548:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880971.390549:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000010:1.0:1788880971.390552:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000131c1a55. 00000100:00000040:0.0:1788880971.390553:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880971.390553:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880971.390557:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880971.390565:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880971.390567:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880971.390571:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880971.390572:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:0.0:1788880971.390577:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880971.390579:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000010:2.0:1788880971.390579:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000978ba7c2. 02000000:00000001:0.0:1788880971.390581:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788880971.390583:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880971.390586:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788880971.390591:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000064bd5a64. 00000100:00100000:0.0:1788880971.390592:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780700416:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880971.390597:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:2.0:1788880971.390600:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000023cb0718. 00000100:00000001:1.0:1788880971.390601:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880971.390608:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000200:2.0:1788880971.390609:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780700288, portal 25 02000000:00000001:0.0:1788880971.390610:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880971.390610:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000dc99d6ba] to pc [ptlrpcd_00_01+1] req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880971.390615:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880971.390615:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788880971.390619:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938059520 : -126823771492096 : ffff8ca78a26b100) 02000000:00000010:0.0:1788880971.390620:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007b91b305. 02000000:00000001:0.0:1788880971.390625:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880971.390629:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000100:00000040:2.0:1788880971.390630:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880987 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880971.390633:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000001:1.0:1788880971.390634:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880971.390640:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:0.0:1788880971.390642:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780700416, portal 10 00000100:00000001:2.0:1788880971.390644:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880971.390646:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880971.390649:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880971.390650:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938062208 : -126823771489408 : ffff8ca78a26bb80) 00000100:00000040:1.0:1788880971.390651:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:2.0:1788880971.390651:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00100000:3.0:1788880971.390652:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:3.0:1788880971.390656:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788880971.390657:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005071f3f0. 00000100:00000040:1.0:1788880971.390660:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11940) 00000100:00000001:3.0:1788880971.390661:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:2.0:1788880971.390662:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780700288, offset 0 00000100:00000040:0.0:1788880971.390664:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880971.390664:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880971.390668:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880971.390671:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880971.390672:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788880971.390676:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880971.390677:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306602447, at 4306602496 00000100:00000040:3.0:1788880971.390679:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880971.390680:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:1.0:1788880971.390686:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000002c2d3615. 00000100:00000040:0.0:1788880971.390687:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880971.390692:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e0ace8ca. 00000400:00000200:2.0:1788880971.390693:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:0.0:1788880971.390697:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780700416, offset 0 00000100:00000001:3.0:1788880971.390697:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:2.0:1788880971.390702:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788880971.390706:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880971.390706:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880971.390710:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880971.390711:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880971.390714:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880971.390715:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880971.390717:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00100000:3.0:1788880971.390720:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000dc99d6ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780700544:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880971.390723:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788880971.390725:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880971.390726:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880971.390727:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880971.390729:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000784e3b9. 02000000:00000001:3.0:1788880971.390731:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880971.390733:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:3.0:1788880971.390735:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b1a6b23c. 00000400:00000200:2.0:1788880971.390740:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [549:549:256:4294967295] 192.168.204.111@tcp LPNI seq info [549:549:8:4294967295] 00000400:00000010:3.0:1788880971.390742:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cd783db0. 00000100:00000200:3.0:1788880971.390747:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780700544, portal 10 00000100:00000001:3.0:1788880971.390749:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880971.390752:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938052352 : -126823771499264 : ffff8ca78a269500) 00000400:00000200:2.0:1788880971.390757:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788880971.390762:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788880971.390766:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880971.390768:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:2.0:1788880971.390773:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f9812869. 00000400:00000200:0.0:1788880971.390776:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880971.390780:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788880971.390782:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788880971.390786:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880971.390789:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:3.0:1788880971.390791:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f73021f1. 00000800:00000200:2.0:1788880971.390792:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880971.390794:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:3.0:1788880971.390796:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780700544, offset 0 00000400:00000200:0.0:1788880971.390801:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [550:550:255:4294967295] 192.168.204.111@tcp LPNI seq info [550:550:7:4294967295] 00000400:00000200:3.0:1788880971.390816:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788880971.390817:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880971.390823:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f9812869 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880971.390826:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788880971.390833:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880971.390837:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880971.390838:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000f9f1055. 00000400:00000200:3.0:1788880971.390838:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788880971.390843:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306607567 00000400:00000200:3.0:1788880971.390845:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1788880971.390849:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:2.0:1788880971.390851:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000076a63e17. 00000400:00000200:3.0:1788880971.390851:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880971.390856:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880971.390858:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:0.0:1788880971.390859:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880971.390862:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788880971.390866:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000f9f1055 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880971.390869:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306607567 00000400:00000200:3.0:1788880971.390870:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [551:551:254:4294967295] 192.168.204.111@tcp LPNI seq info [551:551:6:4294967295] 00000100:00000001:0.0:1788880971.390871:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788880971.390876:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000668fd8a. 00000100:00000040:0.0:1788880971.390879:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 159191 usecs (160 jiffies) 00000100:00000001:2.0:1788880971.390882:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788880971.390884:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306602656 00000400:00000200:3.0:1788880971.390887:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788880971.390888:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306607567 00000800:00000200:1.0:1788880971.390889:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:3.0:1788880971.390894:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880971.390896:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306607567 00000800:00000010:1.0:1788880971.390896:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f9812869. 00000100:00000010:2.0:1788880971.390896:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000076127301. 00000800:00000010:3.0:1788880971.390898:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:0.0:1788880971.390903:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306602656 00000100:00000001:2.0:1788880971.390903:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:1.0:1788880971.390904:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788880971.390908:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306607567 00000100:00000001:0.0:1788880971.390909:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306607567 00000800:00000200:3.0:1788880971.390913:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880971.390915:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306602656 00000100:00000010:2.0:1788880971.390915:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000017cf871b. 00000400:00000200:1.0:1788880971.390916:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880971.390921:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306607567 00000100:00000001:2.0:1788880971.390922:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:3.0:1788880971.390922:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880971.390925:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005071f3f0 00000100:00000001:0.0:1788880971.390927:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306602656 00000800:00000200:3.0:1788880971.390928:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000400:00000010:1.0:1788880971.390929:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005071f3f0. 00000100:00000001:2.0:1788880971.390929:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880971.390932:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306607567 00000100:00000001:3.0:1788880971.390934:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880971.390936:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880971.390937:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880971.390938:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000040:3.0:1788880971.390939:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11816 usecs (12 jiffies) 00000100:00000001:0.0:1788880971.390942:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880971.390942:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880971.390945:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880987 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880971.390948:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880971.390948:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880971.390949:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306602508 00000100:00000001:0.0:1788880971.390955:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880971.390957:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306607567 00000100:00000001:0.0:1788880971.390960:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880971.390961:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306602508 00000100:00000001:0.0:1788880971.390964:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880971.390965:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306607567 00000100:00000001:1.0:1788880971.390967:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880971.390968:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880971.390968:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306602508 00000100:00000001:3.0:1788880971.390971:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306607567 00000100:00000001:0.0:1788880971.390973:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880971.390974:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880987 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880971.390975:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306602508 00000100:00000001:3.0:1788880971.390979:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306607567 00000100:00000001:3.0:1788880971.390982:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880971.390985:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880971.390988:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880971.390991:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880971.390992:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880971.390997:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880971.391010:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880971.391012:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000f9f1055. 00000400:00000200:1.0:1788880971.391015:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880971.391018:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880971.391021:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e0ace8ca 00000400:00000010:1.0:1788880971.391022:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e0ace8ca. 00000100:00000001:1.0:1788880971.391025:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880971.391027:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880971.391039:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880971.391041:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880971.391047:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880971.391048:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788880971.391079:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788880971.391082:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:1.0:1788880971.391084:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880971.391088:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788880971.391090:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f73021f1 00000400:00000010:1.0:1788880971.391091:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f73021f1. 00000100:00000001:1.0:1788880971.391094:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788880971.391096:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880971.391102:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880971.391104:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880971.391110:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880971.391111:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880971.392470:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880971.392480:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880971.392483:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880971.392485:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880971.392491:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880971.392500:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b10a80 00000400:00000200:2.0:1788880971.392507:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x11f1 [1] + 224 00000800:00000001:2.0:1788880971.392513:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880971.392518:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880971.392520:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880971.392523:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880971.392527:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880971.392531:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880971.392540:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880971.392554:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880971.392560:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880971.392565:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880971.392568:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880971.392572:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880971.392573:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880971.392575:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1788880971.392576:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:2.0:1788880971.392579:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788880971.392580:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000064bd5a64. 00000400:00000200:2.0:1788880971.392586:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880971.392586:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000023cb0718 00000400:00000010:3.0:1788880971.392590:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000023cb0718. 00000100:00000001:3.0:1788880971.392593:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880971.392596:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10b00 00000100:00000200:3.0:1788880971.392600:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880971.392604:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x11f5 [1] + 224 00000800:00000001:2.0:1788880971.392612:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880971.392615:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880971.392619:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:3.0:1788880971.392619:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880971.392626:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880971.392632:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880971.392633:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880971.392636:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880971.392640:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880971.392644:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:2.0:1788880971.392649:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880971.392664:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880971.392668:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880971.392670:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880971.392672:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880971.392674:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880971.392677:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880971.392682:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10b80 00000400:00000200:2.0:1788880971.392686:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1201 [1] + 224 00000800:00000001:2.0:1788880971.392689:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880971.392694:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880971.392695:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880971.392697:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880971.392697:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880971.392699:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880971.392701:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880971.392702:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880971.392705:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:0.0:1788880971.392706:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:3.0:1788880971.392706:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880971.392708:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880971.392710:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000400:00000200:0.0:1788880971.392713:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880971.392714:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00001000:3.0:1788880971.392714:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=49 v=5 (5 5 5 5) 00000100:00000001:0.0:1788880971.392717:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880971.392718:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:3.0:1788880971.392718:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=49 v=5 (1 1 1 1) 00000100:00000200:0.0:1788880971.392721:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880971.392723:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880971.392724:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880971.392727:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000200:2.0:1788880971.392730:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880971.392730:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880971.392731:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880971.392733:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880971.392738:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880971.392757:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880971.392759:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880971.392759:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880971.392762:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:2.0:1788880971.392762:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880971.392762:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:0.0:1788880971.392765:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880971.392765:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880971.392766:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:3.0:1788880971.392770:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880971.392774:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880971.392778:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880971.392783:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=39 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880971.392786:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=39 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880971.392788:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00100000:3.0:1788880971.392788:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b5bd1bf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780700288:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880971.392789:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880971.392792:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880971.392794:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788880971.392795:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880971.392797:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880971.392798:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880971.392799:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000040:3.0:1788880971.392800:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b5bd1bf6 x1875776780700288/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880971.392815:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880971.392816:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788880971.392819:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880971.392820:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000800:00000001:2.0:1788880971.392823:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788880971.392823:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:0.0:1788880971.392826:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788880971.392827:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000978ba7c2. 00000800:00000001:2.0:1788880971.392828:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880971.392831:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880971.392834:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880971.392835:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000020:00000040:3.0:1788880971.392836:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880971.392837:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880971.392838:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:3.0:1788880971.392838:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880971.392840:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d7d1e0d5. 00000100:00000001:1.0:1788880971.392841:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880971.392842:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880971.392845:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788880971.392847:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000400:00000001:1.0:1788880971.392849:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880971.392851:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00100000:0.0:1788880971.392854:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780700416:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880971.392854:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000010:1.0:1788880971.392855:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b1a6b23c. 00000100:00000010:3.0:1788880971.392857:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b5bd1bf6. 00000100:00000001:0.0:1788880971.392861:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880971.392861:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cd783db0 00000100:00000001:3.0:1788880971.392862:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000400:00000010:1.0:1788880971.392864:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cd783db0. 00000100:00000001:0.0:1788880971.392865:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788880971.392865:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880971.392869:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000003b122924 x1875776780700416/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880971.392869:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880971.392870:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880971.392874:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880971.392876:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880971.392877:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880971.392878:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788880971.392879:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880971.392880:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880971.392882:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880971.392883:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007b91b305. 02000000:00000001:0.0:1788880971.392888:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880971.392891:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000200:1.0:1788880971.392891:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1788880971.392894:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880971.392900:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880971.392903:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000a371c7dc. 00000100:00000001:1.0:1788880971.392904:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880971.392907:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880971.392909:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788880971.392909:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880971.392910:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880971.392911:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880971.392912:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880971.392913:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880971.392916:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00001000:1.0:1788880971.392918:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=49 v=5 (5 5 5 5) 00000100:00000010:0.0:1788880971.392919:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000003b122924. 00000100:00001000:1.0:1788880971.392922:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=133 v=5 (1 0 0 0) 00000100:00000001:0.0:1788880971.392924:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880971.392926:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880971.392927:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880971.392927:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880971.392931:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788880971.392931:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788880971.392935:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880971.392936:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880971.392939:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880971.392942:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:1.0:1788880971.392943:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880971.392945:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880971.392946:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880971.392948:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880971.392950:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880971.392951:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880971.392953:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880971.392957:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880971.392958:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880971.392959:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880971.392961:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880971.392965:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000dc99d6ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780700544:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880971.392972:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880971.392973:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880971.392974:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000dc99d6ba x1875776780700544/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880987 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880971.392977:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880971.392978:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880971.392979:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000784e3b9. 02000000:00000001:1.0:1788880971.392981:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880971.392982:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880971.392983:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880971.392984:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880971.392985:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000131c1a55. 02000000:00000001:1.0:1788880971.392987:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880971.392988:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880971.392988:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000dc99d6ba. 00000100:00000001:1.0:1788880971.392990:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880971.392991:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880971.392992:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880971.392994:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880971.392995:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880971.392996:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880971.392997:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880971.454343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880971.454348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880971.454353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880971.454354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880971.454413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880971.454417:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880971.454420:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880972.478376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880972.478386:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880972.478395:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880972.478396:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880972.478451:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880972.478455:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880972.478457:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880973.502321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880973.502329:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880973.502335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880973.502336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880973.502391:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880973.502394:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880973.502396:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880974.526308:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880974.526313:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880974.526317:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880974.526318:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880974.526379:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880974.526383:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880974.526386:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880975.550299:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880975.550308:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880975.550311:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880975.550314:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880975.550319:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880975.550320:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880975.550323:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880975.550326:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880975.550330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880975.550331:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880975.550409:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880975.550412:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880975.550414:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788880976.510394:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306607567, at 4306607616 00000100:00000040:0.0:1788880976.510395:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306607567, at 4306607616 00000100:00000040:1.0:1788880976.510405:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306607567, at 4306607616 00000100:00000040:2.0:1788880976.510412:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:3.0:1788880976.510412:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000017cf871b. 00000100:00000010:1.0:1788880976.510414:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000668fd8a. 00000100:00000010:0.0:1788880976.510420:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000076a63e17. 00000100:00000001:2.0:1788880976.510426:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880976.510439:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cc13c26b. 00000020:00000040:2.0:1788880976.510445:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880976.510451:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880976.510456:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880976.510463:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880976.510473:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:2.0:1788880976.510483:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880976.510496:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880976.510506:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880976.510509:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880976.510516:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cc13c26b] to pc [ptlrpcd_00_02+2] req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880976.510547:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880976.510549:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880976.510552:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880976.510554:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000020:00000040:2.0:1788880976.510556:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880976.510558:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880976.510560:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880976.510562:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880976.510566:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000100:00000001:2.0:1788880976.510571:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880976.510575:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880976.510580:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880976.510583:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788880976.510585:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:3.0:1788880976.510589:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880976.510590:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880976.510610:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880976.510615:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880976.510616:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880976.510617:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880976.510624:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880976.510627:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_03+3] req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880976.510628:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880976.510640:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880976.510644:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880976.510646:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880976.510650:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880976.510653:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000005a8bb1de. 00000100:00000040:1.0:1788880976.510654:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:2.0:1788880976.510656:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880976.510660:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788880976.510661:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788880976.510664:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880976.510668:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880976.510670:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 02000000:00000010:2.0:1788880976.510676:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5d1e8ed. 00000100:00000001:3.0:1788880976.510683:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880976.510686:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880976.510694:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788880976.510698:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880976.510700:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880976.510701:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880976.510702:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880976.510716:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880976.510716:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880976.510721:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880976.510726:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880976.510730:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a8bb1de] to pc [ptlrpcd_00_00+0] req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880976.510730:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788880976.510732:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880976.510739:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788880976.510740:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880976.510751:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880976.510756:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880976.510764:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780700672:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880976.510765:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788880976.510774:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880976.510778:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880976.510782:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880976.510786:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880976.510790:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880976.510791:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880976.510791:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:3.0:1788880976.510792:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880976.510798:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880976.510799:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788880976.510802:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880976.510802:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11945) 02000000:00000001:1.0:1788880976.510807:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880976.510807:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880976.510818:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788880976.510823:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306607567, at 4306607616 00000100:00100000:3.0:1788880976.510827:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780700800:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880976.510829:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000010:2.0:1788880976.510832:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000076127301. 00000100:00000001:0.0:1788880976.510835:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788880976.510835:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009a0ddd94. 00000100:00000001:3.0:1788880976.510836:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788880976.510847:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880976.510850:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880976.510854:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788880976.510855:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b43de041. 02000000:00000001:3.0:1788880976.510856:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1788880976.510857:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788880976.510866:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880976.510867:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880976.510871:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005377d7ad. 02000000:00000001:0.0:1788880976.510872:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880976.510873:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880976.510879:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a2bc2248. 00000100:00100000:0.0:1788880976.510880:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780700928:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788880976.510888:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880976.510891:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788880976.510896:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000200:1.0:1788880976.510896:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780700672, portal 25 00000400:00000010:3.0:1788880976.510897:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005881b9ef. 02000000:00000001:0.0:1788880976.510902:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880976.510905:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:0.0:1788880976.510907:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788880976.510910:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938433664 : -126823771117952 : ffff8ca78a2c6680) 00000400:00000010:3.0:1788880976.510911:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8336eb2. 02000000:00000010:0.0:1788880976.510918:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008d432156. 00000100:00000200:3.0:1788880976.510922:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780700800, portal 10 02000000:00000001:0.0:1788880976.510924:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880976.510924:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880992 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880976.510934:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000010:0.0:1788880976.510935:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000100:00000001:3.0:1788880976.510940:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 00000400:00000010:0.0:1788880976.510947:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000001:1.0:1788880976.510961:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880976.510962:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880976.510963:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780700928, portal 10 00000100:00000040:1.0:1788880976.510974:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880976.510975:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000010:1.0:1788880976.510982:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000057cfb06f. 00000100:00000001:0.0:1788880976.510983:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938427392 : -126823771124224 : ffff8ca78a2c4e00) 00000100:00000200:1.0:1788880976.510994:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780700672, offset 0 00000100:00000001:3.0:1788880976.510996:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880976.511002:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880976.511014:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880976.511023:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880976.511028:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:3.0:1788880976.511031:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007e19e793. 00000100:00000040:0.0:1788880976.511032:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880976.511037:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:0.0:1788880976.511038:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005c796512. 00000100:00000200:3.0:1788880976.511038:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780700800, offset 0 00000400:00000200:1.0:1788880976.511046:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:0.0:1788880976.511048:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780700928, offset 0 00000400:00000200:1.0:1788880976.511057:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880976.511061:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880976.511067:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880976.511071:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880976.511076:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880976.511077:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [552:552:256:4294967295] 192.168.204.111@tcp LPNI seq info [552:552:8:4294967295] 00000400:00000200:1.0:1788880976.511095:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880976.511099:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880976.511103:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000da26fd2f. 00000400:00000200:3.0:1788880976.511108:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880976.511112:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880976.511122:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:1.0:1788880976.511127:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880976.511132:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000da26fd2f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880976.511132:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880976.511142:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880976.511146:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880976.511149:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [553:553:255:4294967295] 192.168.204.111@tcp LPNI seq info [553:553:7:4294967295] 00000100:00000001:1.0:1788880976.511151:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880976.511159:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306612687 00000100:00000010:1.0:1788880976.511164:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000400:00000200:3.0:1788880976.511172:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880976.511173:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1788880976.511183:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306612687 00000100:00000010:1.0:1788880976.511189:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000400:00000200:0.0:1788880976.511191:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788880976.511191:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880976.511195:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788880976.511202:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788880976.511203:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306612687 00000800:00000010:3.0:1788880976.511205:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000100:00000010:1.0:1788880976.511210:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000400:00000200:0.0:1788880976.511213:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788880976.511217:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:0.0:1788880976.511224:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788880976.511225:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880976.511227:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306612687 00000400:00000200:0.0:1788880976.511265:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788880976.511271:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [554:554:254:4294967295] 192.168.204.111@tcp LPNI seq info [554:554:6:4294967295] 00000100:00000010:1.0:1788880976.511278:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000800:00000200:3.0:1788880976.511282:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880976.511289:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:0.0:1788880976.511290:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788880976.511291:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880976.511303:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880976.511304:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880976.511305:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880976.511315:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000005f53e16f. 00000100:00000040:3.0:1788880976.511315:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 158237 usecs (159 jiffies) 00000100:00000001:1.0:1788880976.511316:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880976.511327:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880976.511328:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306607776 00000800:00000200:0.0:1788880976.511332:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880976.511336:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880976.511338:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:0.0:1788880976.511345:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788880976.511345:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306612687 00000800:00000010:2.0:1788880976.511346:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000da26fd2f. 00000100:00000001:1.0:1788880976.511347:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:0.0:1788880976.511350:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000005f53e16f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880976.511353:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306607776 00000100:00000001:1.0:1788880976.511354:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880976.511354:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880976.511357:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880976.511365:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306612687 00000100:00000040:0.0:1788880976.511367:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11760 usecs (12 jiffies) 00000100:00000001:1.0:1788880976.511367:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880976.511370:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306607776 00000400:00000200:2.0:1788880976.511371:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880976.511378:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306607629 00000100:00000001:3.0:1788880976.511378:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306612687 00000100:00000001:1.0:1788880976.511380:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880976.511384:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000057cfb06f 00000100:00000001:0.0:1788880976.511391:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306612687 00000100:00000001:3.0:1788880976.511392:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306607776 00000400:00000010:2.0:1788880976.511395:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000057cfb06f. 00000100:00000001:0.0:1788880976.511397:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306607629 00000100:00000001:3.0:1788880976.511399:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306612687 00000100:00000001:3.0:1788880976.511403:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880976.511404:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880976.511408:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306612687 00000100:00000001:3.0:1788880976.511409:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880976.511413:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880992 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880976.511414:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306607629 00000100:00000001:3.0:1788880976.511415:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880976.511417:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306612687 00000100:00000001:0.0:1788880976.511421:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306607629 00000100:00000001:3.0:1788880976.511421:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880976.511425:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306612687 00000100:00000001:0.0:1788880976.511427:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880976.511427:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880976.511429:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880976.511430:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880992 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880976.511431:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880976.511432:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880976.511436:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880976.511436:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880976.511438:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880976.511438:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788880976.511440:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880976.511442:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880976.511442:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880976.511444:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880976.511453:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880976.511455:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:2.0:1788880976.511458:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880976.511461:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880976.511463:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007e19e793 00000400:00000010:2.0:1788880976.511465:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007e19e793. 00000100:00000001:2.0:1788880976.511467:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880976.511470:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880976.511476:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880976.511479:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880976.511485:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880976.511487:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880976.511523:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880976.511525:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000005f53e16f. 00000400:00000200:2.0:1788880976.511528:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880976.511530:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880976.511532:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005c796512 00000400:00000010:2.0:1788880976.511537:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005c796512. 00000100:00000001:2.0:1788880976.511540:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880976.511542:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880976.511546:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880976.511548:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880976.511552:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880976.511552:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788880976.512905:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880976.512919:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880976.512921:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880976.512923:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880976.512929:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880976.512938:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b10c00 00000400:00000200:1.0:1788880976.512944:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1209 [1] + 224 00000800:00000001:1.0:1788880976.512949:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880976.512957:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880976.512959:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788880976.512963:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880976.512967:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880976.512971:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880976.512977:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880976.512988:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880976.512994:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880976.512998:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880976.512999:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880976.513000:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880976.513003:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788880976.513010:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10c80 00000100:00000001:3.0:1788880976.513013:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880976.513014:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x120d [1] + 224 00000100:00000001:3.0:1788880976.513019:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:1.0:1788880976.513020:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880976.513031:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880976.513032:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:1.0:1788880976.513034:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:3.0:1788880976.513038:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:1.0:1788880976.513042:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:3.0:1788880976.513150:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b43de041. 00000400:00000200:3.0:1788880976.513156:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005377d7ad 00000400:00000010:3.0:1788880976.513158:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005377d7ad. 00000100:00000001:3.0:1788880976.513161:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880976.513162:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880976.513170:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880976.513172:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880976.513190:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880976.513206:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788880976.513209:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880976.513213:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788880976.513215:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788880976.513218:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880976.513263:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880976.513266:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788880976.513267:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788880976.513277:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788880976.513282:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000200:3.0:1788880976.513283:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880976.513285:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10d00 00000400:00000010:2.0:1788880976.513286:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005881b9ef. 00000400:00000200:2.0:1788880976.513292:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8336eb2 00000400:00000010:2.0:1788880976.513296:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8336eb2. 00000100:00000001:3.0:1788880976.513309:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880976.513310:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880976.513313:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788880976.513316:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1211 [1] + 224 00000100:00000200:2.0:1788880976.513317:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880976.513325:0:201288:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:1.0:1788880976.513329:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788880976.513331:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880976.513336:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:1.0:1788880976.513339:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788880976.513342:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880976.513343:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880976.513346:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:3.0:1788880976.513355:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880976.513362:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880976.513363:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788880976.513365:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788880976.513367:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=54 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880976.513369:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788880976.513370:0:201278:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:2.0:1788880976.513373:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788880976.513380:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00001000:3.0:1788880976.513381:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=54 v=5 (1 1 1 1) 00000100:00000200:1.0:1788880976.513382:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880976.513383:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880976.513387:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788880976.513398:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=44 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880976.513403:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00001000:2.0:1788880976.513410:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=44 v=5 (1 1 1 1) 00000100:00000001:3.0:1788880976.513410:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880976.513418:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880976.513419:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880976.513421:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00010000:00000001:3.0:1788880976.513426:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880976.513427:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880976.513436:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:2.0:1788880976.513441:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788880976.513441:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880976.513445:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:2.0:1788880976.513451:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788880976.513453:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880976.513456:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880976.513461:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788880976.513468:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880976.513472:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880976.513473:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880976.513481:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880976.513482:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880976.513483:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:0.0:1788880976.513492:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788880976.513495:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000001:0.0:1788880976.513496:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:3.0:1788880976.513497:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880976.513503:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000100:00000001:2.0:1788880976.513503:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880976.513514:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880976.513522:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000040:2.0:1788880976.513524:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880976.513529:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:3.0:1788880976.513531:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780700672:192.168.204.111@tcp:400: 00000100:00000200:0.0:1788880976.513541:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880976.513549:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880976.513560:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880976.513570:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cc13c26b x1875776780700672/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880976.513574:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880976.513585:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880976.513590:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880976.513606:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880976.513609:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880976.513614:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880976.513620:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880976.513625:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009a0ddd94. 02000000:00000001:3.0:1788880976.513641:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880976.513649:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788880976.513650:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000040:3.0:1788880976.513654:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880976.513655:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:2.0:1788880976.513655:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780700800:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880976.513665:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880976.513667:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:3.0:1788880976.513667:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788880976.513672:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000010:3.0:1788880976.513674:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 00000100:00000001:0.0:1788880976.513675:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880976.513675:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880976.513685:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880976.513685:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:3.0:1788880976.513686:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880976.513693:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:2.0:1788880976.513694:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780700800/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788880976.513696:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=54 v=5 (5 5 5 5) 00000100:00000010:3.0:1788880976.513700:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cc13c26b. 00000100:00000001:3.0:1788880976.513716:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880976.513719:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788880976.513722:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=138 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880976.513732:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880976.513733:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880976.513737:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880976.513739:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00010000:00000001:0.0:1788880976.513741:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:2.0:1788880976.513742:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788880976.513746:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a2bc2248. 00000100:00000001:3.0:1788880976.513750:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00080000:0.0:1788880976.513751:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880976.513751:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788880976.513756:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788880976.513756:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788880976.513761:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788880976.513769:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880976.513769:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:0.0:1788880976.513771:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880976.513773:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 00000100:00000001:0.0:1788880976.513775:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788880976.513784:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880976.513785:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880976.513786:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788880976.513791:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788880976.513796:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000010:2.0:1788880976.513796:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000100:00000001:0.0:1788880976.513799:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880976.513801:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788880976.513804:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880976.513807:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880976.513808:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880976.513815:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880976.513816:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880976.513817:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788880976.513818:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880976.513820:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880976.513821:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788880976.513824:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880976.513825:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880976.513829:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780700928:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880976.513835:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880976.513837:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880976.513838:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000005a8bb1de x1875776780700928/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880992 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880976.513843:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880976.513844:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880976.513845:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008d432156. 02000000:00000001:0.0:1788880976.513848:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880976.513849:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880976.513850:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788880976.513856:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880976.513857:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5d1e8ed. 02000000:00000001:0.0:1788880976.513859:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880976.513860:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880976.513861:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000005a8bb1de. 00000100:00000001:0.0:1788880976.513862:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880976.513863:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880976.513864:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880976.513867:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880976.513868:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880976.513869:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880976.513871:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880976.574331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880976.574336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880976.574342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880976.574343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880976.574420:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880976.574427:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880976.574432:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880977.598326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880977.598332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880977.598336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880977.598337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880977.598386:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880977.598389:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880977.598391:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880978.622363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880978.622370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880978.622376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880978.622379:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880978.622381:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880978.622384:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880978.622387:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880978.622389:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880978.622390:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880978.622391:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880978.622429:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880978.622433:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880978.622436:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880979.646393:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880979.646403:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880979.646411:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880979.646412:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880979.646479:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880979.646486:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880979.646490:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880980.670339:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880980.670345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880980.670351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880980.670352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880980.670373:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880980.670378:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880980.670381:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880981.630363:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306612687, at 4306612736 00000100:00000040:3.0:1788880981.630363:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306612687, at 4306612736 00000100:00000040:1.0:1788880981.630364:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306612687, at 4306612736 00000100:00000040:2.0:1788880981.630364:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306612687, at 4306612736 00000100:00000010:3.0:1788880981.630385:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000010:1.0:1788880981.630390:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000010:0.0:1788880981.630393:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000010:2.0:1788880981.630398:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000040:2.0:1788880981.630426:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880981.630431:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880981.630438:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000020:00000040:2.0:1788880981.630442:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880981.630445:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880981.630450:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880981.630455:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880981.630461:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000100:00000001:2.0:1788880981.630467:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880981.630475:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880981.630482:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880981.630484:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880981.630488:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_01+1] req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880981.630509:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880981.630511:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880981.630513:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880981.630515:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 00000020:00000040:2.0:1788880981.630517:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880981.630518:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880981.630520:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880981.630521:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880981.630524:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008eccda10. 00000100:00000001:2.0:1788880981.630538:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880981.630540:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880981.630545:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880981.630546:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880981.630546:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880981.630553:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880981.630555:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880981.630555:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880981.630560:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_02+2] req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880981.630561:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880981.630572:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880981.630580:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880981.630581:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788880981.630583:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788880981.630584:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880981.630586:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880981.630589:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000629edf55. 02000000:00000001:0.0:1788880981.630590:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788880981.630591:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880981.630592:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:0.0:1788880981.630598:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00100000:1.0:1788880981.630599:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:2.0:1788880981.630599:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788880981.630615:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788880981.630617:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788880981.630617:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788880981.630619:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880981.630624:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880981.630625:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880981.630625:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880981.630628:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780701056:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880981.630631:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:2.0:1788880981.630636:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 00000100:00000001:1.0:1788880981.630638:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880981.630641:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880981.630642:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880981.630643:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880981.630645:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788880981.630648:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880981.630656:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880981.630656:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:1.0:1788880981.630657:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880981.630667:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880981.630690:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000c4cba31. 00000100:00000001:2.0:1788880981.630691:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788880981.630695:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880981.630697:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880981.630698:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880981.630700:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880981.630706:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880981.630706:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000629edf55] to pc [ptlrpcd_00_03+3] req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788880981.630709:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 02000000:00000001:1.0:1788880981.630710:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788880981.630718:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880981.630721:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00100000:1.0:1788880981.630728:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780701184:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880981.630730:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880981.630736:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780701056, portal 25 00000100:00000001:1.0:1788880981.630736:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880981.630736:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880981.630745:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788880981.630746:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:2.0:1788880981.630748:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880981.630751:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880981.630756:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 02000000:00000001:1.0:1788880981.630756:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880981.630761:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000040:2.0:1788880981.630762:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11950) 02000000:00000001:1.0:1788880981.630763:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880981.630772:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b1120e74. 00000100:00000001:3.0:1788880981.630773:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880981.630778:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880997 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788880981.630781:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880981.630786:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880981.630790:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:1.0:1788880981.630791:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000100:00000001:3.0:1788880981.630801:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1788880981.630804:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000100:00000001:3.0:1788880981.630805:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880981.630808:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880981.630811:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:1.0:1788880981.630815:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780701184, portal 10 00000100:00000040:0.0:1788880981.630816:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788880981.630822:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880981.630825:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000010:0.0:1788880981.630826:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a08ac001. 00000100:00000001:1.0:1788880981.630830:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000100:00000200:0.0:1788880981.630833:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780701056, offset 0 00000100:00000001:3.0:1788880981.630844:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:1.0:1788880981.630846:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880981.630850:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880981.630851:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880981.630860:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880981.630866:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880981.630870:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788880981.630872:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788880981.630881:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780701312:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788880981.630882:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880981.630883:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:1.0:1788880981.630894:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000068e03da2. 00000100:00000001:3.0:1788880981.630894:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880981.630899:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:1.0:1788880981.630903:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780701184, offset 0 02000000:00000001:3.0:1788880981.630903:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788880981.630906:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788880981.630910:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880981.630918:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880981.630919:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788880981.630926:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000046cece22. 00000400:00000200:0.0:1788880981.630928:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788880981.630934:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788880981.630934:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880981.630937:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [555:555:256:4294967295] 192.168.204.111@tcp LPNI seq info [555:555:8:4294967295] 00000400:00000010:3.0:1788880981.630942:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000df34776e. 00000400:00000010:3.0:1788880981.630948:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000400:00000200:0.0:1788880981.630952:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:3.0:1788880981.630953:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780701312, portal 10 00000100:00000001:3.0:1788880981.630958:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880981.630959:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938432768 : -126823771118848 : ffff8ca78a2c6300) 00000800:00000200:0.0:1788880981.630961:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788880981.630966:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003806e31f. 00000400:00000200:1.0:1788880981.630969:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:0.0:1788880981.630980:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880981.630980:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788880981.630980:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880981.630993:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1788880981.630995:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788880981.630997:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:3.0:1788880981.631003:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788880981.631005:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003806e31f type 1, nob 320 niov 1 nkiov 1 00000400:00000010:3.0:1788880981.631007:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005f2b805f. 00000400:00000200:1.0:1788880981.631009:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788880981.631016:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:3.0:1788880981.631016:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780701312, offset 0 00000100:00000001:0.0:1788880981.631025:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880981.631032:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880981.631034:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306617807 00000400:00000200:1.0:1788880981.631036:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [556:556:255:4294967295] 192.168.204.111@tcp LPNI seq info [556:556:7:4294967295] 00000100:00000010:0.0:1788880981.631044:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003b9d7f95. 00000100:00000001:0.0:1788880981.631051:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:1.0:1788880981.631059:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880981.631067:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306617807 00000800:00000200:1.0:1788880981.631069:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788880981.631070:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880981.631080:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:0.0:1788880981.631082:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a682425c. 00000800:00000010:1.0:1788880981.631083:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000004e0f6846. 00000400:00000200:3.0:1788880981.631084:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880981.631090:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:3.0:1788880981.631092:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880981.631099:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306617807 00000800:00000200:1.0:1788880981.631099:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880981.631100:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:0.0:1788880981.631107:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000089d2bf5f. 00000400:00000200:3.0:1788880981.631113:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [557:557:254:4294967295] 192.168.204.111@tcp LPNI seq info [557:557:6:4294967295] 00000100:00000001:0.0:1788880981.631115:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:1.0:1788880981.631122:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788880981.631126:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306617807 00000800:00000200:2.0:1788880981.631127:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:0.0:1788880981.631136:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000751d0eed. 00000800:00000200:1.0:1788880981.631137:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000004e0f6846 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880981.631137:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000010:2.0:1788880981.631138:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003806e31f. 00000100:00000001:0.0:1788880981.631146:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788880981.631156:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880981.631160:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788880981.631161:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:3.0:1788880981.631164:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880981.631168:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 157298 usecs (158 jiffies) 00000100:00000001:0.0:1788880981.631173:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880981.631179:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306612894 00000800:00000010:3.0:1788880981.631179:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000400:00000200:2.0:1788880981.631181:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788880981.631186:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880981.631191:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306617807 00000400:00000200:2.0:1788880981.631195:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a08ac001 00000100:00000001:0.0:1788880981.631196:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880981.631196:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306612894 00000400:00000010:2.0:1788880981.631197:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a08ac001. 00000100:00000001:1.0:1788880981.631207:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306617807 00000100:00000001:2.0:1788880981.631208:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880981.631209:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880981.631210:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306612894 00000100:00000001:0.0:1788880981.631215:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000200:2.0:1788880981.631216:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880997 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880981.631217:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306617807 00000100:00000001:1.0:1788880981.631226:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306612894 00000100:00000001:0.0:1788880981.631227:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880981.631270:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880981.631273:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880981.631276:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880981.631276:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306617807 00000100:00000001:1.0:1788880981.631281:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880981.631285:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880981.631293:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000200:3.0:1788880981.631293:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880981.631294:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880981.631305:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880981.631305:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880997 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880981.631310:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:3.0:1788880981.631313:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880981.631316:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880981.631322:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788880981.631322:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880981.631335:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880981.631336:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880981.631337:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880981.631349:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11711 usecs (12 jiffies) 00000100:00000001:2.0:1788880981.631350:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880981.631353:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306612749 00000100:00000001:3.0:1788880981.631357:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306617807 00000100:00000001:3.0:1788880981.631359:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306612749 00000100:00000001:3.0:1788880981.631361:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306617807 00000100:00000001:3.0:1788880981.631364:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306612749 00000100:00000001:3.0:1788880981.631366:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306617807 00000100:00000001:3.0:1788880981.631367:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306612749 00000800:00000200:2.0:1788880981.631374:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788880981.631375:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306617807 00000100:00000001:3.0:1788880981.631378:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:2.0:1788880981.631380:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000004e0f6846. 00000100:00000001:3.0:1788880981.631381:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880981.631386:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880981.631391:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788880981.631396:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880981.631401:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880981.631407:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000068e03da2 00000400:00000010:2.0:1788880981.631409:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000068e03da2. 00000100:00000001:2.0:1788880981.631412:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880981.631415:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880981.631421:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880981.631423:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880981.631427:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880981.631428:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880981.631453:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880981.631455:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:2.0:1788880981.631457:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880981.631459:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880981.631461:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005f2b805f 00000400:00000010:2.0:1788880981.631462:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005f2b805f. 00000100:00000001:2.0:1788880981.631463:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880981.631471:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880981.631475:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880981.631476:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880981.631480:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880981.631481:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880981.635416:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880981.635426:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880981.635428:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880981.635430:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880981.635437:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880981.635446:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10e80 00000400:00000200:2.0:1788880981.635451:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1231 [1] + 224 00000800:00000001:2.0:1788880981.635456:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880981.635458:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880981.635460:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880981.635462:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880981.635465:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880981.635468:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880981.635474:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880981.635488:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880981.635493:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880981.635494:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880981.635496:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880981.635498:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880981.635501:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880981.635505:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10e00 00000400:00000200:2.0:1788880981.635507:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1225 [1] + 224 00000100:00000001:3.0:1788880981.635512:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880981.635519:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880981.635520:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788880981.635534:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788880981.635534:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788880981.635543:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:3.0:1788880981.635543:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880981.635553:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000df34776e. 00000400:00000200:2.0:1788880981.635558:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880981.635560:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000010:3.0:1788880981.635563:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:2.0:1788880981.635568:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880981.635568:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880981.635579:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788880981.635580:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880981.635631:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880981.635648:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880981.635658:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880981.635663:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880981.635664:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788880981.635667:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880981.635668:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880981.635677:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880981.635682:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880981.635685:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880981.635688:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880981.635691:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880981.635693:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880981.635696:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788880981.635702:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00001000:3.0:1788880981.635702:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=59 v=5 (5 5 5 5) 00000400:00000001:1.0:1788880981.635706:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880981.635708:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000100:00001000:3.0:1788880981.635711:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=143 v=5 (1 0 0 0) 00000400:00000200:1.0:1788880981.635713:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000400:00000010:1.0:1788880981.635715:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:1.0:1788880981.635721:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880981.635722:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880981.635725:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880981.635727:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880981.635735:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000200:1.0:1788880981.635740:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00080000:3.0:1788880981.635746:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880981.635749:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788880981.635751:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880981.635752:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788880981.635754:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880981.635755:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880981.635757:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880981.635758:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880981.635763:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=49 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880981.635766:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=49 v=5 (1 1 1 1) 00000100:00000001:1.0:1788880981.635767:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880981.635768:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880981.635770:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788880981.635773:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880981.635775:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880981.635776:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880981.635778:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880981.635779:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880981.635782:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880981.635787:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880981.635789:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880981.635790:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880981.635793:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880981.635797:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780701184:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880981.635801:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880981.635802:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880981.635803:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780701184/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880981.635808:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880981.635810:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880981.635812:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b1120e74. 02000000:00000001:1.0:1788880981.635820:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880981.635826:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00010000:00000001:3.0:1788880981.635833:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788880981.635834:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000800:00000001:2.0:1788880981.635835:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880981.635846:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788880981.635847:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788880981.635847:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000010:1.0:1788880981.635852:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008eccda10. 00000800:00000001:2.0:1788880981.635852:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880981.635855:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788880981.635856:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880981.635858:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880981.635859:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 00000100:00000001:1.0:1788880981.635864:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000400:00000200:2.0:1788880981.635864:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880981.635868:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880981.635870:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880981.635874:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b10d80 00000100:00000040:3.0:1788880981.635878:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880981.635879:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788880981.635880:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1221 [1] + 224 00000100:00000001:1.0:1788880981.635883:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880981.635886:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788880981.635886:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880981.635892:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880981.635893:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880981.635899:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880981.635901:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880981.635903:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:3.0:1788880981.635907:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788880981.635913:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880981.635925:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788880981.635931:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788880981.635936:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880981.635936:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880981.635942:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880981.635956:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880981.635967:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880981.636000:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780701312:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880981.636008:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880981.636010:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880981.636010:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880981.636013:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000629edf55 x1875776780701312/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880981.636014:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880981.636021:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880981.636022:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880981.636024:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880981.636028:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000046cece22. 02000000:00000001:3.0:1788880981.636033:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880981.636037:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880981.636039:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880981.636042:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880981.636044:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 02000000:00000001:3.0:1788880981.636047:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880981.636048:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880981.636049:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000629edf55. 00000100:00000001:3.0:1788880981.636051:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880981.636052:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880981.636054:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880981.636054:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880981.636059:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880981.636059:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880981.636062:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880981.636064:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000001:0.0:1788880981.636066:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880981.636066:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788880981.636070:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880981.636074:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000400:00000200:0.0:1788880981.636079:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880981.636081:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788880981.636085:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880981.636091:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880981.636099:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880981.636106:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880981.636107:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880981.636108:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880981.636110:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880981.636117:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880981.636118:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880981.636120:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880981.636123:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=59 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880981.636125:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=59 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880981.636126:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880981.636127:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880981.636129:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880981.636130:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880981.636131:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880981.636134:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880981.636137:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880981.636138:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880981.636139:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880981.636141:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880981.636145:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780701056:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880981.636149:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880981.636150:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880981.636151:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780701056/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788880997 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880981.636155:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788880981.636156:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880981.636157:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000c4cba31. 02000000:00000001:0.0:1788880981.636160:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880981.636160:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880981.636161:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880981.636163:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880981.636164:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 02000000:00000001:0.0:1788880981.636165:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880981.636166:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880981.636168:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000100:00000001:0.0:1788880981.636174:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880981.636176:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880981.636177:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880981.636181:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880981.636182:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880981.636184:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880981.636186:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880981.694340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880981.694347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880981.694352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880981.694353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880981.694413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880981.694419:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880981.694423:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880982.718327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880982.718333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880982.718338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880982.718339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880982.718367:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880982.718372:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880982.718375:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880983.742334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880983.742341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880983.742347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880983.742348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880983.742407:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880983.742411:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880983.742413:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880984.766344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880984.766350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880984.766355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880984.766356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880984.766402:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880984.766408:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880984.766412:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880985.790327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880985.790337:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880985.790473:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880985.790478:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880985.790483:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880985.790485:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880985.790488:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880985.790492:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880985.790502:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880985.790503:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880985.790596:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880985.790599:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880985.790601:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880986.750344:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306617807, at 4306617856 00000100:00000040:1.0:1788880986.750344:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306617807, at 4306617856 00000100:00000040:2.0:1788880986.750344:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788880986.750344:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306617807, at 4306617856 00000100:00000001:2.0:1788880986.750357:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788880986.750360:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003b9d7f95. 00000100:00000010:1.0:1788880986.750363:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a682425c. 00000100:00000010:2.0:1788880986.750363:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000884e74b8. 00000020:00000040:2.0:1788880986.750372:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:3.0:1788880986.750374:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000751d0eed. 00000100:00000001:2.0:1788880986.750377:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880986.750381:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880986.750385:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880986.750392:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 00000100:00000001:2.0:1788880986.750399:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880986.750409:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880986.750418:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880986.750421:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880986.750426:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000884e74b8] to pc [ptlrpcd_00_00+0] req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880986.750448:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880986.750451:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880986.750454:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880986.750457:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002084a2b4. 00000020:00000040:2.0:1788880986.750459:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880986.750462:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880986.750463:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880986.750465:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880986.750469:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 00000100:00000001:0.0:1788880986.750471:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880986.750471:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880986.750476:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880986.750476:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880986.750480:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788880986.750488:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880986.750497:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880986.750498:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880986.750501:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788880986.750504:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788880986.750522:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880986.750523:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880986.750526:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788880986.750530:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780701440:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788880986.750533:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002084a2b4] to pc [ptlrpcd_00_01+1] req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880986.750540:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788880986.750547:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880986.750549:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880986.750550:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880986.750554:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880986.750555:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880986.750559:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 02000000:00000001:0.0:1788880986.750560:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000020:00000040:2.0:1788880986.750563:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880986.750566:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880986.750568:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000010:0.0:1788880986.750569:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d9c9c3cb. 02000000:00000001:2.0:1788880986.750571:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880986.750575:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880986.750577:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003139f6ba. 00000100:00000001:2.0:1788880986.750581:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880986.750583:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788880986.750586:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788880986.750588:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:0.0:1788880986.750590:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000100:00000001:1.0:1788880986.750593:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:0.0:1788880986.750599:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000001:2.0:1788880986.750601:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:1.0:1788880986.750604:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880986.750605:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788880986.750609:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780701440, portal 25 00000100:00000001:0.0:1788880986.750614:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:2.0:1788880986.750614:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_02+2] req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880986.750618:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938423808 : -126823771127808 : ffff8ca78a2c4000) 00000100:00000040:0.0:1788880986.750628:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881002 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880986.750648:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880986.750649:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788880986.750652:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880986.750655:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880986.750656:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788880986.750658:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000010:0.0:1788880986.750659:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009fe01c6d. 00000100:00000040:2.0:1788880986.750661:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788880986.750663:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788880986.750664:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780701440, offset 0 00000100:00000040:2.0:1788880986.750671:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:1.0:1788880986.750672:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880986.750677:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788880986.750681:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780701568:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880986.750687:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880986.750692:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11955) 00000400:00000200:0.0:1788880986.750693:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788880986.750693:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880986.750698:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880986.750702:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788880986.750703:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880986.750708:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880986.750708:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880986.750710:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009a0ddd94. 00000100:00000001:2.0:1788880986.750713:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880986.750715:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880986.750716:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788880986.750717:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:0.0:1788880986.750722:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:1.0:1788880986.750722:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000100:00000040:2.0:1788880986.750725:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880986.750729:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [558:558:256:4294967295] 192.168.204.111@tcp LPNI seq info [558:558:8:4294967295] 00000400:00000010:1.0:1788880986.750731:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000100:00000200:1.0:1788880986.750740:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780701568, portal 10 00000100:00000001:1.0:1788880986.750746:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880986.750749:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938429184 : -126823771122432 : ffff8ca78a2c5500) 00000400:00000200:0.0:1788880986.750755:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:1.0:1788880986.750764:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880986.750770:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000800:00000200:0.0:1788880986.750771:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788880986.750774:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880986.750778:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000fc8f9b7c. 02000000:00000001:2.0:1788880986.750781:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788880986.750786:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788880986.750791:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:2.0:1788880986.750792:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788880986.750801:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780701696:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880986.750805:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788880986.750806:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788880986.750812:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000fc8f9b7c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788880986.750812:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788880986.750813:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880986.750817:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788880986.750822:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880986.750824:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002b77e739. 02000000:00000001:2.0:1788880986.750826:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:1.0:1788880986.750830:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780701568, offset 0 00000100:00000001:0.0:1788880986.750833:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880986.750833:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e4881465. 00000400:00000200:1.0:1788880986.750840:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788880986.750840:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880986.750841:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306622927 00000100:00000010:0.0:1788880986.750848:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003b9d7f95. 00000400:00000010:2.0:1788880986.750850:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001a8af943. 00000100:00000001:0.0:1788880986.750853:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:1.0:1788880986.750854:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:2.0:1788880986.750857:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a3b2cf4a. 00000100:00000001:0.0:1788880986.750860:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306622927 00000400:00000200:1.0:1788880986.750863:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:2.0:1788880986.750864:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780701696, portal 10 00000100:00000010:0.0:1788880986.750866:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000535177f1. 00000100:00000001:2.0:1788880986.750869:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788880986.750871:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:1.0:1788880986.750872:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788880986.750873:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 00000400:00000200:1.0:1788880986.750879:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880986.750882:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306622927 00000400:00000200:1.0:1788880986.750886:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:2.0:1788880986.750887:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880986.750889:0:201285:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306617807 00000100:00000001:0.0:1788880986.750896:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306622927 00000400:00000200:1.0:1788880986.750896:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [559:559:255:4294967295] 192.168.204.111@tcp LPNI seq info [559:559:7:4294967295] 00000100:00000010:0.0:1788880986.750904:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000751d0eed. 00000100:00000001:0.0:1788880986.750910:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788880986.750911:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788880986.750917:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1788880986.750922:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880986.750923:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880986.750924:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880986.750925:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788880986.750932:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788880986.750932:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000fc8f9b7c. 00000400:00000010:2.0:1788880986.750933:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000006bff322d. 00000100:00000001:0.0:1788880986.750939:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:1.0:1788880986.750939:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788880986.750942:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780701696, offset 0 00000400:00000200:3.0:1788880986.750943:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880986.750948:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:1.0:1788880986.750948:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000030024c2c. 00000100:00000001:0.0:1788880986.750951:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788880986.750956:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880986.750956:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880986.750960:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:1.0:1788880986.750962:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788880986.750963:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880986.750967:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880986.750970:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:1.0:1788880986.750976:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788880986.750977:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:1.0:1788880986.750983:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000030024c2c type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788880986.750985:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880986.750986:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880986.750992:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880986.750992:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880986.750992:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788880986.750999:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788880986.751002:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 156391 usecs (157 jiffies) 00000400:00000200:2.0:1788880986.751005:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [560:560:254:4294967295] 192.168.204.111@tcp LPNI seq info [560:560:6:4294967295] 00000100:00000001:1.0:1788880986.751009:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306618013 00000100:00000001:1.0:1788880986.751016:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306622927 00000400:00000200:2.0:1788880986.751018:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880986.751021:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306618013 00000100:00000001:1.0:1788880986.751024:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306622927 00000100:00000001:1.0:1788880986.751030:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306618013 00000100:00000001:1.0:1788880986.751034:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306622927 00000800:00000200:2.0:1788880986.751035:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880986.751041:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306618013 00000800:00000010:2.0:1788880986.751042:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006aca3ff7. 00000100:00000001:1.0:1788880986.751047:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306622927 00000800:00000200:2.0:1788880986.751051:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880986.751053:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880986.751059:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880986.751059:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880986.751064:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006aca3ff7 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880986.751066:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880986.751069:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880986.751071:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788880986.751073:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11677 usecs (12 jiffies) 00000100:00000001:1.0:1788880986.751077:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880986.751078:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306617868 00000100:00000001:1.0:1788880986.751081:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880986.751083:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306622927 00000100:00000001:1.0:1788880986.751086:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880986.751090:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880986.751090:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306617868 00000100:00000001:1.0:1788880986.751093:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880986.751096:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306622927 00000100:00000001:1.0:1788880986.751098:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880986.751102:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306617868 00000100:00000001:2.0:1788880986.751106:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306622927 00000100:00000001:2.0:1788880986.751109:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306617868 00000100:00000001:2.0:1788880986.751112:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306622927 00000100:00000001:2.0:1788880986.751114:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880986.751117:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880986.751122:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880986.751127:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788880986.751127:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880986.751135:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788880986.751138:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880986.751148:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880986.751149:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880986.751151:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880986.751153:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880986.751194:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009fe01c6d 00000400:00000010:3.0:1788880986.751198:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009fe01c6d. 00000100:00000001:3.0:1788880986.751204:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880986.751209:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881002 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880986.751221:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880986.751224:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881002 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880986.751258:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880986.751261:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880986.751281:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880986.751283:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000030024c2c. 00000400:00000200:3.0:1788880986.751288:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880986.751292:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880986.751295:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002b77e739 00000400:00000010:3.0:1788880986.751296:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002b77e739. 00000100:00000001:3.0:1788880986.751299:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880986.751302:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880986.751311:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880986.751313:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880986.751320:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880986.751321:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788880986.751353:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788880986.751355:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006aca3ff7. 00000400:00000200:3.0:1788880986.751359:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788880986.751362:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788880986.751372:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000006bff322d 00000400:00000010:3.0:1788880986.751374:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000006bff322d. 00000100:00000001:3.0:1788880986.751377:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788880986.751380:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880986.751434:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880986.751439:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880986.751446:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880986.751448:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880986.752469:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880986.752475:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880986.752478:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880986.752479:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880986.752484:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880986.752499:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b10f00 00000400:00000200:2.0:1788880986.752504:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1239 [1] + 224 00000800:00000001:2.0:1788880986.752533:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880986.752539:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880986.752542:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880986.752545:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880986.752548:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880986.752551:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880986.752558:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880986.752570:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880986.752575:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880986.752579:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880986.752580:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880986.752582:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880986.752585:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880986.752590:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880986.752590:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b10f80 00000100:00000001:0.0:1788880986.752594:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788880986.752598:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1241 [1] + 224 00000800:00000001:2.0:1788880986.752602:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788880986.752606:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788880986.752606:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880986.752609:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:0.0:1788880986.752610:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880986.752613:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000400:00000200:2.0:1788880986.752615:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788880986.752621:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880986.752624:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788880986.752627:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788880986.752628:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880986.752634:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880986.752635:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880986.752654:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880986.752667:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880986.752672:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880986.752674:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880986.752678:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880986.752678:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880986.752680:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880986.752683:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880986.752685:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880986.752686:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880986.752692:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880986.752696:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788880986.752702:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=64 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880986.752705:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=64 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880986.752708:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788880986.752709:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880986.752709:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880986.752713:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:0.0:1788880986.752714:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880986.752717:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880986.752718:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880986.752719:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880986.752722:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:0.0:1788880986.752726:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880986.752726:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:1.0:1788880986.752732:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000400:00000010:1.0:1788880986.752735:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:0.0:1788880986.752737:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880986.752740:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880986.752740:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788880986.752744:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788880986.752747:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880986.752749:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788880986.752778:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780701440:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788880986.752780:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880986.752785:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880986.752788:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788880986.752793:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000884e74b8 x1875776780701440/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880986.752801:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880986.752802:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788880986.752806:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880986.752807:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880986.752809:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d9c9c3cb. 02000000:00000001:0.0:1788880986.752813:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788880986.752813:0:201286:0:(client.c:1580:after_reply()) Process entered 00000020:00000001:0.0:1788880986.752817:0:201285:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000001:1.0:1788880986.752818:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000040:0.0:1788880986.752819:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788880986.752822:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000001:0.0:1788880986.752823:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:1.0:1788880986.752827:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880986.752828:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 02000000:00000001:1.0:1788880986.752832:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880986.752833:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880986.752836:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880986.752838:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000884e74b8. 00000100:00000001:0.0:1788880986.752841:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788880986.752842:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788880986.752842:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=54 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880986.752847:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=54 v=5 (1 1 1 1) 00000100:00000001:0.0:1788880986.752849:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880986.752851:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880986.752853:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880986.752855:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:1.0:1788880986.752855:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788880986.752859:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880986.752860:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880986.752862:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880986.752864:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788880986.752865:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880986.752869:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880986.752871:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880986.752872:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880986.752876:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880986.752881:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880986.752884:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880986.752885:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788880986.752889:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788880986.752897:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780701568:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880986.752902:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880986.752904:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880986.752907:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002084a2b4 x1875776780701568/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880986.752914:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880986.752916:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880986.752918:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009a0ddd94. 02000000:00000001:1.0:1788880986.752922:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880986.752924:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880986.752925:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880986.752928:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880986.752930:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 02000000:00000001:1.0:1788880986.752934:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880986.752936:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880986.752937:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002084a2b4. 00000800:00000001:2.0:1788880986.752939:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880986.752941:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880986.752943:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880986.752945:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880986.752946:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880986.752949:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000001:2.0:1788880986.752950:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788880986.752953:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880986.752955:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880986.752958:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880986.752961:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880986.752964:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880986.752971:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11000 00000400:00000200:2.0:1788880986.752977:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1249 [1] + 224 00000800:00000001:2.0:1788880986.752980:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880986.752985:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880986.752986:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880986.752989:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880986.752991:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880986.752994:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880986.752999:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880986.753010:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880986.753013:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880986.753016:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880986.753026:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880986.753030:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880986.753034:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788880986.753036:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880986.753038:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001a8af943. 00000400:00000200:3.0:1788880986.753042:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a3b2cf4a 00000400:00000010:3.0:1788880986.753044:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a3b2cf4a. 00000100:00000001:3.0:1788880986.753046:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788880986.753050:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788880986.753058:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880986.753065:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880986.753067:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880986.753069:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880986.753071:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880986.753072:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880986.753074:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880986.753076:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880986.753080:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=64 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880986.753083:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=148 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880986.753085:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880986.753087:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880986.753089:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880986.753102:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880986.753110:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880986.753125:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880986.753127:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880986.753134:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880986.753135:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880986.753139:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880986.753146:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880986.753157:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880986.753159:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880986.753163:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880986.753175:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780701696:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880986.753180:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880986.753182:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880986.753185:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780701696/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881002 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880986.753192:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880986.753194:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880986.753196:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e4881465. 02000000:00000001:3.0:1788880986.753200:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880986.753202:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880986.753203:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880986.753206:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880986.753207:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003139f6ba. 02000000:00000001:3.0:1788880986.753210:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880986.753212:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880986.753213:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000100:00000001:3.0:1788880986.753217:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880986.753225:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880986.753227:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880986.753231:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880986.753256:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880986.753260:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880986.753262:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880986.814343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880986.814349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880986.814353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880986.814354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880986.814381:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880986.814385:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880986.814388:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880987.838308:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880987.838314:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880987.838320:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880987.838320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880987.838367:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880987.838371:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880987.838374:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880988.862346:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880988.862352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880988.862356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880988.862359:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880988.862361:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880988.862364:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880988.862367:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880988.862368:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880988.862370:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880988.862372:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788880988.862416:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788880988.862419:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788880988.862421:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880989.886302:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880989.886310:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880989.886316:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880989.886317:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880989.886376:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880989.886381:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880989.886385:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880990.910353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880990.910359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880990.910364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880990.910366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880990.910453:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880990.910457:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880990.910460:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880991.870378:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306622927, at 4306622976 00000100:00000040:1.0:1788880991.870378:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306622927, at 4306622976 00000100:00000040:3.0:1788880991.870378:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306622927, at 4306622976 00000100:00000010:3.0:1788880991.870395:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000751d0eed. 00000100:00000010:1.0:1788880991.870396:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000535177f1. 00000100:00000010:0.0:1788880991.870402:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003b9d7f95. 00000100:00000040:2.0:1788880991.870424:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880991.870435:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880991.870449:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000020:00000040:2.0:1788880991.870456:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880991.870461:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880991.870465:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880991.870471:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880991.870502:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 00000100:00000001:2.0:1788880991.870512:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880991.870526:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880991.870538:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880991.870541:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880991.870547:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_03+3] req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880991.870572:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880991.870574:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880991.870577:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880991.870581:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 00000020:00000040:2.0:1788880991.870583:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788880991.870586:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880991.870587:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880991.870590:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880991.870594:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c54a5c6f. 00000100:00000001:2.0:1788880991.870598:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880991.870601:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788880991.870604:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880991.870605:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880991.870612:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788880991.870613:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880991.870614:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788880991.870614:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880991.870625:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788880991.870629:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_00+0] req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880991.870638:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880991.870648:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880991.870650:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880991.870654:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788880991.870656:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880991.870658:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788880991.870660:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788880991.870666:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880991.870668:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00100000:0.0:1788880991.870675:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780701824:192.168.204.111@tcp:400: 00000100:00000010:2.0:1788880991.870675:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000100:00000001:3.0:1788880991.870678:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880991.870680:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788880991.870685:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788880991.870687:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880991.870687:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880991.870694:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788880991.870697:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880991.870698:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788880991.870703:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:1.0:1788880991.870709:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880991.870710:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788880991.870710:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880991.870718:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880991.870719:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788880991.870732:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e53a8f34. 00000100:00000001:1.0:1788880991.870734:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788880991.870737:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880991.870738:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788880991.870739:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000058b4ee38. 00000100:00000040:2.0:1788880991.870746:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788880991.870748:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880991.870748:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880991.870756:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788880991.870759:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 02000000:00000001:1.0:1788880991.870770:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788880991.870773:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00100000:1.0:1788880991.870784:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780701952:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880991.870784:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000200:0.0:1788880991.870790:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780701824, portal 25 00000100:00000001:2.0:1788880991.870791:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880991.870804:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788880991.870805:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1788880991.870809:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788880991.870810:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_01+1] req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880991.870811:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 02000000:00000001:1.0:1788880991.870815:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880991.870821:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788880991.870826:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b1120e74. 02000000:00000001:1.0:1788880991.870832:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880991.870841:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000100:00000040:0.0:1788880991.870845:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881007 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880991.870848:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000100:00000001:2.0:1788880991.870854:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788880991.870855:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780701952, portal 10 00000100:00000001:0.0:1788880991.870857:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880991.870863:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788880991.870864:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788880991.870865:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788880991.870871:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938435456 : -126823771116160 : ffff8ca78a2c6d80) 00000100:00000040:2.0:1788880991.870876:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:3.0:1788880991.870878:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788880991.870882:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000033f4d237. 00000100:00000001:3.0:1788880991.870885:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880991.870886:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11960) 00000100:00000040:1.0:1788880991.870887:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788880991.870893:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780701824, offset 0 00000100:00000001:3.0:1788880991.870893:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788880991.870900:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880991.870904:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788880991.870911:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306622927, at 4306622976 00000400:00000200:0.0:1788880991.870912:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788880991.870915:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880991.870915:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880991.870923:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788880991.870925:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a0d72f03. 00000100:00000010:2.0:1788880991.870928:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000089d2bf5f. 00000100:00000200:1.0:1788880991.870930:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780701952, offset 0 02000000:00000001:3.0:1788880991.870930:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788880991.870937:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788880991.870939:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788880991.870943:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880991.870945:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788880991.870949:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00100000:3.0:1788880991.870949:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780702080:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788880991.870956:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880991.870957:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880991.870962:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:3.0:1788880991.870963:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880991.870967:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880991.870968:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788880991.870971:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788880991.870975:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [561:561:256:4294967295] 192.168.204.111@tcp LPNI seq info [561:561:8:4294967295] 02000000:00000010:3.0:1788880991.870978:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d022af59. 02000000:00000001:3.0:1788880991.870982:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788880991.870987:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:3.0:1788880991.870987:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000df34776e. 00000400:00000010:3.0:1788880991.870994:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000800:00000200:0.0:1788880991.870998:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880991.871000:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:3.0:1788880991.871000:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780702080, portal 10 00000800:00000010:0.0:1788880991.871004:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e494c472. 00000400:00000200:1.0:1788880991.871005:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788880991.871010:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788880991.871012:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788880991.871012:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 00000800:00000200:0.0:1788880991.871014:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788880991.871018:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788880991.871019:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788880991.871022:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788880991.871027:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788880991.871030:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [562:562:255:4294967295] 192.168.204.111@tcp LPNI seq info [562:562:7:4294967295] 00000800:00000200:0.0:1788880991.871033:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e494c472 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880991.871033:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788880991.871038:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788880991.871041:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:3.0:1788880991.871041:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005b101ac6. 00000100:00000200:3.0:1788880991.871043:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780702080, offset 0 00000800:00000200:1.0:1788880991.871047:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788880991.871051:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006df406c6. 00000400:00000200:3.0:1788880991.871051:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788880991.871056:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788880991.871056:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880991.871059:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880991.871061:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788880991.871063:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788880991.871063:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880991.871067:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306628047 00000800:00000200:1.0:1788880991.871068:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006df406c6 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788880991.871068:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:0.0:1788880991.871074:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003b9d7f95. 00000400:00000200:3.0:1788880991.871074:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788880991.871084:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:3.0:1788880991.871085:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [563:563:254:4294967295] 192.168.204.111@tcp LPNI seq info [563:563:6:4294967295] 00000100:00000001:1.0:1788880991.871087:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880991.871096:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306628047 00000100:00000040:1.0:1788880991.871096:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 155471 usecs (156 jiffies) 00000400:00000200:3.0:1788880991.871102:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788880991.871103:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306623132 00000100:00000010:0.0:1788880991.871104:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a682425c. 00000100:00000001:1.0:1788880991.871108:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306628047 00000800:00000200:3.0:1788880991.871110:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880991.871112:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788880991.871112:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306623132 00000800:00000010:3.0:1788880991.871115:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000001:0.0:1788880991.871119:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306628047 00000100:00000001:1.0:1788880991.871119:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306628047 00000800:00000200:3.0:1788880991.871124:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788880991.871128:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306623132 00000100:00000010:0.0:1788880991.871129:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000455e6647. 00000800:00000200:3.0:1788880991.871133:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788880991.871135:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880991.871137:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:3.0:1788880991.871138:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000800:00000010:2.0:1788880991.871140:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e494c472. 00000100:00000001:0.0:1788880991.871148:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306628047 00000100:00000001:1.0:1788880991.871148:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306628047 00000100:00000001:3.0:1788880991.871149:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880991.871150:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880991.871159:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306623132 00000100:00000040:3.0:1788880991.871159:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11625 usecs (12 jiffies) 00000100:00000001:3.0:1788880991.871164:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306622988 00000100:00000010:1.0:1788880991.871166:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000400:00000200:2.0:1788880991.871167:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880991.871176:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306628047 00000100:00000001:1.0:1788880991.871178:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000400:00000200:2.0:1788880991.871181:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000033f4d237 00000100:00000001:3.0:1788880991.871183:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306622988 00000400:00000010:2.0:1788880991.871184:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000033f4d237. 00000100:00000001:1.0:1788880991.871186:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880991.871191:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306623132 00000100:00000001:3.0:1788880991.871196:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306628047 00000100:00000001:1.0:1788880991.871197:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880991.871199:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788880991.871201:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880991.871201:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306622988 00000100:00000001:1.0:1788880991.871203:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880991.871209:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880991.871210:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881007 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880991.871211:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880991.871211:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306628047 00000100:00000001:0.0:1788880991.871223:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880991.871229:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880991.871229:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306622988 00000100:00000001:0.0:1788880991.871269:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880991.871271:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788880991.871277:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306628047 00000100:00000001:0.0:1788880991.871286:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880991.871286:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880991.871291:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880991.871294:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788880991.871294:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788880991.871298:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880991.871298:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880991.871299:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788880991.871302:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880991.871307:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880991.871307:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881007 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880991.871307:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880991.871308:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880991.871316:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880991.871317:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880991.871331:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880991.871332:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880991.871336:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880991.871338:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880991.871343:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788880991.871344:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788880991.871347:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880991.871349:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880991.871353:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788880991.871376:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880991.871380:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006df406c6. 00000400:00000200:2.0:1788880991.871386:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880991.871390:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880991.871393:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a0d72f03 00000400:00000010:2.0:1788880991.871394:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a0d72f03. 00000100:00000001:2.0:1788880991.871397:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880991.871399:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880991.871406:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880991.871407:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880991.871425:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880991.871426:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880991.871503:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880991.871506:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:2.0:1788880991.871509:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880991.871513:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880991.871516:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005b101ac6 00000400:00000010:2.0:1788880991.871519:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005b101ac6. 00000100:00000001:2.0:1788880991.871522:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880991.871525:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880991.871533:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880991.871534:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880991.871539:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880991.871540:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880991.873433:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880991.873446:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880991.873449:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880991.873451:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880991.873457:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880991.873469:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b11080 00000400:00000200:2.0:1788880991.873486:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1251 [1] + 224 00000800:00000001:2.0:1788880991.873495:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880991.873503:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880991.873505:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880991.873509:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880991.873513:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880991.873517:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880991.873527:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880991.873543:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880991.873551:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880991.873555:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880991.873577:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880991.873582:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788880991.873588:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880991.873590:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880991.873593:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000400:00000200:0.0:1788880991.873598:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880991.873601:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788880991.873606:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788880991.873615:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880991.873621:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880991.873628:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788880991.873630:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880991.873634:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880991.873641:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880991.873654:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880991.873663:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880991.873663:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788880991.873676:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880991.873680:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880991.873686:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000200:2.0:1788880991.873686:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11100 00000100:00000001:0.0:1788880991.873690:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880991.873694:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880991.873700:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1255 [1] + 224 00000100:00001000:0.0:1788880991.873701:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=69 v=5 (5 5 5 5) 00000100:00001000:0.0:1788880991.873719:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=69 v=5 (1 1 1 1) 00000800:00000001:2.0:1788880991.873719:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880991.873730:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:2.0:1788880991.873732:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880991.873734:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880991.873748:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00010000:00000001:0.0:1788880991.873749:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788880991.873756:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880991.873757:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880991.873760:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880991.873766:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880991.873769:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880991.873785:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880991.873788:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880991.873817:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880991.873819:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880991.873823:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880991.873824:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880991.873830:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880991.873837:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:0.0:1788880991.873838:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780701824:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880991.873847:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880991.873847:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880991.873850:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880991.873857:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880991.873858:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:2.0:1788880991.873858:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788880991.873861:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788880991.873864:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:2.0:1788880991.873868:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880991.873869:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780701824/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788880991.873869:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:1.0:1788880991.873886:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000400:00000010:1.0:1788880991.873892:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000400:00000200:2.0:1788880991.873895:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788880991.873904:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788880991.873916:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880991.873917:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000200:2.0:1788880991.873922:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11180 02000000:00000001:0.0:1788880991.873925:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788880991.873933:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000058b4ee38. 02000000:00000001:0.0:1788880991.873941:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000200:2.0:1788880991.873942:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1261 [1] + 224 00000020:00000001:0.0:1788880991.873946:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788880991.873950:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788880991.873960:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000800:00000001:2.0:1788880991.873960:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000010:0.0:1788880991.873968:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 00000100:00000200:1.0:1788880991.873973:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880991.873974:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788880991.873978:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788880991.873981:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:1.0:1788880991.873984:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880991.873986:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000800:00000001:2.0:1788880991.873986:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880991.873987:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880991.873991:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880991.873998:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880991.873998:0:201285:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:2.0:1788880991.873998:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1788880991.874003:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880991.874007:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788880991.874010:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880991.874012:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:1.0:1788880991.874015:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880991.874016:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880991.874021:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880991.874023:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880991.874023:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=59 v=5 (5 5 5 5) 00000100:00001000:1.0:1788880991.874029:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=59 v=5 (1 1 1 1) 00000400:00000200:2.0:1788880991.874031:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880991.874033:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880991.874037:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880991.874040:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788880991.874041:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:1.0:1788880991.874048:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788880991.874051:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880991.874055:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880991.874061:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880991.874069:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788880991.874072:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880991.874077:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788880991.874086:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880991.874119:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880991.874120:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788880991.874121:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788880991.874125:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:1.0:1788880991.874126:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880991.874133:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00100000:1.0:1788880991.874136:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780701952:192.168.204.111@tcp:400: 00000800:00000001:2.0:1788880991.874138:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880991.874147:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880991.874150:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880991.874152:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000175072aa x1875776780701952/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880991.874163:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880991.874166:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788880991.874167:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788880991.874168:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b1120e74. 00000100:00000001:3.0:1788880991.874171:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788880991.874172:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880991.874174:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880991.874176:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000001:3.0:1788880991.874176:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000001:1.0:1788880991.874178:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000001:3.0:1788880991.874179:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000010:1.0:1788880991.874180:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c54a5c6f. 00000400:00000010:3.0:1788880991.874182:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000df34776e. 02000000:00000001:1.0:1788880991.874183:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880991.874186:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000200:3.0:1788880991.874188:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000100:00000010:1.0:1788880991.874189:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000175072aa. 00000400:00000010:3.0:1788880991.874193:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:1.0:1788880991.874194:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880991.874197:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880991.874197:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788880991.874201:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788880991.874205:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880991.874206:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880991.874209:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880991.874214:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880991.874216:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788880991.874220:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880991.874226:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880991.874227:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880991.874229:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788880991.874230:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880991.874250:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880991.874251:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880991.874253:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880991.874256:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=69 v=5 (5 5 5 5) 00000100:00001000:3.0:1788880991.874258:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=153 v=5 (1 0 0 0) 00000100:00000001:3.0:1788880991.874260:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880991.874261:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880991.874268:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788880991.874271:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788880991.874276:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880991.874278:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788880991.874279:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788880991.874280:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880991.874281:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880991.874282:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880991.874287:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880991.874288:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880991.874289:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788880991.874291:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880991.874295:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780702080:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880991.874298:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880991.874299:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788880991.874301:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780702080/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881007 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880991.874304:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788880991.874306:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880991.874307:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d022af59. 02000000:00000001:3.0:1788880991.874309:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788880991.874310:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880991.874311:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788880991.874313:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788880991.874314:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e53a8f34. 02000000:00000001:3.0:1788880991.874316:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788880991.874317:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788880991.874317:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000001:3.0:1788880991.874322:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880991.874323:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880991.874324:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880991.874327:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788880991.874328:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880991.874329:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880991.874331:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880991.934389:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880991.934395:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880991.934400:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880991.934401:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880991.934473:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880991.934477:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880991.934479:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880992.958330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880992.958336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880992.958340:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880992.958341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880992.958390:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880992.958403:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880992.958406:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880993.982345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880993.982351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880993.982356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880993.982357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880993.982424:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880993.982426:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880993.982429:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880995.006329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880995.006336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880995.006350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880995.006351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880995.006430:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880995.006434:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880995.006437:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880996.030373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880996.030381:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880996.030383:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880996.030387:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880996.030391:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880996.030393:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880996.030394:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880996.030395:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880996.030399:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880996.030399:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880996.030459:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880996.030463:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880996.030467:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788880996.990369:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306628047, at 4306628096 00000100:00000040:1.0:1788880996.990369:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306628047, at 4306628096 00000100:00000040:3.0:1788880996.990369:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306628047, at 4306628096 00000100:00000040:2.0:1788880996.990385:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:3.0:1788880996.990389:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000010:1.0:1788880996.990393:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a682425c. 00000100:00000010:0.0:1788880996.990396:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003b9d7f95. 00000100:00000001:2.0:1788880996.990397:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788880996.990412:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000078d5a15d. 00000020:00000040:2.0:1788880996.990420:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788880996.990426:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880996.990430:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788880996.990437:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880996.990450:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e53a8f34. 00000100:00000001:2.0:1788880996.990459:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880996.990473:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788880996.990488:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880996.990491:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880996.990518:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000078d5a15d] to pc [ptlrpcd_00_02+2] req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880996.990549:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880996.990553:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788880996.990558:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788880996.990561:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:2.0:1788880996.990562:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788880996.990567:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788880996.990567:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788880996.990571:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788880996.990573:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880996.990576:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:3.0:1788880996.990578:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788880996.990581:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880996.990583:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788880996.990589:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c54a5c6f. 00000100:00000001:0.0:1788880996.990591:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880996.990591:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788880996.990597:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880996.990599:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880996.990604:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788880996.990608:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:3.0:1788880996.990610:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788880996.990617:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880996.990626:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:3.0:1788880996.990627:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780702208:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788880996.990630:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880996.990637:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788880996.990639:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_00_03+3] req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788880996.990647:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788880996.990652:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880996.990656:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788880996.990660:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788880996.990661:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000606f995b. 00000100:00000040:2.0:1788880996.990666:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788880996.990667:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880996.990674:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788880996.990675:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788880996.990678:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000df34776e. 00000100:00000001:0.0:1788880996.990679:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000010:2.0:1788880996.990679:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000100:00000001:0.0:1788880996.990683:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:2.0:1788880996.990685:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:0.0:1788880996.990690:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880996.990691:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788880996.990694:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000400:00000010:3.0:1788880996.990695:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 02000000:00000001:2.0:1788880996.990699:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880996.990703:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788880996.990705:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788880996.990707:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000045330cf1. 02000000:00000001:0.0:1788880996.990708:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:3.0:1788880996.990709:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780702208, portal 25 02000000:00000001:0.0:1788880996.990711:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788880996.990714:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880996.990715:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880996.990722:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:2.0:1788880996.990723:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788880996.990724:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780702336:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880996.990725:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938426496 : -126823771125120 : ffff8ca78a2c4a80) 00000100:00000001:0.0:1788880996.990733:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:3.0:1788880996.990738:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881012 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880996.990739:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788880996.990743:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788880996.990747:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788880996.990753:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000f8c34a6a. 00000100:00000001:3.0:1788880996.990753:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788880996.990758:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788880996.990765:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880996.990766:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000400:00000010:3.0:1788880996.990769:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d23804de. 00000400:00000010:0.0:1788880996.990773:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000200:3.0:1788880996.990774:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780702208, offset 0 00000100:00000200:0.0:1788880996.990781:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780702336, portal 10 00000100:00000001:0.0:1788880996.990787:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788880996.990788:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788880996.990790:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000100:00000001:2.0:1788880996.990791:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788880996.990796:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788880996.990801:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880996.990805:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d838cab4] to pc [ptlrpcd_00_00+0] req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788880996.990806:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788880996.990817:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788880996.990825:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788880996.990831:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788880996.990838:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788880996.990845:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [564:564:256:4294967295] 192.168.204.111@tcp LPNI seq info [564:564:8:4294967295] 00000100:00000001:0.0:1788880996.990848:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788880996.990853:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788880996.990856:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a64a1915. 00000100:00000200:0.0:1788880996.990861:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780702336, offset 0 00000100:00000001:2.0:1788880996.990861:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788880996.990863:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:2.0:1788880996.990868:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:0.0:1788880996.990875:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788880996.990877:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788880996.990882:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788880996.990883:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:3.0:1788880996.990885:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000400:00000200:0.0:1788880996.990888:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788880996.990888:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788880996.990890:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11965) 00000100:00000001:1.0:1788880996.990892:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:3.0:1788880996.990898:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788880996.990901:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788880996.990902:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880996.990908:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788880996.990914:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788880996.990915:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:2.0:1788880996.990915:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306628047, at 4306628096 00000800:00000200:3.0:1788880996.990922:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788880996.990924:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:2.0:1788880996.990927:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000455e6647. 00000400:00000200:0.0:1788880996.990932:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [565:565:255:4294967295] 192.168.204.111@tcp LPNI seq info [565:565:7:4294967295] 00000100:00000001:1.0:1788880996.990933:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788880996.990936:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880996.990938:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880996.990941:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788880996.990944:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880996.990945:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306633167 02000000:00000001:1.0:1788880996.990949:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880996.990951:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000016ab1a15. 00000100:00100000:1.0:1788880996.990956:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780702464:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788880996.990959:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1788880996.990966:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788880996.990967:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788880996.990967:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306633167 02000000:00000001:1.0:1788880996.990971:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788880996.990975:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788880996.990977:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000067a34a59. 02000000:00000001:1.0:1788880996.990978:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:0.0:1788880996.990983:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000010:1.0:1788880996.990984:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009a0ddd94. 00000100:00000001:3.0:1788880996.990985:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 02000000:00000001:1.0:1788880996.990989:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788880996.990992:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000013bf63f4. 00000100:00000001:3.0:1788880996.990994:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306633167 00000400:00000010:1.0:1788880996.990995:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000400:00000010:1.0:1788880996.991002:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000800:00000200:0.0:1788880996.991003:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:3.0:1788880996.991003:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000007dfb4281. 00000100:00000200:1.0:1788880996.991011:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780702464, portal 10 00000800:00000200:0.0:1788880996.991015:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788880996.991016:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788880996.991018:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788880996.991019:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000800:00000200:0.0:1788880996.991023:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000013bf63f4 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788880996.991025:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306633167 00000100:00000001:0.0:1788880996.991029:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788880996.991029:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:3.0:1788880996.991033:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000945234cf. 00000100:00000040:0.0:1788880996.991038:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 154571 usecs (155 jiffies) 00000100:00000001:3.0:1788880996.991041:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788880996.991046:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306628251 00000100:00000001:3.0:1788880996.991048:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880996.991054:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306633167 00000100:00000001:3.0:1788880996.991055:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880996.991062:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306628251 00000100:00000001:1.0:1788880996.991062:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788880996.991063:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880996.991069:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880996.991070:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306633167 00000800:00000200:2.0:1788880996.991074:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788880996.991075:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306628251 00000100:00000040:1.0:1788880996.991076:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880996.991079:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:1.0:1788880996.991081:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000974f52f4. 00000800:00000010:2.0:1788880996.991081:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000100:00000001:0.0:1788880996.991082:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306633167 00000100:00000200:1.0:1788880996.991088:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780702464, offset 0 00000100:00000001:0.0:1788880996.991089:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306628251 00000400:00000200:2.0:1788880996.991090:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788880996.991097:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306633167 00000400:00000200:1.0:1788880996.991097:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788880996.991100:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:2.0:1788880996.991104:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788880996.991105:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880996.991108:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788880996.991109:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880996.991110:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788880996.991113:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880996.991117:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788880996.991118:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880996.991119:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880996.991124:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788880996.991125:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880996.991129:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788880996.991130:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880996.991133:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788880996.991135:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788880996.991136:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880996.991137:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880996.991140:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:1.0:1788880996.991144:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [566:566:254:4294967295] 192.168.204.111@tcp LPNI seq info [566:566:6:4294967295] 00000100:00000001:0.0:1788880996.991145:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788880996.991148:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880996.991150:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880996.991154:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788880996.991164:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788880996.991172:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788880996.991174:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d23804de 00000800:00000010:1.0:1788880996.991177:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f0a0057b. 00000400:00000010:2.0:1788880996.991179:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d23804de. 00000800:00000200:1.0:1788880996.991186:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788880996.991187:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880996.991194:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881012 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788880996.991198:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788880996.991205:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f0a0057b type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788880996.991214:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880996.991214:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880996.991221:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11580 usecs (12 jiffies) 00000100:00000040:2.0:1788880996.991222:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881012 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880996.991230:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306628108 00000100:00000001:2.0:1788880996.991259:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880996.991262:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788880996.991268:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306633167 00000100:00000001:1.0:1788880996.991273:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306628109 00000100:00000001:1.0:1788880996.991275:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306633167 00000100:00000001:1.0:1788880996.991278:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306628109 00000100:00000001:1.0:1788880996.991281:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306633167 00000800:00000200:2.0:1788880996.991283:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788880996.991284:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306628109 00000800:00000010:2.0:1788880996.991288:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000013bf63f4. 00000100:00000001:1.0:1788880996.991290:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306633167 00000400:00000200:2.0:1788880996.991293:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788880996.991295:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880996.991302:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880996.991302:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788880996.991309:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880996.991310:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a64a1915 00000100:00000001:1.0:1788880996.991326:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880996.991333:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:2.0:1788880996.991335:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a64a1915. 00000100:00000001:1.0:1788880996.991337:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788880996.991342:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788880996.991343:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880996.991346:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:2.0:1788880996.991349:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880996.991351:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880996.991357:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880996.991367:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880996.991370:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880996.991378:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880996.991380:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788880996.991417:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788880996.991419:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f0a0057b. 00000400:00000200:2.0:1788880996.991423:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880996.991427:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788880996.991430:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000974f52f4 00000400:00000010:2.0:1788880996.991432:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000974f52f4. 00000100:00000001:2.0:1788880996.991434:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788880996.991437:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880996.991445:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788880996.991447:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880996.991454:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788880996.991456:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788880996.992591:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880996.992605:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880996.992608:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880996.992611:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880996.992620:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880996.992632:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b11200 00000400:00000200:2.0:1788880996.992639:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1269 [1] + 224 00000800:00000001:2.0:1788880996.992645:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880996.992652:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880996.992662:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880996.992667:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880996.992674:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880996.992677:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880996.992686:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880996.992703:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880996.992711:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880996.992718:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880996.992720:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880996.992723:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880996.992728:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788880996.992736:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11280 00000100:00000001:3.0:1788880996.992741:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788880996.992742:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1271 [1] + 224 00000800:00000001:2.0:1788880996.992746:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880996.992751:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880996.992753:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788880996.992756:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788880996.992759:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880996.992762:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788880996.992770:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880996.992786:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788880996.992790:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880996.992794:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880996.992804:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880996.992815:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000001:3.0:1788880996.992816:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788880996.992821:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788880996.992822:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788880996.992920:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000df34776e. 00000800:00000001:2.0:1788880996.992933:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880996.992941:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788880996.992943:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788880996.992945:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880996.992949:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788880996.992965:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000200:2.0:1788880996.992966:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11300 00000400:00000010:3.0:1788880996.992971:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:3.0:1788880996.992978:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:0.0:1788880996.992979:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788880996.992983:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788880996.992987:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000100:00000200:3.0:1788880996.992990:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788880996.992996:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788880996.993003:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000200:3.0:1788880996.993008:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880996.993012:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788880996.993024:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1279 [1] + 224 00000100:00000001:3.0:1788880996.993027:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788880996.993030:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880996.993035:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000200:0.0:1788880996.993036:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788880996.993036:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788880996.993039:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788880996.993042:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788880996.993044:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880996.993046:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788880996.993057:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:3.0:1788880996.993059:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788880996.993068:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:0.0:1788880996.993074:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788880996.993074:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=74 v=5 (5 5 5 5) 00000100:00000001:2.0:1788880996.993075:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788880996.993084:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880996.993085:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880996.993088:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880996.993096:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788880996.993099:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788880996.993106:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788880996.993109:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788880996.993110:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=74 v=5 (1 1 1 1) 02000000:00000001:0.0:1788880996.993119:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788880996.993123:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788880996.993124:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00001000:0.0:1788880996.993128:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=64 v=5 (5 5 5 5) 00000100:00000001:3.0:1788880996.993129:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788880996.993142:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00001000:0.0:1788880996.993145:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=64 v=5 (1 1 1 1) 00010000:00000001:3.0:1788880996.993148:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880996.993161:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788880996.993161:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880996.993168:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788880996.993172:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:3.0:1788880996.993173:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788880996.993175:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:0.0:1788880996.993177:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788880996.993184:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788880996.993188:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000800:00000001:2.0:1788880996.993194:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880996.993194:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788880996.993200:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788880996.993202:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880996.993211:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788880996.993213:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880996.993214:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000001:1.0:1788880996.993218:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788880996.993224:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880996.993226:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788880996.993226:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788880996.993260:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:1.0:1788880996.993268:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000400:00000010:1.0:1788880996.993271:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:1.0:1788880996.993275:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1788880996.993278:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880996.993286:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788880996.993287:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788880996.993350:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788880996.993367:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780702208:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788880996.993368:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788880996.993380:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788880996.993383:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788880996.993388:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880996.993391:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788880996.993393:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788880996.993399:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880996.993401:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788880996.993403:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:1.0:1788880996.993409:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788880996.993414:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:3.0:1788880996.993417:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000078d5a15d x1875776780702208/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880996.993422:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788880996.993435:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788880996.993451:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=74 v=5 (5 5 5 5) 00000100:00100000:0.0:1788880996.993452:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780702336:192.168.204.111@tcp:400: 00000100:00001000:1.0:1788880996.993473:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=8 v=5 (1 1 0 0) 00000100:00000001:3.0:1788880996.993473:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788880996.993479:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788880996.993480:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788880996.993488:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000606f995b. 00000100:00000001:0.0:1788880996.993489:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788880996.993489:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788880996.993498:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788880996.993500:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:0.0:1788880996.993504:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780702336/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788880996.993508:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:3.0:1788880996.993508:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788880996.993518:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788880996.993519:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00010000:00080000:1.0:1788880996.993525:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788880996.993526:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000020:00000001:3.0:1788880996.993527:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788880996.993535:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000f8c34a6a. 02000000:00000010:3.0:1788880996.993535:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e53a8f34. 02000000:00000001:0.0:1788880996.993547:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788880996.993551:0:201288:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000001:3.0:1788880996.993552:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00010000:00000001:1.0:1788880996.993556:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788880996.993563:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:3.0:1788880996.993563:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788880996.993565:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788880996.993576:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000020:00000001:0.0:1788880996.993577:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000010:3.0:1788880996.993577:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000078d5a15d. 02000000:00000010:0.0:1788880996.993590:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c54a5c6f. 00000100:00000001:1.0:1788880996.993591:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788880996.993592:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880996.993597:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880996.993600:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880996.993602:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788880996.993604:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788880996.993605:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:1.0:1788880996.993609:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788880996.993611:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:0.0:1788880996.993616:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788880996.993617:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880996.993619:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880996.993621:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788880996.993623:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880996.993631:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788880996.993633:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880996.993634:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788880996.993635:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788880996.993642:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788880996.993649:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788880996.993649:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788880996.993651:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788880996.993662:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788880996.993663:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788880996.993664:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780702464:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788880996.993686:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880996.993689:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788880996.993691:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d838cab4 x1875776780702464/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881012 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788880996.993698:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788880996.993701:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788880996.993702:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009a0ddd94. 02000000:00000001:1.0:1788880996.993706:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788880996.993708:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788880996.993709:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788880996.993711:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788880996.993712:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000045330cf1. 02000000:00000001:1.0:1788880996.993714:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788880996.993715:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788880996.993716:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d838cab4. 00000100:00000001:1.0:1788880996.993718:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788880996.993719:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880996.993721:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788880996.993724:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788880996.993725:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788880996.993727:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788880996.993728:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880997.054384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880997.054393:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880997.054398:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880997.054400:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880997.054486:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880997.054500:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880997.054506:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880998.078400:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880998.078409:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880998.078415:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880998.078416:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788880998.078483:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788880998.078489:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788880998.078494:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788880999.102363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880999.102370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880999.102377:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788880999.102380:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788880999.102383:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788880999.102386:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880999.102389:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788880999.102391:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880999.102393:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788880999.102396:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788880999.102457:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788880999.102462:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788880999.102465:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881000.126385:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881000.126392:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881000.126401:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881000.126405:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881000.126482:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881000.126488:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881000.126492:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881001.150348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881001.150355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881001.150361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881001.150363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881001.150420:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881001.150423:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881001.150425:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881002.110335:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306633167, at 4306633216 00000100:00000040:1.0:1788881002.110335:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306633167, at 4306633216 00000100:00000040:2.0:1788881002.110342:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306633167, at 4306633216 00000100:00000010:1.0:1788881002.110354:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000067a34a59. 00000100:00000010:0.0:1788881002.110360:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000016ab1a15. 00000100:00000010:2.0:1788881002.110364:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000007dfb4281. 00000100:00000040:2.0:1788881002.110398:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881002.110403:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000040:3.0:1788881002.110406:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306633167, at 4306633216 00000100:00000010:3.0:1788881002.110421:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000945234cf. 00000100:00000010:2.0:1788881002.110422:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d838cab4. 00000020:00000040:2.0:1788881002.110433:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788881002.110437:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881002.110440:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881002.110445:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881002.110454:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003139f6ba. 00000100:00000001:2.0:1788881002.110460:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881002.110473:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788881002.110482:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788881002.110483:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881002.110488:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d838cab4] to pc [ptlrpcd_00_01+1] req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881002.110526:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881002.110528:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881002.110530:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881002.110533:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d6fc4b5d. 00000020:00000040:2.0:1788881002.110535:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788881002.110537:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881002.110539:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881002.110542:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881002.110545:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000781b5970. 00000100:00000001:2.0:1788881002.110548:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881002.110552:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788881002.110553:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881002.110560:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881002.110561:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881002.110568:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881002.110569:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881002.110569:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788881002.110579:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881002.110586:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881002.110587:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6fc4b5d] to pc [ptlrpcd_00_02+2] req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881002.110624:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881002.110629:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881002.110637:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788881002.110642:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881002.110645:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881002.110647:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881002.110649:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881002.110655:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881002.110659:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000078d5a15d. 00000100:00100000:3.0:1788881002.110661:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780702592:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881002.110663:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881002.110670:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:2.0:1788881002.110670:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881002.110678:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788881002.110680:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881002.110682:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:0.0:1788881002.110686:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881002.110686:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881002.110686:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788881002.110695:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788881002.110696:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000eaa15d72. 02000000:00000001:3.0:1788881002.110700:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788881002.110702:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881002.110702:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881002.110708:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.110711:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:2.0:1788881002.110716:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788881002.110717:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881002.110718:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:0.0:1788881002.110728:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788881002.110731:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788881002.110732:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000010:3.0:1788881002.110733:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000c4cba31. 02000000:00000001:0.0:1788881002.110747:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881002.110747:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881002.110748:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881002.110763:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780702720:192.168.204.111@tcp:400: 00000400:00000010:3.0:1788881002.110768:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000df34776e. 00000100:00000001:0.0:1788881002.110769:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788881002.110769:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000078d5a15d] to pc [ptlrpcd_00_03+3] req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881002.110776:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881002.110779:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881002.110784:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788881002.110788:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003a17641d. 02000000:00000001:0.0:1788881002.110796:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881002.110800:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881002.110803:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000100:00000040:2.0:1788881002.110803:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788881002.110806:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000400:00000010:0.0:1788881002.110813:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000040:2.0:1788881002.110814:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000200:3.0:1788881002.110821:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780702592, portal 25 00000100:00000040:2.0:1788881002.110822:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11970) 00000100:00000200:0.0:1788881002.110823:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780702720, portal 10 00000100:00000001:1.0:1788881002.110829:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881002.110837:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881002.110837:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881002.110838:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881002.110843:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430976 : -126823771120640 : ffff8ca78a2c5c00) 00000100:00000001:0.0:1788881002.110844:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938439040 : -126823771112576 : ffff8ca78a2c7b80) 00000100:00000001:1.0:1788881002.110844:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788881002.110855:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788881002.110868:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881002.110868:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881018 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881002.110875:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788881002.110881:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881002.110895:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788881002.110900:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788881002.110914:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788881002.110929:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780702848:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881002.110938:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788881002.110938:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788881002.110941:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788881002.110948:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881002.110951:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881002.110960:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788881002.110960:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788881002.110960:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:1.0:1788881002.110972:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000356bd8da. 00000400:00000010:0.0:1788881002.110974:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004aa60c6e. 00000400:00000010:3.0:1788881002.110975:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ec90f18d. 02000000:00000001:1.0:1788881002.110976:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881002.110980:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780702720, offset 0 00000100:00000200:3.0:1788881002.110985:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780702592, offset 0 00000400:00000010:1.0:1788881002.110987:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000400:00000010:1.0:1788881002.111000:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000400:00000200:3.0:1788881002.111008:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881002.111019:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:1.0:1788881002.111026:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780702848, portal 10 00000400:00000200:3.0:1788881002.111031:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788881002.111033:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881002.111040:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938426496 : -126823771125120 : ffff8ca78a2c4a80) 00000400:00000200:3.0:1788881002.111046:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788881002.111055:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881002.111056:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788881002.111064:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788881002.111074:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788881002.111081:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:3.0:1788881002.111085:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [567:567:256:4294967295] 192.168.204.111@tcp LPNI seq info [567:567:8:4294967295] 00000100:00000040:1.0:1788881002.111093:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881002.111099:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000038c5ac58. 00000100:00000200:1.0:1788881002.111105:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780702848, offset 0 00000400:00000200:3.0:1788881002.111122:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:1.0:1788881002.111123:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788881002.111141:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881002.111146:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788881002.111151:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788881002.111154:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788881002.111159:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788881002.111161:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000010:3.0:1788881002.111162:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000400:00000200:0.0:1788881002.111167:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [568:568:255:4294967295] 192.168.204.111@tcp LPNI seq info [568:568:7:4294967295] 00000800:00000200:3.0:1788881002.111185:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788881002.111188:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788881002.111201:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788881002.111206:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000000f9f1055. 00000400:00000200:1.0:1788881002.111206:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788881002.111207:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788881002.111256:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788881002.111263:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788881002.111265:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788881002.111267:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788881002.111269:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788881002.111279:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788881002.111281:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:0.0:1788881002.111284:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000000f9f1055 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788881002.111289:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [569:569:254:4294967295] 192.168.204.111@tcp LPNI seq info [569:569:6:4294967295] 00000100:00000001:3.0:1788881002.111292:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.111293:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881002.111306:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 153675 usecs (154 jiffies) 00000100:00000001:3.0:1788881002.111306:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306638288 00000400:00000200:1.0:1788881002.111308:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788881002.111318:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306633371 00000100:00000010:3.0:1788881002.111318:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000945234cf. 00000800:00000200:1.0:1788881002.111327:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881002.111330:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000010:1.0:1788881002.111334:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b016e4f4. 00000100:00000001:3.0:1788881002.111337:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306638288 00000100:00000001:0.0:1788881002.111338:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306638288 00000100:00000010:3.0:1788881002.111347:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000837757ae. 00000100:00000001:0.0:1788881002.111348:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306633371 00000800:00000200:1.0:1788881002.111348:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788881002.111354:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:2.0:1788881002.111355:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:1.0:1788881002.111362:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000010:2.0:1788881002.111362:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000100:00000001:0.0:1788881002.111365:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306638288 00000100:00000001:3.0:1788881002.111365:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306638288 00000400:00000200:2.0:1788881002.111370:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:1.0:1788881002.111373:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b016e4f4 type 1, nob 320 niov 1 nkiov 1 00000100:00000010:3.0:1788881002.111378:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000fbefa178. 00000100:00000001:0.0:1788881002.111381:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306633371 00000100:00000001:3.0:1788881002.111388:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788881002.111390:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881002.111390:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881002.111400:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306638288 00000100:00000001:0.0:1788881002.111402:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306638288 00000100:00000040:1.0:1788881002.111403:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11530 usecs (12 jiffies) 00000400:00000200:2.0:1788881002.111404:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ec90f18d 00000100:00000010:3.0:1788881002.111405:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000dba5f48d. 00000400:00000010:2.0:1788881002.111413:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ec90f18d. 00000100:00000001:0.0:1788881002.111414:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306633371 00000100:00000001:3.0:1788881002.111415:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788881002.111416:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306633229 00000100:00000001:2.0:1788881002.111419:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788881002.111425:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881002.111427:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306638288 00000100:00000001:1.0:1788881002.111427:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306638288 00000100:00000200:2.0:1788881002.111431:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881018 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881002.111434:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881002.111439:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306633229 00000100:00000001:0.0:1788881002.111440:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881002.111446:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.111448:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881002.111448:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306638288 00000100:00000001:3.0:1788881002.111457:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881002.111459:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881002.111459:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306633229 00000100:00000001:0.0:1788881002.111468:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881002.111469:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306638288 00000100:00000001:3.0:1788881002.111473:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881002.111476:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306633229 00000100:00000001:0.0:1788881002.111477:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881002.111478:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788881002.111482:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788881002.111485:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306638288 00000100:00000001:3.0:1788881002.111488:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881002.111490:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881002.111492:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788881002.111493:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881018 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881002.111495:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881002.111499:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.111500:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881002.111504:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.111508:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881002.111509:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881002.111512:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.111513:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.111524:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881002.111526:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881002.111529:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881002.111533:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788881002.111534:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881002.111537:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881002.111543:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881002.111545:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881002.111547:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881002.111550:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881002.111554:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788881002.111558:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000000f9f1055. 00000400:00000200:2.0:1788881002.111562:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881002.111566:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881002.111569:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004aa60c6e 00000400:00000010:2.0:1788881002.111570:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004aa60c6e. 00000100:00000001:2.0:1788881002.111574:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881002.111577:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881002.111586:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881002.111597:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881002.111605:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881002.111607:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788881002.111667:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788881002.111671:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b016e4f4. 00000400:00000200:2.0:1788881002.111674:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881002.111678:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881002.111681:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000038c5ac58 00000400:00000010:2.0:1788881002.111682:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000038c5ac58. 00000100:00000001:2.0:1788881002.111685:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881002.111688:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881002.111696:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881002.111698:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881002.111705:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881002.111707:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788881002.113535:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881002.113547:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881002.113549:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881002.113551:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881002.113556:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881002.113566:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b11380 00000400:00000200:2.0:1788881002.113574:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1281 [1] + 224 00000800:00000001:2.0:1788881002.113580:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881002.113583:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881002.113584:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881002.113587:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881002.113591:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881002.113594:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881002.113601:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881002.113624:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881002.113632:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881002.113635:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881002.113637:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881002.113639:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881002.113645:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788881002.113647:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881002.113650:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788881002.113656:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11400 00000400:00000001:3.0:1788881002.113660:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788881002.113662:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788881002.113664:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000df34776e. 00000400:00000200:3.0:1788881002.113668:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000010:3.0:1788881002.113748:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:3.0:1788881002.113753:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788881002.113759:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1285 [1] + 224 00000800:00000001:2.0:1788881002.113763:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:3.0:1788881002.113765:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881002.113768:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881002.113775:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881002.113784:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881002.113796:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881002.113802:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881002.113809:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881002.113819:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881002.113822:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881002.113824:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881002.113825:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881002.113826:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881002.113829:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881002.113833:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11480 00000400:00000200:2.0:1788881002.113836:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1291 [1] + 224 00000800:00000001:2.0:1788881002.113839:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881002.113845:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881002.113848:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:3.0:1788881002.113854:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881002.113858:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788881002.113858:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881002.113863:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881002.113871:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:0.0:1788881002.113873:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881002.113876:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788881002.113876:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000010:0.0:1788881002.113883:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000100:00000001:3.0:1788881002.113883:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788881002.113884:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881002.113898:0:201286:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:0.0:1788881002.113899:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788881002.113902:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000040:2.0:1788881002.113903:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881002.113903:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881002.113907:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881002.113909:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788881002.113911:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881002.113921:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881002.113921:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881002.113933:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881002.113945:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788881002.113947:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1788881002.113947:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=80 v=5 (5 5 5 5) 00000800:00000001:2.0:1788881002.113950:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788881002.113967:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=80 v=5 (1 1 1 1) 00000100:00000001:0.0:1788881002.113969:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881002.113975:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881002.113976:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881002.113981:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.113983:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788881002.113983:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:3.0:1788881002.113986:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788881002.113990:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:3.0:1788881002.113991:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881002.113992:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000001:1.0:1788881002.113997:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788881002.114000:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881002.114003:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.114005:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788881002.114005:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788881002.114010:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000100:00000040:3.0:1788881002.114010:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881002.114012:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881002.114022:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000100:00001000:0.0:1788881002.114026:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=70 v=5 (5 5 5 5) 00000400:00000010:1.0:1788881002.114033:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:3.0:1788881002.114048:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00001000:0.0:1788881002.114049:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=70 v=5 (1 1 1 1) 00000100:00000001:1.0:1788881002.114049:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788881002.114057:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881002.114058:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881002.114063:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881002.114064:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881002.114064:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788881002.114078:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000040:3.0:1788881002.114081:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788881002.114083:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.114096:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788881002.114102:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881002.114112:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881002.114118:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788881002.114135:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d838cab4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780702592:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788881002.114136:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881002.114140:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881002.114154:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881002.114166:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881002.114177:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d838cab4 x1875776780702592/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881002.114211:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881002.114231:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881002.114265:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788881002.114268:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788881002.114277:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000010:3.0:1788881002.114277:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000c4cba31. 02000000:00000001:1.0:1788881002.114285:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:3.0:1788881002.114286:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788881002.114287:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881002.114287:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000001:3.0:1788881002.114290:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788881002.114292:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788881002.114297:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 02000000:00000001:1.0:1788881002.114298:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.114299:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000020:00000001:3.0:1788881002.114309:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788881002.114310:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788881002.114311:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=80 v=5 (5 5 5 5) 02000000:00000010:3.0:1788881002.114313:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003139f6ba. 00000100:00001000:1.0:1788881002.114326:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=14 v=5 (1 1 0 0) 02000000:00000001:3.0:1788881002.114327:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:0.0:1788881002.114328:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881002.114331:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881002.114337:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d838cab4. 00000100:00000001:1.0:1788881002.114339:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881002.114342:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881002.114347:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00010000:00000001:1.0:1788881002.114348:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788881002.114357:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:0.0:1788881002.114358:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d6fc4b5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780702720:192.168.204.111@tcp:400: 00010000:00080000:1.0:1788881002.114363:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881002.114369:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881002.114375:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881002.114384:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:1.0:1788881002.114385:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881002.114386:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788881002.114387:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881002.114391:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788881002.114392:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881002.114394:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788881002.114400:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881002.114402:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d6fc4b5d x1875776780702720/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881002.114402:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788881002.114408:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881002.114418:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881002.114423:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881002.114427:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881002.114430:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003a17641d. 00000100:00000001:1.0:1788881002.114431:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881002.114436:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788881002.114436:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000020:00000001:0.0:1788881002.114440:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788881002.114440:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:0.0:1788881002.114444:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000040:1.0:1788881002.114448:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788881002.114450:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881002.114454:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000781b5970. 02000000:00000001:0.0:1788881002.114461:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00100000:1.0:1788881002.114464:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000078d5a15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780702848:192.168.204.111@tcp:400: 02000000:00000001:0.0:1788881002.114465:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881002.114468:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d6fc4b5d. 00000100:00000001:1.0:1788881002.114474:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881002.114475:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881002.114479:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881002.114479:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881002.114484:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881002.114485:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000078d5a15d x1875776780702848/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881018 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881002.114492:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881002.114495:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788881002.114496:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881002.114499:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788881002.114501:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788881002.114502:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000356bd8da. 00000100:00000001:0.0:1788881002.114504:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881002.114508:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788881002.114511:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788881002.114512:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788881002.114515:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881002.114516:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000eaa15d72. 02000000:00000001:1.0:1788881002.114520:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788881002.114521:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881002.114523:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000078d5a15d. 00000100:00000001:1.0:1788881002.114526:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881002.114527:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881002.114528:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881002.114533:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881002.114534:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881002.114536:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881002.114538:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881002.174338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881002.174346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881002.174354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881002.174355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881002.174421:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881002.174427:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881002.174430:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881003.198375:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881003.198383:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881003.198390:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881003.198392:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881003.198458:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881003.198462:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881003.198464:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881004.222379:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881004.222388:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881004.222396:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881004.222398:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881004.222473:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881004.222478:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881004.222482:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881005.246343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881005.246356:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881005.246359:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881005.246363:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881005.246369:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881005.246372:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881005.246374:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881005.246377:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881005.246382:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881005.246383:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881005.246440:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881005.246445:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881005.246448:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881006.270364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881006.270386:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881006.270395:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881006.270396:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881006.270457:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881006.270462:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881006.270465:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881007.230394:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306638288, at 4306638336 00000100:00000040:1.0:1788881007.230394:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306638288, at 4306638336 00000100:00000040:2.0:1788881007.230395:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788881007.230395:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306638288, at 4306638336 00000100:00000001:2.0:1788881007.230417:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881007.230421:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000837757ae. 00000100:00000010:0.0:1788881007.230423:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000945234cf. 00000100:00000010:2.0:1788881007.230429:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000095fca6f3. 00000100:00000010:3.0:1788881007.230429:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000dba5f48d. 00000020:00000040:2.0:1788881007.230437:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788881007.230443:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881007.230448:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881007.230455:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881007.230466:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006aca3ff7. 00000100:00000001:2.0:1788881007.230475:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881007.230490:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788881007.230505:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788881007.230509:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881007.230518:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000095fca6f3] to pc [ptlrpcd_00_00+0] req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881007.230549:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881007.230552:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881007.230556:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881007.230559:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000175072aa. 00000020:00000040:2.0:1788881007.230563:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788881007.230565:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881007.230567:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881007.230570:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881007.230573:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c52725cf. 00000100:00000001:2.0:1788881007.230592:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881007.230593:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881007.230597:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788881007.230598:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788881007.230598:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:3.0:1788881007.230605:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881007.230606:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881007.230626:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788881007.230628:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881007.230630:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:2.0:1788881007.230634:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000175072aa] to pc [ptlrpcd_00_01+1] req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881007.230635:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881007.230641:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881007.230646:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881007.230651:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788881007.230658:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881007.230660:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881007.230666:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881007.230672:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1788881007.230677:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:1.0:1788881007.230682:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881007.230684:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881007.230688:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788881007.230689:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881007.230691:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000010:2.0:1788881007.230694:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000025994245. 00000100:00000001:1.0:1788881007.230697:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788881007.230700:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788881007.230706:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788881007.230706:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:3.0:1788881007.230707:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881007.230711:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881007.230717:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881007.230719:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881007.230720:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788881007.230722:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881007.230728:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881007.230733:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788881007.230739:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000009b8e844. 00000100:00100000:3.0:1788881007.230739:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780702976:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881007.230749:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881007.230758:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788881007.230763:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:3.0:1788881007.230772:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788881007.230781:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:3.0:1788881007.230782:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881007.230787:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881007.230791:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:1.0:1788881007.230799:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788881007.230800:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000010:3.0:1788881007.230800:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000058b4ee38. 00000100:00000001:2.0:1788881007.230807:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881007.230807:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881007.230808:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788881007.230819:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000025994245] to pc [ptlrpcd_00_02+2] req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881007.230820:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881007.230821:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000df34776e. 00000400:00000010:3.0:1788881007.230830:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000100:00100000:1.0:1788881007.230837:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780703104:192.168.204.111@tcp:400: 00000100:00000200:3.0:1788881007.230842:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780702976, portal 25 00000100:00000001:3.0:1788881007.230852:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881007.230857:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788881007.230858:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938425600 : -126823771126016 : ffff8ca78a2c4700) 02000000:00000001:1.0:1788881007.230869:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881007.230875:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881007.230876:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881023 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881007.230881:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881007.230888:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788881007.230892:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788881007.230897:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:1.0:1788881007.230900:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000009cd0104b. 00000100:00000040:2.0:1788881007.230900:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788881007.230900:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d514436. 02000000:00000001:1.0:1788881007.230911:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788881007.230913:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780702976, offset 0 00000100:00000040:2.0:1788881007.230914:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788881007.230920:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1788881007.230923:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000400:00000200:3.0:1788881007.230923:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788881007.230924:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11976) 00000400:00000010:1.0:1788881007.230937:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000400:00000200:3.0:1788881007.230938:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788881007.230941:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881007.230946:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:1.0:1788881007.230950:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780703104, portal 10 00000400:00000200:3.0:1788881007.230953:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:2.0:1788881007.230954:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306638288, at 4306638336 00000100:00000040:0.0:1788881007.230959:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881007.230959:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881007.230965:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938435456 : -126823771116160 : ffff8ca78a2c6d80) 00000400:00000200:3.0:1788881007.230969:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:2.0:1788881007.230970:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000fbefa178. 00000400:00000200:3.0:1788881007.230981:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:1.0:1788881007.230987:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881007.230991:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788881007.231001:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788881007.231008:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [570:570:256:4294967295] 192.168.204.111@tcp LPNI seq info [570:570:8:4294967295] 00000100:00000001:2.0:1788881007.231011:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881007.231012:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881007.231022:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881007.231035:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788881007.231040:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788881007.231044:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788881007.231056:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788881007.231056:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788881007.231057:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881007.231069:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000061cfcb81. 00000100:00100000:0.0:1788881007.231075:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780703232:192.168.204.111@tcp:400: 00000800:00000200:3.0:1788881007.231078:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:1.0:1788881007.231079:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780703104, offset 0 00000800:00000010:3.0:1788881007.231088:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:0.0:1788881007.231092:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788881007.231100:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000800:00000200:3.0:1788881007.231108:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:0.0:1788881007.231109:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881007.231109:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:0.0:1788881007.231131:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:3.0:1788881007.231134:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788881007.231139:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 02000000:00000010:0.0:1788881007.231141:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000015664212. 00000400:00000200:1.0:1788881007.231142:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:0.0:1788881007.231150:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881007.231160:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000400:00000200:1.0:1788881007.231160:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788881007.231166:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881007.231169:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000400:00000200:1.0:1788881007.231169:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788881007.231174:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306643407 00000100:00000200:0.0:1788881007.231177:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780703232, portal 10 00000400:00000200:1.0:1788881007.231178:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:3.0:1788881007.231181:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000dba5f48d. 00000400:00000200:1.0:1788881007.231190:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788881007.231192:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881007.231194:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788881007.231197:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938428288 : -126823771123328 : ffff8ca78a2c5180) 00000400:00000200:1.0:1788881007.231206:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [571:571:255:4294967295] 192.168.204.111@tcp LPNI seq info [571:571:7:4294967295] 00000100:00000001:3.0:1788881007.231206:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306643407 00000100:00000040:0.0:1788881007.231215:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:3.0:1788881007.231223:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c451d2b9. 00000100:00000001:3.0:1788881007.231230:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:1.0:1788881007.231279:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788881007.231289:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306643408 00000100:00000001:0.0:1788881007.231293:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1788881007.231295:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:1.0:1788881007.231296:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788881007.231297:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e62b7132. 00000800:00000010:2.0:1788881007.231306:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000040:0.0:1788881007.231308:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881007.231310:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000010:1.0:1788881007.231313:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000131c1a55. 00000400:00000200:2.0:1788881007.231317:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788881007.231318:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306643408 00000400:00000010:0.0:1788881007.231319:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cffd24. 00000100:00000010:3.0:1788881007.231332:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001f846689. 00000100:00000200:0.0:1788881007.231333:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780703232, offset 0 00000800:00000200:1.0:1788881007.231333:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788881007.231333:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881007.231342:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788881007.231353:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788881007.231354:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d514436 00000400:00000010:2.0:1788881007.231361:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d514436. 00000800:00000200:1.0:1788881007.231363:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788881007.231365:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881007.231368:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788881007.231370:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881007.231375:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000131c1a55 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788881007.231380:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881007.231381:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788881007.231383:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881023 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881007.231394:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881007.231397:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881007.231404:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788881007.231405:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881007.231408:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 152795 usecs (153 jiffies) 00000100:00000040:2.0:1788881007.231415:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881023 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881007.231417:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788881007.231417:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306638490 00000100:00000001:1.0:1788881007.231429:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306643407 00000400:00000200:0.0:1788881007.231437:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788881007.231440:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306638490 00000100:00000001:2.0:1788881007.231440:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881007.231446:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788881007.231449:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306643407 00000400:00000200:0.0:1788881007.231452:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788881007.231456:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306638490 00000100:00000001:3.0:1788881007.231457:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788881007.231462:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788881007.231465:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306643408 00000100:00000001:3.0:1788881007.231466:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881007.231470:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306638490 00000100:00000001:3.0:1788881007.231475:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881007.231478:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306643408 00000100:00000001:1.0:1788881007.231484:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788881007.231485:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [572:572:254:4294967295] 192.168.204.111@tcp LPNI seq info [572:572:6:4294967295] 00000100:00000001:1.0:1788881007.231490:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881007.231497:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881007.231503:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881007.231509:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788881007.231518:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788881007.231523:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000087d24527. 00000800:00000200:0.0:1788881007.231529:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788881007.231536:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788881007.231540:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000087d24527 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788881007.231546:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881007.231550:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11487 usecs (12 jiffies) 00000800:00000200:2.0:1788881007.231551:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788881007.231562:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000131c1a55. 00000100:00000001:0.0:1788881007.231563:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306638349 00000400:00000200:2.0:1788881007.231569:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881007.231570:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306643407 00000400:00000200:2.0:1788881007.231574:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881007.231576:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000061cfcb81 00000400:00000010:2.0:1788881007.231578:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000061cfcb81. 00000100:00000001:2.0:1788881007.231580:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881007.231582:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881007.231592:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306638349 00000100:00000001:2.0:1788881007.231596:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881007.231600:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306643407 00000100:00000040:2.0:1788881007.231601:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881007.231613:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306638349 00000100:00000001:2.0:1788881007.231620:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881007.231621:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306643408 00000100:00000001:2.0:1788881007.231625:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788881007.231628:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306638349 00000100:00000001:0.0:1788881007.231633:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306643408 00000100:00000001:0.0:1788881007.231635:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881007.231639:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881007.231643:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881007.231644:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788881007.231650:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:2.0:1788881007.231652:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000087d24527. 00000100:00000001:0.0:1788881007.231659:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788881007.231659:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881007.231663:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788881007.231666:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881007.231670:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788881007.231673:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881007.231676:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881007.231678:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cffd24 00000100:00000001:0.0:1788881007.231682:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881007.231682:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cffd24. 00000100:00000001:2.0:1788881007.231691:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881007.231699:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881007.231706:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881007.231707:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881007.231715:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881007.231717:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788881007.233003:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881007.233013:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881007.233015:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881007.233017:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881007.233021:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881007.233030:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b11500 00000400:00000200:2.0:1788881007.233036:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1299 [1] + 224 00000800:00000001:2.0:1788881007.233040:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881007.233044:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881007.233046:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881007.233048:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881007.233052:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881007.233055:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881007.233061:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881007.233071:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881007.233083:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881007.233086:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881007.233197:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881007.233202:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788881007.233207:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788881007.233210:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:2.0:1788881007.233211:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000010:3.0:1788881007.233213:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000df34776e. 00000800:00000001:2.0:1788881007.233218:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881007.233219:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000800:00000001:2.0:1788881007.233227:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881007.233229:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881007.233248:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881007.233254:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11580 00000400:00000010:3.0:1788881007.233284:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:3.0:1788881007.233289:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788881007.233293:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x12a1 [1] + 224 00000100:00000200:3.0:1788881007.233297:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881007.233299:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881007.233304:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881007.233308:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881007.233315:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881007.233319:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788881007.233320:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881007.233324:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881007.233345:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881007.233363:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881007.233364:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788881007.233366:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881007.233370:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788881007.233371:0:201286:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788881007.233375:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788881007.233376:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788881007.233381:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788881007.233382:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788881007.233384:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881007.233386:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881007.233392:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 02000000:00000001:3.0:1788881007.233393:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881007.233399:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11600 00000100:00000001:1.0:1788881007.233402:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:3.0:1788881007.233402:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=85 v=5 (5 5 5 5) 00000100:00000001:1.0:1788881007.233407:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788881007.233408:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x12a9 [1] + 224 00000100:00001000:3.0:1788881007.233416:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=85 v=5 (1 1 1 1) 00000100:00000001:3.0:1788881007.233422:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000001:1.0:1788881007.233427:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788881007.233427:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788881007.233427:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788881007.233436:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00010000:00000001:3.0:1788881007.233437:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000010:1.0:1788881007.233440:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000800:00000001:2.0:1788881007.233442:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788881007.233442:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881007.233448:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000400:00000200:2.0:1788881007.233451:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788881007.233451:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881007.233452:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:1.0:1788881007.233467:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788881007.233470:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:3.0:1788881007.233473:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881007.233476:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881007.233478:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881007.233485:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881007.233555:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788881007.233562:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881007.233563:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881007.233564:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881007.233569:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881007.233585:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881007.233642:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881007.233647:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1788881007.233648:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000095fca6f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780702976:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881007.233658:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881007.233665:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881007.233669:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881007.233671:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881007.233675:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788881007.233676:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881007.233679:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:3.0:1788881007.233687:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000095fca6f3 x1875776780702976/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881007.233688:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881007.233691:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:1.0:1788881007.233698:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=75 v=5 (5 5 5 5) 00000800:00000001:2.0:1788881007.233698:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881007.233702:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00001000:1.0:1788881007.233711:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=75 v=5 (1 1 1 1) 02000000:00000001:3.0:1788881007.233712:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788881007.233719:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881007.233719:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000010:3.0:1788881007.233720:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000058b4ee38. 00000100:00000001:1.0:1788881007.233722:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881007.233725:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:1.0:1788881007.233727:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788881007.233733:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881007.233734:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000001:0.0:1788881007.233739:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000020:00000001:3.0:1788881007.233741:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000001:0.0:1788881007.233742:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788881007.233743:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000010:0.0:1788881007.233748:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000100:00000040:1.0:1788881007.233751:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000020:00000040:3.0:1788881007.233751:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000200:0.0:1788881007.233753:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788881007.233755:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788881007.233759:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788881007.233759:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788881007.233761:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881007.233770:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788881007.233772:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000040:1.0:1788881007.233774:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788881007.233782:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006aca3ff7. 02000000:00000001:3.0:1788881007.233797:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788881007.233809:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788881007.233809:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000200:0.0:1788881007.233810:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:3.0:1788881007.233817:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000095fca6f3. 00000100:00000001:1.0:1788881007.233819:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881007.233823:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881007.233834:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788881007.233834:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:1.0:1788881007.233835:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881007.233839:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881007.233844:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788881007.233844:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881007.233848:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881007.233850:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881007.233856:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881007.233857:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881007.233864:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788881007.233871:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000175072aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780703104:192.168.204.111@tcp:400: 00000100:00001000:0.0:1788881007.233872:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=85 v=5 (5 5 5 5) 00000100:00000001:3.0:1788881007.233874:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881007.233878:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881007.233883:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881007.233884:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881007.233889:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881007.233891:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=19 v=5 (1 1 0 0) 00000100:00000001:1.0:1788881007.233892:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881007.233897:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:1.0:1788881007.233900:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000175072aa x1875776780703104/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881007.233901:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881007.233907:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788881007.233913:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00010000:00080000:0.0:1788881007.233915:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881007.233917:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788881007.233920:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000009cd0104b. 00010000:00000001:0.0:1788881007.233926:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881007.233926:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788881007.233929:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788881007.233930:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000020:00000040:1.0:1788881007.233931:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000040:0.0:1788881007.233933:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000020:00000001:1.0:1788881007.233935:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788881007.233936:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000010:1.0:1788881007.233938:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c52725cf. 00000100:00000001:0.0:1788881007.233939:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881007.233946:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881007.233946:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788881007.233950:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881007.233953:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000175072aa. 00000100:00000001:0.0:1788881007.233958:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881007.233959:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881007.233960:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881007.233962:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881007.233963:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881007.233967:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881007.233968:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881007.233970:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881007.233973:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788881007.233974:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881007.233976:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881007.233984:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000025994245 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780703232:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881007.233991:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881007.233993:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881007.233994:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000025994245 x1875776780703232/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881023 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881007.233998:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881007.233999:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881007.234001:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000015664212. 02000000:00000001:0.0:1788881007.234003:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788881007.234004:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881007.234004:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881007.234006:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881007.234007:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000009b8e844. 02000000:00000001:0.0:1788881007.234008:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881007.234010:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881007.234010:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000025994245. 00000100:00000001:0.0:1788881007.234013:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881007.234014:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881007.234015:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881007.234018:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881007.234019:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881007.234020:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881007.234021:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881007.294396:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881007.294408:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881007.294417:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881007.294418:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881007.294501:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881007.294509:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881007.294514:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881008.318356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881008.318364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881008.318370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881008.318374:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881008.318376:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881008.318378:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881008.318381:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881008.318384:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881008.318385:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881008.318387:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881008.318438:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881008.318443:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881008.318447:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881009.342358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881009.342367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881009.342373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881009.342374:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881009.342432:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881009.342436:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881009.342439:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881010.366321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881010.366327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881010.366332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881010.366332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881010.366399:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881010.366404:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881010.366407:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881011.390326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881011.390332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881011.390336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881011.390337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881011.390405:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881011.390409:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881011.390412:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788881012.350363:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306643407, at 4306643456 00000100:00000040:0.0:1788881012.350365:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306643407, at 4306643456 00000100:00000010:0.0:1788881012.350380:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000dba5f48d. 00000100:00000040:2.0:1788881012.350380:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:1.0:1788881012.350382:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c451d2b9. 00000100:00000001:2.0:1788881012.350392:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881012.350401:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000520cae44. 00000020:00000040:2.0:1788881012.350407:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788881012.350413:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881012.350418:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881012.350424:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881012.350433:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000008eccda10. 00000100:00000001:2.0:1788881012.350441:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881012.350452:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788881012.350466:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:3.0:1788881012.350466:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306643408, at 4306643456 00000100:00000001:2.0:1788881012.350469:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881012.350480:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000520cae44] to pc [ptlrpcd_00_03+3] req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:3.0:1788881012.350486:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001f846689. 00000100:00000001:2.0:1788881012.350520:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881012.350523:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881012.350528:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881012.350531:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002084a2b4. 00000020:00000040:2.0:1788881012.350535:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881012.350536:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881012.350538:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788881012.350541:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788881012.350543:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788881012.350545:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:2.0:1788881012.350547:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881012.350551:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881012.350554:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:2.0:1788881012.350554:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5d1e8ed. 00000100:00000001:0.0:1788881012.350561:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881012.350562:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881012.350563:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788881012.350568:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881012.350573:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:2.0:1788881012.350574:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788881012.350576:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788881012.350582:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788881012.350588:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780703360:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881012.350593:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788881012.350597:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881012.350598:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881012.350600:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788881012.350605:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a2f4800f. 02000000:00000001:1.0:1788881012.350609:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881012.350615:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000100:00000001:2.0:1788881012.350617:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788881012.350620:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881012.350626:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002084a2b4] to pc [ptlrpcd_00_00+0] req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788881012.350628:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000100:00000200:1.0:1788881012.350639:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780703360, portal 25 00000100:00000001:1.0:1788881012.350642:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881012.350643:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938434560 : -126823771117056 : ffff8ca78a2c6a00) 00000100:00000040:1.0:1788881012.350648:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881028 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881012.350654:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788881012.350656:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881012.350657:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881012.350660:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d71e0867. 00000100:00000040:2.0:1788881012.350662:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:1.0:1788881012.350663:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780703360, offset 0 00000100:00000001:2.0:1788881012.350671:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000400:00000200:1.0:1788881012.350675:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788881012.350677:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000884e74b8. 00000100:00000001:0.0:1788881012.350682:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:2.0:1788881012.350682:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788881012.350687:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000200:1.0:1788881012.350689:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:2.0:1788881012.350690:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788881012.350692:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788881012.350695:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881012.350697:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881012.350698:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1788881012.350698:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:2.0:1788881012.350701:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000011e1ab42. 00000100:00000001:3.0:1788881012.350707:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881012.350708:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788881012.350709:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881012.350716:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881012.350718:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788881012.350725:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:2.0:1788881012.350726:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:1.0:1788881012.350736:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [573:573:256:4294967295] 192.168.204.111@tcp LPNI seq info [573:573:8:4294967295] 00000100:00000001:0.0:1788881012.350752:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881012.350756:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881012.350756:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788881012.350762:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788881012.350762:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881012.350764:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788881012.350766:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881012.350772:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881012.350772:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000884e74b8] to pc [ptlrpcd_00_01+1] req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788881012.350778:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00100000:0.0:1788881012.350781:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780703488:192.168.204.111@tcp:400: 00000800:00000010:1.0:1788881012.350786:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000004e0f6846. 00000100:00000001:0.0:1788881012.350789:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788881012.350794:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881012.350798:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881012.350798:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:0.0:1788881012.350801:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788881012.350805:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000099f3ac40. 00000800:00000200:1.0:1788881012.350809:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:0.0:1788881012.350810:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881012.350810:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881012.350814:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000004e0f6846 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:2.0:1788881012.350817:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:0.0:1788881012.350818:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000400:00000010:0.0:1788881012.350824:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000040:2.0:1788881012.350826:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000200:0.0:1788881012.350828:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780703488, portal 10 00000100:00000001:1.0:1788881012.350829:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881012.350833:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881012.350834:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306648527 00000100:00000040:2.0:1788881012.350835:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11981) 00000100:00000001:0.0:1788881012.350837:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938429184 : -126823771122432 : ffff8ca78a2c5500) 00000100:00000010:1.0:1788881012.350844:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d9f2f285. 00000100:00000001:3.0:1788881012.350847:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788881012.350848:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881012.350851:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788881012.350855:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881012.350859:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306648527 00000100:00000001:3.0:1788881012.350864:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000010:1.0:1788881012.350866:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000100:00000001:0.0:1788881012.350873:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788881012.350875:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788881012.350879:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306648527 00000100:00000040:0.0:1788881012.350881:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788881012.350882:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881012.350887:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306643408 00000400:00000010:0.0:1788881012.350890:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d6cef011. 00000100:00000001:1.0:1788881012.350895:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306648527 00000100:00000200:0.0:1788881012.350901:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780703488, offset 0 00000100:00000010:1.0:1788881012.350907:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 00000100:00000001:1.0:1788881012.350918:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788881012.350926:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788881012.350930:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881012.350938:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881012.350938:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:0.0:1788881012.350943:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788881012.350944:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881012.350949:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788881012.350951:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:2.0:1788881012.350955:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:0.0:1788881012.350958:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788881012.350959:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881012.350962:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:2.0:1788881012.350962:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000004e0f6846. 00000400:00000200:0.0:1788881012.350966:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788881012.350967:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000001:3.0:1788881012.350969:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:2.0:1788881012.350971:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788881012.350972:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:0.0:1788881012.350973:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788881012.350975:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881012.350976:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881012.350982:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881012.350989:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881012.350990:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881012.350992:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788881012.350993:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [574:574:255:4294967295] 192.168.204.111@tcp LPNI seq info [574:574:7:4294967295] 00000100:00100000:3.0:1788881012.351007:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780703616:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788881012.351038:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788881012.351050:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788881012.351052:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d71e0867 00000800:00000010:0.0:1788881012.351053:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e494c472. 00000400:00000010:2.0:1788881012.351057:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d71e0867. 00000800:00000200:0.0:1788881012.351061:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788881012.351066:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:0.0:1788881012.351068:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788881012.351071:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e494c472 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788881012.351073:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788881012.351075:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881028 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881012.351080:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:0.0:1788881012.351082:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 151923 usecs (152 jiffies) 00000100:00000001:0.0:1788881012.351087:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306643608 00000100:00000001:0.0:1788881012.351094:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306648527 02000000:00000001:3.0:1788881012.351095:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788881012.351102:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306643608 02000000:00000001:3.0:1788881012.351102:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881012.351105:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881012.351116:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306648527 02000000:00000001:3.0:1788881012.351117:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788881012.351120:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881028 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881012.351124:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306643608 02000000:00000010:3.0:1788881012.351128:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000006010d39b. 00000100:00000001:0.0:1788881012.351129:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306648527 00000100:00000001:0.0:1788881012.351137:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306643608 02000000:00000001:3.0:1788881012.351137:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881012.351140:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881012.351144:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306648527 00000100:00000001:2.0:1788881012.351147:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788881012.351148:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788881012.351150:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000df34776e. 00000100:00000001:0.0:1788881012.351152:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881012.351160:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881012.351162:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788881012.351164:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000100:00000001:0.0:1788881012.351166:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881012.351170:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788881012.351173:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881012.351175:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881012.351176:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788881012.351177:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780703616, portal 10 00000100:00000001:0.0:1788881012.351178:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881012.351186:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881012.351189:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938423808 : -126823771127808 : ffff8ca78a2c4000) 00000100:00000040:3.0:1788881012.351201:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788881012.351205:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788881012.351211:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e494c472. 00000100:00000001:3.0:1788881012.351219:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:2.0:1788881012.351221:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:3.0:1788881012.351228:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788881012.351265:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000035d99bc2. 00000400:00000200:2.0:1788881012.351271:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:3.0:1788881012.351272:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780703616, offset 0 00000400:00000200:2.0:1788881012.351285:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d6cef011 00000400:00000010:2.0:1788881012.351288:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d6cef011. 00000100:00000001:2.0:1788881012.351292:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:3.0:1788881012.351297:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788881012.351299:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881012.351309:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881012.351312:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881012.351319:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881012.351320:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:3.0:1788881012.351356:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788881012.351360:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788881012.351363:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788881012.351366:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788881012.351368:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788881012.351370:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [575:575:256:4294967295] 192.168.204.111@tcp LPNI seq info [575:575:8:4294967295] 00000400:00000200:3.0:1788881012.351379:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788881012.351383:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788881012.351387:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000800:00000200:3.0:1788881012.351398:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788881012.351404:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788881012.351407:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788881012.351421:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881012.351425:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11441 usecs (12 jiffies) 00000100:00000001:3.0:1788881012.351428:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306643469 00000100:00000001:3.0:1788881012.351431:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306648527 00000100:00000001:3.0:1788881012.351433:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306643469 00000100:00000001:3.0:1788881012.351435:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306648527 00000100:00000001:3.0:1788881012.351437:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306643469 00000100:00000001:3.0:1788881012.351439:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306648527 00000100:00000001:3.0:1788881012.351441:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306643469 00000100:00000001:3.0:1788881012.351442:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306648527 00000100:00000001:3.0:1788881012.351444:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881012.351446:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881012.351449:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881012.351452:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881012.351458:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881012.351460:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881012.351463:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881012.351464:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881012.351465:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881012.351467:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881012.351491:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788881012.351495:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:2.0:1788881012.351500:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881012.351506:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881012.351509:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000035d99bc2 00000400:00000010:2.0:1788881012.351511:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000035d99bc2. 00000100:00000001:2.0:1788881012.351514:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881012.351518:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881012.351537:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881012.351539:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881012.351543:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881012.351545:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788881012.352651:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881012.352663:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881012.352666:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881012.352669:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881012.352676:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881012.352689:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b11680 00000400:00000200:2.0:1788881012.352696:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x12b1 [1] + 224 00000800:00000001:2.0:1788881012.352702:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881012.352706:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881012.352709:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881012.352712:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881012.352717:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881012.352720:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881012.352730:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881012.352747:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881012.352755:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788881012.352773:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881012.352776:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788881012.352781:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881012.352783:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788881012.352784:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:1.0:1788881012.352787:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000400:00000010:1.0:1788881012.352788:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:1.0:1788881012.352791:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881012.352793:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881012.352800:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881012.352804:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881012.352806:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881012.352809:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881012.352810:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788881012.352812:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881012.352816:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881012.352818:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788881012.352818:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881012.352821:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881012.352824:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881012.352828:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:1.0:1788881012.352829:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=90 v=5 (5 5 5 5) 00000100:00001000:1.0:1788881012.352836:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=90 v=5 (1 1 1 1) 00000100:00000001:1.0:1788881012.352838:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881012.352840:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881012.352844:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:2.0:1788881012.352844:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11700 00010000:00000001:1.0:1788881012.352847:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881012.352851:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881012.352855:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x12b9 [1] + 224 00000100:00000040:1.0:1788881012.352856:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881012.352866:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788881012.352867:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881012.352870:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881012.352871:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881012.352874:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788881012.352878:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000520cae44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780703360:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881012.352881:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881012.352885:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881012.352889:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000520cae44 x1875776780703360/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881012.352893:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881012.352894:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788881012.352897:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000200:2.0:1788881012.352899:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000010:1.0:1788881012.352900:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a2f4800f. 02000000:00000001:1.0:1788881012.352905:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788881012.352907:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000200:2.0:1788881012.352909:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000020:00000040:1.0:1788881012.352910:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788881012.352915:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881012.352917:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000008eccda10. 00000100:00000001:2.0:1788881012.352919:0:201279:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:1.0:1788881012.352920:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788881012.352923:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881012.352924:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000520cae44. 00000100:00000200:2.0:1788881012.352926:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881012.352929:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881012.352932:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881012.352936:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881012.352942:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881012.352944:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881012.352947:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881012.352948:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881012.352950:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881012.352971:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881012.352976:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881012.352979:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881012.352981:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881012.352983:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881012.352987:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881012.352994:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11780 00000400:00000200:2.0:1788881012.353000:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x12c1 [1] + 224 00000800:00000001:2.0:1788881012.353005:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881012.353008:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881012.353012:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788881012.353012:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788881012.353015:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788881012.353016:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:0.0:1788881012.353018:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788881012.353034:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:0.0:1788881012.353035:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000400:00000200:0.0:1788881012.353040:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788881012.353047:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788881012.353049:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788881012.353050:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788881012.353054:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881012.353059:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881012.353069:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881012.353086:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881012.353088:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788881012.353088:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881012.353093:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881012.353094:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881012.353096:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881012.353099:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881012.353102:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881012.353107:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=80 v=5 (5 5 5 5) 00000100:00001000:0.0:1788881012.353111:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=80 v=5 (1 1 1 1) 00000100:00000001:0.0:1788881012.353113:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881012.353114:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881012.353115:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788881012.353117:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881012.353118:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881012.353120:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788881012.353122:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881012.353125:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000800:00000001:2.0:1788881012.353125:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881012.353129:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881012.353132:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881012.353133:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881012.353140:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881012.353141:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881012.353142:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881012.353144:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881012.353151:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002084a2b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780703488:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881012.353156:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881012.353163:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788881012.353165:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788881012.353168:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002084a2b4 x1875776780703488/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881012.353170:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788881012.353177:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788881012.353180:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000001:3.0:1788881012.353180:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:0.0:1788881012.353188:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000010:3.0:1788881012.353188:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000df34776e. 00000400:00000200:3.0:1788881012.353196:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 02000000:00000010:0.0:1788881012.353197:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000099f3ac40. 00000400:00000010:3.0:1788881012.353201:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 02000000:00000001:0.0:1788881012.353210:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788881012.353210:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000001:0.0:1788881012.353219:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000200:3.0:1788881012.353221:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1788881012.353223:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881012.353227:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881012.353266:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5d1e8ed. 00000100:00000200:3.0:1788881012.353266:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881012.353278:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788881012.353286:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788881012.353287:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881012.353292:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002084a2b4. 00000100:00000001:3.0:1788881012.353293:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881012.353301:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788881012.353303:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881012.353307:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788881012.353308:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881012.353312:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881012.353313:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881012.353319:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881012.353322:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:3.0:1788881012.353322:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881012.353330:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788881012.353335:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=90 v=5 (5 5 5 5) 00000100:00000001:0.0:1788881012.353336:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881012.353340:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788881012.353342:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=24 v=5 (1 1 0 0) 00000100:00000001:3.0:1788881012.353347:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881012.353348:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881012.353350:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788881012.353355:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788881012.353360:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881012.353363:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788881012.353364:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788881012.353366:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788881012.353367:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881012.353369:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881012.353374:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881012.353376:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881012.353377:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881012.353379:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788881012.353384:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000884e74b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780703616:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881012.353394:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881012.353404:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881012.353406:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000884e74b8 x1875776780703616/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881028 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881012.353411:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788881012.353414:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788881012.353416:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000006010d39b. 02000000:00000001:3.0:1788881012.353418:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881012.353420:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881012.353421:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788881012.353423:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881012.353425:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000011e1ab42. 02000000:00000001:3.0:1788881012.353427:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881012.353429:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881012.353430:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000884e74b8. 00000100:00000001:3.0:1788881012.353433:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881012.353434:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881012.353435:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881012.353438:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881012.353439:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881012.353441:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881012.353443:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881012.414319:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881012.414326:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881012.414331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881012.414332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881012.414386:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881012.414390:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881012.414392:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881013.438304:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881013.438310:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881013.438317:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881013.438319:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881013.438377:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881013.438381:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881013.438383:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881014.462319:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881014.462325:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881014.462329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881014.462330:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881014.462381:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881014.462385:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881014.462387:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881015.486377:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881015.486387:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881015.486390:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881015.486393:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881015.486398:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881015.486400:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881015.486402:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881015.486403:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881015.486412:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881015.486413:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881015.486493:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881015.486497:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881015.486499:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881016.510353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881016.510362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881016.510369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881016.510370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881016.510449:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881016.510452:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881016.510455:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881017.470320:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306648527, at 4306648576 00000100:00000040:1.0:1788881017.470320:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306648527, at 4306648576 00000100:00000040:3.0:1788881017.470320:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306648527, at 4306648576 00000100:00000040:2.0:1788881017.470326:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306648527, at 4306648576 00000100:00000010:3.0:1788881017.470339:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 00000100:00000010:0.0:1788881017.470341:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000d9f2f285. 00000100:00000010:2.0:1788881017.470347:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e62b7132. 00000100:00000010:1.0:1788881017.470348:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000040:2.0:1788881017.470363:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881017.470385:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881017.470390:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000246f7e1b. 00000020:00000040:2.0:1788881017.470395:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788881017.470399:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881017.470401:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881017.470406:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881017.470413:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f9812869. 00000100:00000001:2.0:1788881017.470420:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881017.470432:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788881017.470440:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788881017.470443:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881017.470447:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000246f7e1b] to pc [ptlrpcd_00_02+2] req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881017.470468:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881017.470470:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881017.470472:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881017.470474:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000629edf55. 00000020:00000040:2.0:1788881017.470475:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788881017.470477:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881017.470478:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881017.470480:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881017.470482:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 00000100:00000001:2.0:1788881017.470484:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881017.470487:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881017.470490:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881017.470494:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:1.0:1788881017.470496:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:2.0:1788881017.470496:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881017.470499:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000629edf55] to pc [ptlrpcd_00_03+3] req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881017.470500:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881017.470501:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881017.470506:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881017.470509:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881017.470512:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.470514:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881017.470516:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881017.470517:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881017.470520:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881017.470520:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788881017.470525:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881017.470525:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788881017.470529:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881017.470531:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881017.470534:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c7465fc8. 00000100:00000001:1.0:1788881017.470540:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000020:00000040:2.0:1788881017.470540:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881017.470543:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881017.470544:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881017.470546:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881017.470547:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788881017.470547:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881017.470549:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 02000000:00000001:1.0:1788881017.470550:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881017.470553:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881017.470556:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788881017.470561:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881017.470567:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:1.0:1788881017.470568:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780703872:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881017.470570:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881017.470573:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788881017.470576:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c7465fc8] to pc [ptlrpcd_00_00+0] req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881017.470579:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881017.470583:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881017.470585:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788881017.470589:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007e76baee. 02000000:00000001:1.0:1788881017.470591:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881017.470592:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881017.470595:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:1.0:1788881017.470597:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000100:00000040:2.0:1788881017.470598:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:2.0:1788881017.470604:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11986) 00000400:00000010:1.0:1788881017.470606:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000100:00000200:1.0:1788881017.470614:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780703872, portal 10 00000100:00000001:1.0:1788881017.470616:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881017.470618:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881017.470618:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938432768 : -126823771118848 : ffff8ca78a2c6300) 00000100:00000001:3.0:1788881017.470619:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881017.470622:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881017.470623:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881017.470627:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788881017.470629:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788881017.470636:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881017.470637:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788881017.470640:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881017.470643:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000641230df. 00000100:00000200:1.0:1788881017.470647:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780703872, offset 0 00000400:00000200:1.0:1788881017.470653:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788881017.470666:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788881017.470667:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881017.470670:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881017.470672:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788881017.470674:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788881017.470677:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881017.470680:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881017.470681:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00100000:0.0:1788881017.470685:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780704000:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788881017.470688:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788881017.470691:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788881017.470695:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:0.0:1788881017.470696:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881017.470699:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881017.470703:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [576:576:256:4294967295] 192.168.204.111@tcp LPNI seq info [576:576:8:4294967295] 02000000:00000001:0.0:1788881017.470704:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788881017.470709:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005bd099ee. 02000000:00000001:0.0:1788881017.470712:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881017.470716:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000400:00000010:0.0:1788881017.470723:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000400:00000200:1.0:1788881017.470723:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:0.0:1788881017.470729:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780704000, portal 10 00000100:00000001:0.0:1788881017.470732:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:1.0:1788881017.470735:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881017.470736:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938424704 : -126823771126912 : ffff8ca78a2c4380) 00000800:00000010:1.0:1788881017.470744:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000da26fd2f. 00000100:00000040:0.0:1788881017.470747:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788881017.470755:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788881017.470761:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788881017.470765:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788881017.470766:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:0.0:1788881017.470769:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a09430ba. 00000800:00000200:1.0:1788881017.470770:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000da26fd2f type 1, nob 320 niov 1 nkiov 1 00000100:00000200:0.0:1788881017.470774:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780704000, offset 0 00000400:00000200:0.0:1788881017.470780:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881017.470782:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881017.470787:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 151058 usecs (152 jiffies) 00000400:00000200:0.0:1788881017.470788:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788881017.470791:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306648728 00000400:00000200:0.0:1788881017.470794:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:1.0:1788881017.470796:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000400:00000200:0.0:1788881017.470800:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788881017.470802:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000400:00000200:0.0:1788881017.470805:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788881017.470807:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306648728 00000400:00000200:0.0:1788881017.470809:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:1.0:1788881017.470811:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000100:00000001:1.0:1788881017.470815:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000400:00000200:0.0:1788881017.470816:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [577:577:255:4294967295] 192.168.204.111@tcp LPNI seq info [577:577:7:4294967295] 00000100:00000001:1.0:1788881017.470818:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306648728 00000100:00000010:1.0:1788881017.470822:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000100:00000001:1.0:1788881017.470825:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788881017.470827:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306648728 00000100:00000010:1.0:1788881017.470831:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000069720654. 00000100:00000001:1.0:1788881017.470835:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000400:00000200:0.0:1788881017.470836:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788881017.470839:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881017.470841:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881017.470843:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788881017.470847:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788881017.470855:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881017.470857:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788881017.470859:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881017.470861:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000010:0.0:1788881017.470862:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000050c5542b. 02000000:00000001:1.0:1788881017.470863:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788881017.470867:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788881017.470870:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780703744:192.168.204.111@tcp:400: 00000800:00000200:0.0:1788881017.470873:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:0.0:1788881017.470878:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788881017.470881:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000050c5542b type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788881017.470882:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788881017.470888:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881017.470889:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881017.470891:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788881017.470892:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881017.470893:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 02000000:00000010:1.0:1788881017.470894:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008cdf44b0. 00000100:00000040:0.0:1788881017.470896:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11392 usecs (12 jiffies) 02000000:00000001:1.0:1788881017.470899:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881017.470900:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306648588 00000800:00000010:2.0:1788881017.470901:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000da26fd2f. 00000400:00000010:1.0:1788881017.470903:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000831ba52e. 00000100:00000001:0.0:1788881017.470905:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306648728 00000100:00000001:0.0:1788881017.470909:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306648588 00000400:00000010:1.0:1788881017.470909:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000066e8f89d. 00000100:00000001:0.0:1788881017.470911:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306648728 00000400:00000200:2.0:1788881017.470911:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:1.0:1788881017.470913:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780703744, portal 25 00000100:00000001:0.0:1788881017.470917:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306648588 00000100:00000001:1.0:1788881017.470918:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881017.470920:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938431872 : -126823771119744 : ffff8ca78a2c5f80) 00000100:00000001:0.0:1788881017.470930:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306648728 00000100:00000001:0.0:1788881017.470934:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306648588 00000100:00000040:1.0:1788881017.470936:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881033 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788881017.470937:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881017.470940:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306648728 00000100:00000001:0.0:1788881017.470947:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788881017.470948:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000641230df 00000100:00000001:1.0:1788881017.470949:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:2.0:1788881017.470951:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000641230df. 00000100:00000001:0.0:1788881017.470953:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881017.470957:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881017.470958:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788881017.470959:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881017.470962:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000062e8e6f4. 00000100:00000001:0.0:1788881017.470963:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788881017.470965:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881017.470969:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780703744, offset 0 00000100:00000001:0.0:1788881017.470970:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881017.470973:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788881017.470977:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881017.470980:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788881017.470981:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881017.470982:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881017.470982:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881017.470986:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881017.470988:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881017.470990:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788881017.470998:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788881017.471001:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788881017.471007:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788881017.471013:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788881017.471015:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881017.471019:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:1.0:1788881017.471020:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [578:578:254:4294967295] 192.168.204.111@tcp LPNI seq info [578:578:6:4294967295] 00000400:00000200:1.0:1788881017.471030:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788881017.471034:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788881017.471037:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000a371c7dc. 00000800:00000200:1.0:1788881017.471041:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788881017.471044:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788881017.471046:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000a371c7dc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788881017.471048:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881017.471050:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306653647 00000800:00000200:2.0:1788881017.471053:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788881017.471054:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306648728 00000800:00000010:2.0:1788881017.471058:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000050c5542b. 00000100:00000001:1.0:1788881017.471059:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306653647 00000100:00000001:1.0:1788881017.471064:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306648728 00000400:00000200:2.0:1788881017.471064:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788881017.471067:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306653647 00000100:00000001:1.0:1788881017.471071:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306648728 00000400:00000200:2.0:1788881017.471072:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881017.471076:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306653647 00000400:00000200:2.0:1788881017.471077:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a09430ba 00000100:00000001:1.0:1788881017.471080:0:201288:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306648728 00000400:00000010:2.0:1788881017.471080:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a09430ba. 00000100:00000001:2.0:1788881017.471083:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788881017.471084:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881017.471087:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788881017.471088:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881017.471090:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881017.471094:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.471099:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881017.471101:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788881017.471101:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788881017.471104:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788881017.471104:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881017.471106:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881017.471108:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881017.471112:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881017.471116:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881017.471118:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788881017.471144:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788881017.471146:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000a371c7dc. 00000400:00000200:2.0:1788881017.471148:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881017.471150:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881017.471152:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000062e8e6f4 00000400:00000010:2.0:1788881017.471153:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000062e8e6f4. 00000100:00000001:2.0:1788881017.471155:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881017.471156:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881033 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881017.471161:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881017.471162:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881033 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881017.471166:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881017.471166:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788881017.472098:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881017.472108:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881017.472115:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881017.472117:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881017.472122:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881017.472131:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11880 00000400:00000200:2.0:1788881017.472136:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x12c9 [1] + 224 00000800:00000001:2.0:1788881017.472142:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881017.472146:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881017.472148:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881017.472150:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881017.472153:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881017.472155:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881017.472162:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881017.472173:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881017.472179:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881017.472190:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881017.472191:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881017.472193:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881017.472197:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881017.472201:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11900 00000100:00000001:1.0:1788881017.472205:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788881017.472205:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x12d1 [1] + 224 00000100:00000001:1.0:1788881017.472208:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788881017.472211:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:1.0:1788881017.472214:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788881017.472215:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788881017.472217:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788881017.472218:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:1.0:1788881017.472220:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:2.0:1788881017.472224:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881017.472225:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000400:00000010:1.0:1788881017.472228:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:1.0:1788881017.472230:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788881017.472230:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881017.472250:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881017.472257:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881017.472264:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881017.472271:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881017.472274:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881017.472277:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.472281:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881017.472283:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881017.472285:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788881017.472287:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881017.472290:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788881017.472299:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=85 v=5 (5 5 5 5) 00000100:00001000:1.0:1788881017.472303:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=85 v=5 (1 1 1 1) 00000100:00000001:2.0:1788881017.472304:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881017.472305:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881017.472308:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881017.472310:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788881017.472313:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881017.472313:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788881017.472318:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788881017.472320:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:2.0:1788881017.472320:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.472323:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788881017.472326:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881017.472329:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881017.472333:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881017.472338:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881017.472340:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881017.472342:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000001:0.0:1788881017.472343:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788881017.472344:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788881017.472347:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:1.0:1788881017.472349:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788881017.472351:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000400:00000200:0.0:1788881017.472358:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000100:00100000:1.0:1788881017.472358:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000629edf55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780703872:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881017.472362:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881017.472363:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881017.472364:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000629edf55 x1875776780703872/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788881017.472369:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:1.0:1788881017.472371:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788881017.472375:0:201285:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:1.0:1788881017.472375:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788881017.472378:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007e76baee. 00000100:00000200:0.0:1788881017.472383:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881017.472383:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788881017.472385:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788881017.472388:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788881017.472392:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881017.472395:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 02000000:00000001:1.0:1788881017.472401:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000200:0.0:1788881017.472403:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881017.472403:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881017.472406:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000629edf55. 00000100:00000001:1.0:1788881017.472410:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881017.472413:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881017.472416:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788881017.472417:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881017.472418:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881017.472420:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881017.472422:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.472423:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788881017.472424:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.472425:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881017.472427:0:201285:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788881017.472428:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788881017.472429:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881017.472431:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788881017.472433:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881017.472435:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881017.472437:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881017.472441:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881017.472441:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:0.0:1788881017.472448:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=95 v=5 (5 5 5 5) 00000400:00000200:2.0:1788881017.472451:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b11800 00000100:00001000:0.0:1788881017.472456:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=29 v=5 (1 1 0 0) 00000400:00000200:2.0:1788881017.472459:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x12d9 [1] + 224 00000100:00000001:0.0:1788881017.472461:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881017.472464:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881017.472465:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:0.0:1788881017.472468:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788881017.472473:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881017.472479:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788881017.472482:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881017.472483:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881017.472488:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000200:2.0:1788881017.472489:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000040:0.0:1788881017.472491:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788881017.472493:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881017.472496:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881017.472506:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881017.472509:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881017.472510:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788881017.472517:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881017.472531:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881017.472535:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881017.472535:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881017.472537:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881017.472542:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881017.472543:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881017.472548:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.472551:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881017.472554:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788881017.472556:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881017.472557:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788881017.472560:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000831ba52e. 00000400:00000200:1.0:1788881017.472565:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000066e8f89d 00000400:00000010:1.0:1788881017.472568:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000066e8f89d. 00000100:00000001:1.0:1788881017.472572:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:0.0:1788881017.472575:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c7465fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780704000:192.168.204.111@tcp:400: 00000100:00000200:1.0:1788881017.472577:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881017.472585:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788881017.472589:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881017.472590:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881017.472596:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c7465fc8 x1875776780704000/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881017.472597:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881017.472599:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.472601:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881017.472603:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881017.472605:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788881017.472607:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881017.472610:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881017.472614:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00001000:1.0:1788881017.472614:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=95 v=5 (5 5 5 5) 02000000:00000001:0.0:1788881017.472618:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00001000:1.0:1788881017.472618:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=95 v=5 (1 1 1 1) 02000000:00000010:0.0:1788881017.472622:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005bd099ee. 00000100:00000001:1.0:1788881017.472623:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881017.472625:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881017.472628:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788881017.472629:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881017.472630:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788881017.472631:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881017.472633:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:1.0:1788881017.472635:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1788881017.472636:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881017.472642:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881017.472645:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 00000100:00000001:1.0:1788881017.472645:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881017.472649:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:0.0:1788881017.472650:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788881017.472651:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881017.472654:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:1.0:1788881017.472655:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788881017.472658:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c7465fc8. 00000100:00000001:0.0:1788881017.472665:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00100000:1.0:1788881017.472665:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000246f7e1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780703744:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881017.472669:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.472669:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.472674:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881017.472675:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881017.472676:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000246f7e1b x1875776780703744/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881033 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881017.472683:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881017.472686:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881017.472687:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788881017.472690:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788881017.472691:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788881017.472692:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008cdf44b0. 00000100:00000001:0.0:1788881017.472695:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881017.472697:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788881017.472700:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788881017.472701:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:1.0:1788881017.472703:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881017.472704:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f9812869. 02000000:00000001:1.0:1788881017.472707:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788881017.472708:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881017.472710:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000246f7e1b. 00000100:00000001:1.0:1788881017.472712:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881017.472714:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.472715:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881017.472719:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881017.472720:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881017.472722:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881017.472723:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881017.534322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881017.534331:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881017.534338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881017.534340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881017.534399:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881017.534404:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881017.534408:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881018.558348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881018.558355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881018.558361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881018.558363:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881018.558365:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881018.558368:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881018.558371:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881018.558373:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881018.558375:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881018.558376:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881018.558426:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881018.558430:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881018.558452:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881019.582345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881019.582353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881019.582359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881019.582360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881019.582427:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881019.582432:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881019.582435:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881020.606314:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881020.606320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881020.606325:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881020.606326:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881020.606366:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881020.606369:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881020.606371:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881021.630325:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881021.630330:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881021.630334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881021.630335:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881021.630380:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881021.630383:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881021.630385:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788881022.590285:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306653647, at 4306653696 00000100:00000010:3.0:1788881022.590299:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000069720654. 00000100:00000040:1.0:1788881022.590301:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306653647, at 4306653696 00000100:00000040:0.0:1788881022.590308:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306653647, at 4306653696 00000100:00000010:1.0:1788881022.590309:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000010:0.0:1788881022.590324:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000040:2.0:1788881022.591268:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306653647, at 4306653697 00000100:00000010:2.0:1788881022.591274:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000040:0.0:1788881022.591301:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788881022.591307:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788881022.591312:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000fd20c80. 00000020:00000040:0.0:1788881022.591317:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788881022.591321:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788881022.591323:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788881022.591327:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788881022.591334:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000fc8f9b7c. 00000100:00000001:0.0:1788881022.591341:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881022.591352:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788881022.591361:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788881022.591364:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881022.591370:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000fd20c80] to pc [ptlrpcd_00_01+1] req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881022.591395:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881022.591398:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788881022.591401:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788881022.591404:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d4e901df. 00000100:00000001:3.0:1788881022.591406:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:0.0:1788881022.591407:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881022.591411:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881022.591412:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788881022.591413:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788881022.591415:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788881022.591418:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881022.591419:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788881022.591424:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000003806e31f. 00000100:00000001:1.0:1788881022.591428:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881022.591430:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881022.591431:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881022.591432:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881022.591434:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881022.591437:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:0.0:1788881022.591438:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:3.0:1788881022.591440:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788881022.591444:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788881022.591450:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780704128:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881022.591456:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788881022.591457:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788881022.591460:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881022.591461:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788881022.591464:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881022.591465:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788881022.591467:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d4e901df] to pc [ptlrpcd_00_02+2] req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788881022.591469:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007b91b305. 02000000:00000001:3.0:1788881022.591471:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881022.591476:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000df34776e. 00000400:00000010:3.0:1788881022.591483:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000100:00000200:3.0:1788881022.591489:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780704128, portal 25 00000100:00000001:3.0:1788881022.591491:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881022.591492:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849357824 : -126822860193792 : ffff8ca7c0780000) 00000100:00000040:3.0:1788881022.591496:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881022.591501:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881022.591502:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788881022.591506:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788881022.591507:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:3.0:1788881022.591509:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000082116f14. 00000100:00000001:0.0:1788881022.591511:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000200:3.0:1788881022.591520:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780704128, offset 0 00000100:00000001:1.0:1788881022.591521:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:0.0:1788881022.591522:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000b4a4221. 00000100:00000001:2.0:1788881022.591524:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881022.591526:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881022.591527:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788881022.591529:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881022.591529:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:3.0:1788881022.591532:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881022.591534:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788881022.591537:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:1.0:1788881022.591538:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881022.591540:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788881022.591547:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000005f53e16f. 00000400:00000200:3.0:1788881022.591548:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788881022.591553:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881022.591558:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:3.0:1788881022.591559:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788881022.591561:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881022.591562:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788881022.591566:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:3.0:1788881022.591567:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:1.0:1788881022.591570:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788881022.591573:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:1.0:1788881022.591577:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881022.591579:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788881022.591580:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788881022.591583:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788881022.591585:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780704256:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788881022.591587:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [579:579:256:4294967295] 192.168.204.111@tcp LPNI seq info [579:579:8:4294967295] 00000100:00000040:0.0:1788881022.591591:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000b4a4221] to pc [ptlrpcd_00_03+3] req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881022.591594:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788881022.591598:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881022.591600:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881022.591605:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788881022.591610:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000005842920. 00000400:00000200:3.0:1788881022.591610:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788881022.591614:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881022.591614:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881022.591619:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000200:3.0:1788881022.591620:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881022.591621:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000831ba52e. 00000800:00000010:3.0:1788881022.591626:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000040:0.0:1788881022.591627:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:1.0:1788881022.591628:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000066e8f89d. 00000100:00000200:1.0:1788881022.591634:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780704256, portal 10 00000100:00000040:0.0:1788881022.591636:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11991) 00000800:00000200:3.0:1788881022.591637:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788881022.591640:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881022.591643:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849369472 : -126822860182144 : ffff8ca7c0782d80) 00000800:00000200:3.0:1788881022.591647:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788881022.591649:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:3.0:1788881022.591650:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000040:1.0:1788881022.591653:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881022.591654:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881022.591657:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788881022.591662:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881022.591666:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881022.591668:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788881022.591668:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306658768 00000100:00000010:3.0:1788881022.591675:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001f846689. 00000100:00000040:1.0:1788881022.591676:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881022.591681:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000010:1.0:1788881022.591683:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005767f9c9. 00000100:00000001:3.0:1788881022.591686:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306658768 00000100:00000200:1.0:1788881022.591687:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780704256, offset 0 00000100:00000010:3.0:1788881022.591691:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a9d0cfcc. 00000100:00000001:3.0:1788881022.591696:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000400:00000200:1.0:1788881022.591700:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881022.591702:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306658768 00000100:00000001:2.0:1788881022.591707:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000010:3.0:1788881022.591708:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000015631aee. 00000100:00000001:2.0:1788881022.591710:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881022.591711:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788881022.591713:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 02000000:00000001:2.0:1788881022.591715:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:0.0:1788881022.591717:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788881022.591717:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306658768 02000000:00000001:2.0:1788881022.591718:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788881022.591720:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:3.0:1788881022.591722:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014c076ca. 00000800:00000010:0.0:1788881022.591723:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 02000000:00000001:2.0:1788881022.591724:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881022.591726:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:1.0:1788881022.591727:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788881022.591731:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00100000:2.0:1788881022.591731:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780704384:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788881022.591733:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788881022.591736:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881022.591740:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788881022.591742:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788881022.591742:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881022.591744:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:2.0:1788881022.591744:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788881022.591748:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881022.591750:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [580:580:255:4294967295] 192.168.204.111@tcp LPNI seq info [580:580:7:4294967295] 00000100:00000001:3.0:1788881022.591752:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881022.591753:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788881022.591757:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000afd3911b. 00000100:00000001:3.0:1788881022.591757:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788881022.591762:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881022.591766:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ffb4c8a2. 00000100:00000001:3.0:1788881022.591766:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788881022.591769:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788881022.591770:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000010:2.0:1788881022.591771:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000078d4a4ff. 00000100:00000001:3.0:1788881022.591773:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788881022.591777:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780704384, portal 10 00000100:00000001:3.0:1788881022.591777:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788881022.591781:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000082116f14 00000800:00000200:1.0:1788881022.591781:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881022.591781:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788881022.591783:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849373056 : -126822860178560 : ffff8ca7c0783b80) 00000400:00000010:0.0:1788881022.591785:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000082116f14. 00000800:00000010:1.0:1788881022.591789:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000a371c7dc. 00000100:00000001:0.0:1788881022.591791:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000040:2.0:1788881022.591792:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788881022.591798:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:0.0:1788881022.591799:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881022.591806:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:1.0:1788881022.591810:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:2.0:1788881022.591812:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881022.591816:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881022.591818:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000a371c7dc type 1, nob 320 niov 1 nkiov 1 00000400:00000010:2.0:1788881022.591818:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f779aa21. 00000100:00000001:3.0:1788881022.591821:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:2.0:1788881022.591822:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780704384, offset 0 00000100:00000001:1.0:1788881022.591836:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881022.591842:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 150197 usecs (151 jiffies) 00000400:00000200:2.0:1788881022.591843:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881022.591849:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306653848 00000400:00000200:2.0:1788881022.591851:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788881022.591854:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306658768 00000100:00000001:0.0:1788881022.591855:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788881022.591856:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788881022.591860:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306653848 00000400:00000200:2.0:1788881022.591862:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:0.0:1788881022.591863:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881038 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881022.591864:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306658768 00000400:00000200:2.0:1788881022.591867:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788881022.591869:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306653848 00000400:00000200:2.0:1788881022.591872:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788881022.591874:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306658768 00000400:00000200:2.0:1788881022.591878:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [581:581:254:4294967295] 192.168.204.111@tcp LPNI seq info [581:581:6:4294967295] 00000100:00000001:1.0:1788881022.591880:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306653848 00000100:00000001:0.0:1788881022.591885:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881022.591887:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306658768 00000100:00000001:0.0:1788881022.591888:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788881022.591893:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788881022.591894:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788881022.591897:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881022.591902:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881022.591905:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788881022.591905:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881022.591910:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:2.0:1788881022.591910:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000100:00000001:1.0:1788881022.591913:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000200:2.0:1788881022.591915:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788881022.591917:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:2.0:1788881022.591920:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788881022.591921:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881022.591925:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881022.591925:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788881022.591931:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881022.591931:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881022.591936:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11340 usecs (12 jiffies) 00000800:00000200:0.0:1788881022.591938:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788881022.591939:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306653709 00000800:00000010:0.0:1788881022.591944:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000a371c7dc. 00000100:00000001:2.0:1788881022.591944:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306658768 00000100:00000001:2.0:1788881022.591948:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306653709 00000400:00000200:0.0:1788881022.591950:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788881022.591951:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306658768 00000100:00000001:2.0:1788881022.591954:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306653709 00000100:00000001:2.0:1788881022.591957:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306658768 00000400:00000200:0.0:1788881022.591959:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881022.591960:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306653709 00000100:00000001:2.0:1788881022.591963:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306658768 00000400:00000200:0.0:1788881022.591966:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005767f9c9 00000100:00000001:2.0:1788881022.591966:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788881022.591970:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005767f9c9. 00000100:00000001:2.0:1788881022.591971:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881022.591975:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788881022.591976:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881022.591979:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788881022.591980:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881022.591984:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881022.591987:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788881022.591991:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881022.591993:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788881022.591994:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881022.591998:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881022.591999:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881022.592001:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881022.592011:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881022.592013:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788881022.592046:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788881022.592048:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000400:00000200:0.0:1788881022.592051:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788881022.592054:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788881022.592056:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f779aa21 00000400:00000010:0.0:1788881022.592058:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f779aa21. 00000100:00000001:0.0:1788881022.592060:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788881022.592062:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881022.592068:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881022.592070:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881022.592086:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881022.592088:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788881022.593665:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881022.593671:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881022.593673:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881022.593674:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881022.593679:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881022.593690:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b11980 00000400:00000200:2.0:1788881022.593694:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x12e1 [1] + 224 00000800:00000001:2.0:1788881022.593699:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881022.593702:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881022.593704:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881022.593707:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881022.593711:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881022.593714:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881022.593723:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881022.593737:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881022.593743:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881022.593761:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881022.593764:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881022.593766:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881022.593769:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881022.593772:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788881022.593773:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:3.0:1788881022.593776:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788881022.593779:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788881022.593781:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000df34776e. 00000400:00000200:2.0:1788881022.593782:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11a80 00000400:00000200:3.0:1788881022.593785:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000010:3.0:1788881022.593788:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:3.0:1788881022.593790:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788881022.593793:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x12f1 [1] + 224 00000100:00000200:3.0:1788881022.593794:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881022.593801:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881022.593805:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881022.593807:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881022.593811:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881022.593814:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881022.593817:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881022.593835:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788881022.593853:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881022.593860:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881022.593862:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788881022.593862:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881022.593867:0:201286:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788881022.593869:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788881022.593869:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881022.593872:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788881022.593879:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881022.593881:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881022.593883:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:3.0:1788881022.593885:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881022.593886:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:2.0:1788881022.593887:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881022.593890:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788881022.593893:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00001000:3.0:1788881022.593893:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=100 v=5 (5 5 5 5) 00000400:00000200:2.0:1788881022.593897:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000001:1.0:1788881022.593898:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:3.0:1788881022.593899:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=100 v=5 (1 1 1 1) 00000400:00000010:1.0:1788881022.593901:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ffb4c8a2. 00000400:00000200:1.0:1788881022.593906:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000078d4a4ff 00000100:00000001:3.0:1788881022.593907:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000010:1.0:1788881022.593910:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000078d4a4ff. 00000400:00000200:2.0:1788881022.593911:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11a00 00000100:00000001:3.0:1788881022.593911:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881022.593914:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:3.0:1788881022.593915:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788881022.593918:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881022.593921:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881022.593921:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881022.593924:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x12e9 [1] + 224 00000100:00000040:3.0:1788881022.593928:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881022.593934:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788881022.593945:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881022.593956:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881022.593959:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881022.593961:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881022.593962:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788881022.593964:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:1.0:1788881022.593966:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881022.593967:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881022.593972:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881022.593975:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00100000:3.0:1788881022.593979:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000fd20c80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780704128:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881022.593982:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881022.593982:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881022.593984:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881022.593986:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1788881022.593988:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881022.593989:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788881022.593993:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00001000:1.0:1788881022.593994:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=100 v=5 (5 5 5 5) 00000100:00000040:3.0:1788881022.593996:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000fd20c80 x1875776780704128/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881022.593999:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:1.0:1788881022.594002:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=34 v=5 (1 1 0 0) 00000100:00000200:2.0:1788881022.594006:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881022.594010:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881022.594014:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881022.594017:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00010000:00000001:1.0:1788881022.594019:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000001:3.0:1788881022.594022:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00010000:00080000:1.0:1788881022.594027:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1788881022.594027:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007b91b305. 00000100:00000040:2.0:1788881022.594029:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881022.594032:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881022.594034:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881022.594036:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788881022.594041:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881022.594044:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000fc8f9b7c. 02000000:00000001:3.0:1788881022.594048:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881022.594051:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881022.594053:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000fd20c80. 00000100:00000001:3.0:1788881022.594057:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881022.594059:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881022.594062:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788881022.594062:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881022.594068:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:1.0:1788881022.594069:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881022.594070:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788881022.594070:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881022.594074:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881022.594075:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881022.594076:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788881022.594078:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881022.594080:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788881022.594084:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788881022.594086:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881022.594092:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881022.594096:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881022.594100:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881022.594101:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881022.594104:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000001:0.0:1788881022.594105:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788881022.594106:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788881022.594108:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788881022.594111:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000831ba52e. 00000100:00000040:1.0:1788881022.594112:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881022.594116:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000066e8f89d 00000400:00000010:0.0:1788881022.594119:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000066e8f89d. 00000100:00000001:0.0:1788881022.594123:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:1.0:1788881022.594128:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000b4a4221 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780704384:192.168.204.111@tcp:400: 00000100:00000200:0.0:1788881022.594129:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881022.594136:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881022.594140:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000200:0.0:1788881022.594143:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881022.594146:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000b4a4221 x1875776780704384/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881022.594159:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881022.594161:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788881022.594163:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788881022.594163:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788881022.594165:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000afd3911b. 00000100:00000001:0.0:1788881022.594167:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881022.594169:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:0.0:1788881022.594171:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:1.0:1788881022.594171:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788881022.594173:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881022.594175:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000001:1.0:1788881022.594175:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788881022.594178:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881022.594178:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000005f53e16f. 02000000:00000001:1.0:1788881022.594181:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881022.594182:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881022.594183:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881022.594185:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000b4a4221. 00000100:00001000:0.0:1788881022.594186:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=90 v=5 (5 5 5 5) 00000100:00000001:1.0:1788881022.594188:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881022.594189:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788881022.594191:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=90 v=5 (1 1 1 1) 00000100:00000001:1.0:1788881022.594192:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881022.594196:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881022.594197:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881022.594198:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881022.594199:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881022.594200:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788881022.594202:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:0.0:1788881022.594203:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881022.594204:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881022.594206:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788881022.594207:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881022.594209:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881022.594210:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881022.594213:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881022.594217:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881022.594224:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881022.594225:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881022.594227:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881022.594247:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780704256:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881022.594250:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881022.594251:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881022.594253:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d4e901df x1875776780704256/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881038 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881022.594256:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881022.594258:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881022.594259:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000005842920. 02000000:00000001:0.0:1788881022.594262:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788881022.594263:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881022.594263:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881022.594265:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881022.594266:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000003806e31f. 02000000:00000001:0.0:1788881022.594267:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881022.594268:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881022.594269:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d4e901df. 00000100:00000001:0.0:1788881022.594271:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881022.594272:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881022.594272:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881022.594275:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881022.594276:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881022.594277:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881022.594278:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881022.654347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881022.654353:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881022.654357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881022.654358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881022.654441:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881022.654448:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881022.654454:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881023.678372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881023.678383:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881023.678391:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881023.678392:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881023.678461:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881023.678467:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881023.678471:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881024.702317:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881024.702325:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881024.702331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881024.702332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881024.702382:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881024.702385:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881024.702401:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881025.726306:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881025.726312:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881025.726313:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881025.726315:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881025.726319:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881025.726321:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881025.726322:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881025.726323:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881025.726326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881025.726327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881025.726360:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881025.726364:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881025.726366:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881026.750314:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881026.750324:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881026.750332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881026.750333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881026.750370:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881026.750374:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881026.750376:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881027.710356:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306658768, at 4306658816 00000100:00000040:1.0:1788881027.710356:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306658768, at 4306658816 00000100:00000040:2.0:1788881027.710356:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306658768, at 4306658816 00000100:00000040:3.0:1788881027.710356:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306658768, at 4306658816 00000100:00000010:1.0:1788881027.710380:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a9d0cfcc. 00000100:00000010:2.0:1788881027.710385:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000015631aee. 00000100:00000010:3.0:1788881027.710388:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014c076ca. 00000100:00000010:0.0:1788881027.710392:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001f846689. 00000100:00000040:1.0:1788881027.710423:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881027.710430:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881027.710439:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000dc99d6ba. 00000020:00000040:1.0:1788881027.710445:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788881027.710452:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881027.710456:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881027.710462:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881027.710473:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 00000100:00000001:1.0:1788881027.710481:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881027.710494:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881027.710506:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881027.710509:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881027.710516:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000dc99d6ba] to pc [ptlrpcd_00_00+0] req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881027.710554:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881027.710557:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881027.710561:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881027.710565:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006a9c2e02. 00000020:00000040:1.0:1788881027.710568:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881027.710570:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881027.710571:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881027.710574:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788881027.710575:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788881027.710577:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881027.710579:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:1.0:1788881027.710583:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f0a0057b. 00000100:00000040:0.0:1788881027.710585:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881027.710587:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881027.710589:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881027.710593:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881027.710600:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788881027.710602:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881027.710605:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881027.710610:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788881027.710614:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881027.710614:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881027.710618:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881027.710619:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881027.710625:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000dc99d6ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780704512:192.168.204.111@tcp:400: 00000100:00000040:1.0:1788881027.710628:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006a9c2e02] to pc [ptlrpcd_00_01+1] req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881027.710629:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788881027.710638:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881027.710641:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881027.710643:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788881027.710647:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000006c62949. 00000100:00000001:1.0:1788881027.710648:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881027.710650:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881027.710654:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881027.710659:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000400:00000010:0.0:1788881027.710661:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000100:00000010:1.0:1788881027.710664:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001cf64abe. 00000020:00000040:1.0:1788881027.710668:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788881027.710668:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788881027.710671:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000001:2.0:1788881027.710672:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881027.710673:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788881027.710676:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788881027.710677:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881027.710680:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881027.710681:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780704512, portal 25 00000100:00000040:2.0:1788881027.710684:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881027.710685:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881027.710689:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938052352 : -126823771499264 : ffff8ca78a269500) 02000000:00000010:1.0:1788881027.710693:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000004e0f6846. 00000100:00000040:0.0:1788881027.710697:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881027.710699:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881027.710709:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788881027.710719:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881027.710721:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788881027.710723:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881027.710737:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788881027.710738:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788881027.710742:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788881027.710743:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000115221d8. 00000100:00000001:1.0:1788881027.710745:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788881027.710748:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881027.710749:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780704512, offset 0 00000100:00000001:1.0:1788881027.710749:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881027.710759:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001cf64abe] to pc [ptlrpcd_00_02+2] req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881027.710760:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:2.0:1788881027.710761:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006a9c2e02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780704640:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881027.710770:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788881027.710775:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788881027.710779:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:2.0:1788881027.710779:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881027.710787:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788881027.710790:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:2.0:1788881027.710791:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788881027.710796:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881027.710800:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:1.0:1788881027.710800:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000010:2.0:1788881027.710800:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d4ae78a4. 02000000:00000001:2.0:1788881027.710805:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881027.710808:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788881027.710809:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:2.0:1788881027.710813:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 00000400:00000200:0.0:1788881027.710817:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [582:582:256:4294967295] 192.168.204.111@tcp LPNI seq info [582:582:8:4294967295] 00000100:00000040:1.0:1788881027.710817:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (11996) 00000100:00000001:3.0:1788881027.710817:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:2.0:1788881027.710820:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c6ecbca9. 00000100:00000001:3.0:1788881027.710823:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:2.0:1788881027.710826:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780704640, portal 10 00000100:00000001:3.0:1788881027.710827:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788881027.710830:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881027.710832:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881027.710833:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938058624 : -126823771492992 : ffff8ca78a26ad80) 00000100:00000001:1.0:1788881027.710837:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881027.710837:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881027.710841:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:2.0:1788881027.710844:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788881027.710855:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788881027.710863:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003806e31f. 00000100:00000001:2.0:1788881027.710868:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788881027.710868:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881027.710871:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881027.710872:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788881027.710876:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000010:2.0:1788881027.710877:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000078d4a4ff. 02000000:00000001:3.0:1788881027.710878:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:2.0:1788881027.710882:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780704640, offset 0 02000000:00000001:3.0:1788881027.710883:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788881027.710887:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:3.0:1788881027.710889:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881027.710892:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788881027.710893:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003806e31f type 1, nob 320 niov 1 nkiov 1 00000100:00100000:3.0:1788881027.710897:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780704768:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788881027.710901:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788881027.710904:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788881027.710905:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:3.0:1788881027.710908:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788881027.710909:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881027.710909:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788881027.710912:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881027.710914:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788881027.710915:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306663887 02000000:00000001:3.0:1788881027.710917:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788881027.710919:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:0.0:1788881027.710920:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e6f9f09c. 02000000:00000010:3.0:1788881027.710923:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000826ae8f4. 00000400:00000200:2.0:1788881027.710924:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [583:583:255:4294967295] 192.168.204.111@tcp LPNI seq info [583:583:7:4294967295] 00000100:00000001:0.0:1788881027.710927:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 02000000:00000001:3.0:1788881027.710928:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881027.710934:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306663887 00000400:00000010:3.0:1788881027.710935:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000df34776e. 00000100:00000010:0.0:1788881027.710940:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000506ae1c0. 00000400:00000200:2.0:1788881027.710940:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:3.0:1788881027.710943:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000da9ee886. 00000100:00000001:0.0:1788881027.710945:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:2.0:1788881027.710949:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881027.710950:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306663887 00000100:00000200:3.0:1788881027.710951:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780704768, portal 10 00000800:00000010:2.0:1788881027.710955:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f47bc5fc. 00000100:00000001:3.0:1788881027.710956:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000010:0.0:1788881027.710960:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e245be24. 00000800:00000200:2.0:1788881027.710960:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788881027.710961:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938054144 : -126823771497472 : ffff8ca78a269c00) 00000100:00000001:0.0:1788881027.710966:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:2.0:1788881027.710966:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788881027.710971:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f47bc5fc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788881027.710972:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306663887 00000100:00000010:0.0:1788881027.710975:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003b9d7f95. 00000100:00000001:0.0:1788881027.710977:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788881027.710983:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881027.710983:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881027.710983:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881027.710988:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881027.710989:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 149353 usecs (150 jiffies) 00000100:00000001:2.0:1788881027.710994:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306658966 00000100:00000001:0.0:1788881027.710995:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881027.711000:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881027.711000:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306663887 00000100:00000001:2.0:1788881027.711005:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306658966 00000100:00000001:3.0:1788881027.711007:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788881027.711008:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306663887 00000100:00000001:0.0:1788881027.711011:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881027.711012:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306658966 00000100:00000040:3.0:1788881027.711012:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881027.711015:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788881027.711016:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306663887 00000100:00000001:0.0:1788881027.711018:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881027.711020:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881027.711020:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306658966 00000400:00000010:3.0:1788881027.711020:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000cf82a3fa. 00000100:00000001:2.0:1788881027.711022:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306663887 00000100:00000001:2.0:1788881027.711024:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881027.711025:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881027.711026:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881027.711030:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881027.711030:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788881027.711032:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780704768, offset 0 00000100:00000001:0.0:1788881027.711034:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881027.711035:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881027.711038:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788881027.711040:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:1.0:1788881027.711041:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:3.0:1788881027.711041:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881027.711042:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881027.711045:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788881027.711048:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003806e31f. 00000100:00000001:2.0:1788881027.711048:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881027.711052:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788881027.711056:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881027.711058:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788881027.711064:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788881027.711065:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881027.711070:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788881027.711073:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788881027.711076:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [584:584:254:4294967295] 192.168.204.111@tcp LPNI seq info [584:584:6:4294967295] 00000400:00000200:3.0:1788881027.711084:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788881027.711088:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788881027.711089:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000115221d8 00000400:00000010:1.0:1788881027.711092:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000115221d8. 00000800:00000010:3.0:1788881027.711092:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:1.0:1788881027.711099:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:3.0:1788881027.711100:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:1.0:1788881027.711106:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788881027.711108:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788881027.711113:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788881027.711120:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881027.711124:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881043 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881027.711136:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881027.711138:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788881027.711154:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881027.711156:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f47bc5fc. 00000400:00000200:1.0:1788881027.711159:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881027.711163:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881027.711165:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000078d4a4ff 00000400:00000010:1.0:1788881027.711167:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000078d4a4ff. 00000100:00000001:1.0:1788881027.711169:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881027.711171:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881027.711178:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881027.711180:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881027.711186:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881027.711187:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788881027.711265:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881027.711268:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11291 usecs (12 jiffies) 00000100:00000001:3.0:1788881027.711270:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306658829 00000800:00000200:0.0:1788881027.711275:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788881027.711279:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:0.0:1788881027.711284:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788881027.711288:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788881027.711292:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000cf82a3fa 00000400:00000010:0.0:1788881027.711294:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000cf82a3fa. 00000100:00000001:0.0:1788881027.711310:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788881027.711314:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881027.711322:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881027.711324:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881027.711325:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306663887 00000100:00000001:3.0:1788881027.711331:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306658829 00000100:00000001:0.0:1788881027.711334:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881027.711336:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306663887 00000100:00000001:0.0:1788881027.711337:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788881027.711340:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306658829 00000100:00000001:3.0:1788881027.711343:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306663887 00000100:00000001:3.0:1788881027.711344:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306658829 00000100:00000001:3.0:1788881027.711346:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306663887 00000100:00000001:3.0:1788881027.711348:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881027.711350:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881027.711353:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881027.711355:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881027.711364:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881027.711365:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881027.711368:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881027.711369:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881027.711402:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881027.711404:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881027.712466:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881027.712479:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881027.712482:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881027.712485:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881027.712493:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881027.712505:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b11b00 00000400:00000200:2.0:1788881027.712514:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x12f9 [1] + 224 00000800:00000001:2.0:1788881027.712521:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881027.712527:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881027.712530:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881027.712533:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881027.712544:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881027.712548:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881027.712557:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881027.712571:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881027.712576:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881027.712579:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881027.712601:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881027.712603:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788881027.712608:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881027.712610:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788881027.712613:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000400:00000200:0.0:1788881027.712617:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000400:00000010:0.0:1788881027.712619:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:0.0:1788881027.712622:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788881027.712626:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881027.712635:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881027.712643:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881027.712644:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881027.712646:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881027.712650:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881027.712653:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:1.0:1788881027.712653:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881027.712655:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:1.0:1788881027.712656:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881027.712660:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881027.712660:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881027.712667:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881027.712668:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881027.712675:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11b80 00000400:00000200:1.0:1788881027.712680:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1301 [1] + 224 00000100:00001000:0.0:1788881027.712682:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=105 v=5 (5 5 5 5) 00000800:00000001:1.0:1788881027.712683:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881027.712687:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788881027.712689:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=105 v=5 (1 1 1 1) 00000400:00000200:1.0:1788881027.712691:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881027.712693:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881027.712695:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881027.712696:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881027.712701:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:0.0:1788881027.712702:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788881027.712705:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881027.712707:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881027.712710:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881027.712718:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881027.712723:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881027.712745:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881027.712750:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881027.712752:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881027.712756:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881027.712758:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:0.0:1788881027.712764:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000dc99d6ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780704512:192.168.204.111@tcp:400: 00000800:00000001:1.0:1788881027.712767:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881027.712772:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881027.712772:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881027.712777:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881027.712782:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000dc99d6ba x1875776780704512/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881027.712783:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881027.712790:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881027.712794:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788881027.712794:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:0.0:1788881027.712798:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000006c62949. 00000400:00000001:2.0:1788881027.712800:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:0.0:1788881027.712802:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000001:2.0:1788881027.712804:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000020:00000001:0.0:1788881027.712806:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000010:2.0:1788881027.712807:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000020:00000040:0.0:1788881027.712808:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788881027.712813:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000400:00000200:2.0:1788881027.712813:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c6ecbca9 02000000:00000010:0.0:1788881027.712815:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 00000400:00000010:2.0:1788881027.712816:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c6ecbca9. 00000100:00000001:2.0:1788881027.712821:0:201286:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:0.0:1788881027.712822:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881027.712825:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000200:2.0:1788881027.712825:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788881027.712828:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000dc99d6ba. 00000100:00000001:0.0:1788881027.712833:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881027.712836:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788881027.712836:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881027.712840:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881027.712843:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881027.712845:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881027.712849:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788881027.712850:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881027.712851:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881027.712852:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881027.712854:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788881027.712854:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788881027.712855:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881027.712860:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788881027.712860:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788881027.712862:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881027.712863:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788881027.712864:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=95 v=5 (5 5 5 5) 00000800:00000001:3.0:1788881027.712868:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:2.0:1788881027.712869:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=95 v=5 (1 1 1 1) 00000100:00000001:2.0:1788881027.712873:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788881027.712874:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788881027.712875:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788881027.712876:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788881027.712879:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881027.712884:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000200:3.0:1788881027.712885:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000040:2.0:1788881027.712893:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788881027.712896:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788881027.712898:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881027.712902:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881027.712909:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881027.712911:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881027.712912:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788881027.712915:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788881027.712921:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006a9c2e02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780704640:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881027.712926:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881027.712928:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881027.712930:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006a9c2e02 x1875776780704640/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881027.712938:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000200:3.0:1788881027.712938:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11c00 02000000:00000001:2.0:1788881027.712941:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788881027.712944:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d4ae78a4. 00000400:00000200:3.0:1788881027.712944:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1309 [1] + 224 02000000:00000001:2.0:1788881027.712950:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000800:00000001:3.0:1788881027.712951:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000020:00000001:2.0:1788881027.712953:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788881027.712955:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:2.0:1788881027.712958:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000800:00000001:3.0:1788881027.712958:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:2.0:1788881027.712960:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f0a0057b. 00000400:00000200:3.0:1788881027.712962:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:2.0:1788881027.712963:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788881027.712966:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788881027.712967:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006a9c2e02. 00000400:00000200:3.0:1788881027.712972:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881027.712977:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881027.712977:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788881027.712980:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881027.712984:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788881027.712985:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881027.712989:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881027.712991:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881027.712994:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881027.712996:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881027.713002:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881027.713018:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788881027.713023:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881027.713026:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881027.713045:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881027.713048:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788881027.713050:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881027.713051:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788881027.713052:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000df34776e. 00000400:00000200:0.0:1788881027.713055:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000da9ee886 00000400:00000010:0.0:1788881027.713056:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000da9ee886. 00000100:00000001:0.0:1788881027.713058:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788881027.713061:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881027.713066:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881027.713071:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881027.713071:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881027.713072:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881027.713073:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881027.713074:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881027.713075:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881027.713076:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881027.713079:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=105 v=5 (5 5 5 5) 00000100:00001000:0.0:1788881027.713082:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=39 v=5 (1 1 0 0) 00000100:00000001:0.0:1788881027.713084:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881027.713085:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881027.713086:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788881027.713088:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788881027.713092:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881027.713093:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788881027.713094:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881027.713095:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881027.713095:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881027.713102:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881027.713106:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881027.713107:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881027.713108:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881027.713110:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881027.713114:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780704768:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881027.713117:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881027.713118:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881027.713119:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000001cf64abe x1875776780704768/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881043 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881027.713122:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881027.713123:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881027.713124:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000826ae8f4. 02000000:00000001:0.0:1788881027.713126:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788881027.713127:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881027.713128:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881027.713129:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881027.713130:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000004e0f6846. 02000000:00000001:0.0:1788881027.713131:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881027.713132:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881027.713133:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000001cf64abe. 00000100:00000001:0.0:1788881027.713134:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881027.713135:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881027.713136:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881027.713146:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881027.713146:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881027.713148:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881027.713149:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881027.774315:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881027.774324:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881027.774330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881027.774331:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881027.774376:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881027.774380:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881027.774382:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881028.798343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881028.798351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881028.798356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881028.798360:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881028.798362:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881028.798365:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881028.798368:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881028.798370:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881028.798371:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881028.798373:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881028.798417:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881028.798421:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881028.798423:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881029.822347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881029.822357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881029.822364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881029.822365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881029.822429:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881029.822435:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881029.822439:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881030.846350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881030.846358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881030.846364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881030.846365:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881030.846425:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881030.846430:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881030.846434:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881031.870315:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881031.870324:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881031.870329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881031.870330:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881031.870382:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881031.870387:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881031.870389:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881032.830320:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306663887, at 4306663936 00000100:00000040:2.0:1788881032.830320:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788881032.830322:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306663887, at 4306663936 00000100:00000001:2.0:1788881032.830331:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788881032.830335:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e6f9f09c. 00000100:00000010:1.0:1788881032.830337:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000506ae1c0. 00000100:00000010:2.0:1788881032.830338:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000005a8bb1de. 00000020:00000040:2.0:1788881032.830345:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788881032.830349:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881032.830352:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881032.830358:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881032.830365:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f9812869. 00000100:00000040:3.0:1788881032.830365:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306663887, at 4306663936 00000100:00000010:3.0:1788881032.830372:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003b9d7f95. 00000100:00000001:2.0:1788881032.830375:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881032.830386:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788881032.830394:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788881032.830396:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881032.830402:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a8bb1de] to pc [ptlrpcd_00_03+3] req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881032.830419:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881032.830421:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881032.830424:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881032.830428:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000cc13c26b. 00000020:00000040:2.0:1788881032.830430:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788881032.830433:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881032.830434:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788881032.830437:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788881032.830437:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881032.830441:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:2.0:1788881032.830442:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000001:1.0:1788881032.830444:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788881032.830446:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881032.830448:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881032.830450:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881032.830451:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881032.830454:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881032.830468:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788881032.830470:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881032.830475:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:1.0:1788881032.830476:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788881032.830477:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881032.830479:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788881032.830484:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000cc13c26b] to pc [ptlrpcd_00_00+0] req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881032.830485:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788881032.830492:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780704896:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881032.830497:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788881032.830503:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881032.830506:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881032.830509:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788881032.830515:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000009e54fce. 00000100:00000001:2.0:1788881032.830518:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881032.830522:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788881032.830523:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881032.830529:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000400:00000010:1.0:1788881032.830532:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000050d7b176. 00000100:00000001:0.0:1788881032.830533:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:2.0:1788881032.830533:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000910cb22c. 00000100:00000001:0.0:1788881032.830537:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881032.830538:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:2.0:1788881032.830538:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:2.0:1788881032.830540:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:1.0:1788881032.830542:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b8cfbba8. 02000000:00000001:2.0:1788881032.830542:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:0.0:1788881032.830544:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881032.830544:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881032.830549:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780704896, portal 25 02000000:00000010:2.0:1788881032.830549:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 00000100:00000001:1.0:1788881032.830554:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881032.830554:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881032.830555:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881032.830555:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881032.830558:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938427392 : -126823771124224 : ffff8ca78a2c4e00) 00000100:00000001:0.0:1788881032.830559:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881032.830559:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881032.830561:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788881032.830564:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788881032.830566:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881032.830569:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881032.830571:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:0.0:1788881032.830572:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780705024:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881032.830573:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881032.830575:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881032.830576:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:2.0:1788881032.830579:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000910cb22c] to pc [ptlrpcd_00_01+1] req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881032.830580:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881032.830582:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881032.830586:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788881032.830590:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000783b292f. 02000000:00000001:0.0:1788881032.830594:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881032.830594:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788881032.830598:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788881032.830600:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000400:00000010:1.0:1788881032.830601:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000022eb8f73. 00000100:00000001:2.0:1788881032.830604:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881032.830606:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780704896, offset 0 00000100:00000040:2.0:1788881032.830607:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:0.0:1788881032.830608:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000040:2.0:1788881032.830611:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000200:0.0:1788881032.830612:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780705024, portal 10 00000400:00000200:1.0:1788881032.830615:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881032.830616:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:2.0:1788881032.830617:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12001) 00000100:00000001:0.0:1788881032.830619:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938433664 : -126823771117952 : ffff8ca78a2c6680) 00000100:00000001:3.0:1788881032.830621:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881032.830624:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881032.830626:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788881032.830628:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881032.830643:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:3.0:1788881032.830644:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881032.830650:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306663887, at 4306663936 00000400:00000200:1.0:1788881032.830651:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788881032.830658:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000010:2.0:1788881032.830658:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e245be24. 00000400:00000200:1.0:1788881032.830665:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788881032.830671:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788881032.830679:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [585:585:256:4294967295] 192.168.204.111@tcp LPNI seq info [585:585:8:4294967295] 00000100:00000001:0.0:1788881032.830681:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788881032.830682:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881032.830684:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881032.830688:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788881032.830688:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788881032.830691:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788881032.830694:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000004409ca8b. 00000400:00000200:1.0:1788881032.830696:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:3.0:1788881032.830696:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881032.830698:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780705024, offset 0 00000100:00100000:3.0:1788881032.830703:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780705152:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788881032.830709:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881032.830710:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:1.0:1788881032.830711:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788881032.830714:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000800:00000010:1.0:1788881032.830717:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000131c1a55. 00000400:00000200:0.0:1788881032.830718:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788881032.830718:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881032.830723:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788881032.830724:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:3.0:1788881032.830727:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a5da162d. 00000800:00000200:1.0:1788881032.830728:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788881032.830731:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788881032.830731:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881032.830736:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:3.0:1788881032.830736:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004f63ce78. 00000800:00000200:1.0:1788881032.830740:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:3.0:1788881032.830742:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000400:00000200:0.0:1788881032.830743:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:1.0:1788881032.830745:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000131c1a55 type 1, nob 320 niov 1 nkiov 1 00000100:00000200:3.0:1788881032.830747:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780705152, portal 10 00000400:00000200:0.0:1788881032.830750:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [586:586:255:4294967295] 192.168.204.111@tcp LPNI seq info [586:586:7:4294967295] 00000100:00000001:3.0:1788881032.830752:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881032.830755:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938437248 : -126823771114368 : ffff8ca78a2c7480) 00000100:00000001:1.0:1788881032.830759:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881032.830761:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881032.830765:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306669007 00000400:00000200:0.0:1788881032.830767:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000010:1.0:1788881032.830772:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000cf4d1652. 00000100:00000001:1.0:1788881032.830778:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788881032.830778:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788881032.830782:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881032.830784:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306669007 00000100:00000040:3.0:1788881032.830784:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788881032.830789:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b3259090. 00000100:00000010:1.0:1788881032.830790:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000800:00000010:0.0:1788881032.830791:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003806e31f. 00000100:00000200:3.0:1788881032.830793:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780705152, offset 0 00000100:00000001:1.0:1788881032.830794:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788881032.830798:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306669007 00000800:00000200:0.0:1788881032.830800:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788881032.830802:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:1.0:1788881032.830804:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b245c5fd. 00000100:00000001:1.0:1788881032.830808:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:0.0:1788881032.830812:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788881032.830812:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788881032.830813:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306669007 00000800:00000200:0.0:1788881032.830819:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003806e31f type 1, nob 320 niov 1 nkiov 1 00000100:00000010:1.0:1788881032.830819:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045cabc93. 00000400:00000200:3.0:1788881032.830820:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788881032.830821:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788881032.830825:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881032.830827:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881032.830828:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881032.830830:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:0.0:1788881032.830833:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 148516 usecs (149 jiffies) 00000100:00000001:1.0:1788881032.830833:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881032.830834:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788881032.830838:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788881032.830838:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788881032.830839:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306664085 00000400:00000200:3.0:1788881032.830839:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788881032.830845:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306669007 00000800:00000010:2.0:1788881032.830845:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000131c1a55. 00000400:00000200:3.0:1788881032.830847:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [587:587:254:4294967295] 192.168.204.111@tcp LPNI seq info [587:587:6:4294967295] 00000100:00000001:1.0:1788881032.830848:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881032.830849:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306664085 00000400:00000200:2.0:1788881032.830851:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788881032.830852:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788881032.830854:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306669007 00000100:00000001:1.0:1788881032.830856:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881032.830858:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306664085 00000100:00000001:1.0:1788881032.830859:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788881032.830861:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881032.830863:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306669007 00000100:00000001:1.0:1788881032.830863:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881032.830866:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788881032.830867:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881032.830869:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306664085 00000100:00000001:0.0:1788881032.830875:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306669007 00000400:00000200:2.0:1788881032.830878:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000022eb8f73 00000800:00000200:3.0:1788881032.830878:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788881032.830883:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000022eb8f73. 00000800:00000010:3.0:1788881032.830884:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:0.0:1788881032.830885:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881032.830889:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881032.830892:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881032.830892:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788881032.830894:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788881032.830896:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881032.830897:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881032.830899:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788881032.830901:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881032.830903:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881032.830904:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881032.830905:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881032.830906:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881032.830907:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881048 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881032.830913:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881032.830915:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788881032.830920:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788881032.830926:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788881032.830928:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788881032.830930:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881032.830932:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11242 usecs (12 jiffies) 00000100:00000001:3.0:1788881032.830935:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306663948 00000100:00000001:3.0:1788881032.830936:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306669007 00000100:00000001:3.0:1788881032.830938:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306663948 00000100:00000001:3.0:1788881032.830939:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306669007 00000100:00000001:3.0:1788881032.830940:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306663948 00000100:00000001:3.0:1788881032.830941:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306669007 00000100:00000001:3.0:1788881032.830942:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306663948 00000100:00000001:3.0:1788881032.830942:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306669007 00000100:00000001:3.0:1788881032.830943:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881032.830944:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881032.830946:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881032.830947:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881032.830950:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881032.830951:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881032.830952:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881032.830953:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881032.830954:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881032.830958:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788881032.830959:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788881032.830963:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003806e31f. 00000400:00000200:2.0:1788881032.830966:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881032.830970:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881032.830972:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000004409ca8b 00000400:00000010:2.0:1788881032.830974:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000004409ca8b. 00000100:00000001:2.0:1788881032.830975:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881032.830977:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881032.830982:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881032.830983:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881032.830987:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881032.830988:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788881032.830996:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788881032.830997:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:2.0:1788881032.830999:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881032.831000:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881032.831002:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b3259090 00000400:00000010:2.0:1788881032.831003:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b3259090. 00000100:00000001:2.0:1788881032.831004:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881032.831005:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881032.831009:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881032.831010:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881032.831017:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881032.831019:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788881032.832330:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881032.832340:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881032.832342:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881032.832343:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881032.832349:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881032.832358:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b11c80 00000400:00000200:1.0:1788881032.832364:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1311 [1] + 224 00000800:00000001:1.0:1788881032.832370:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881032.832375:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881032.832378:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881032.832381:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881032.832384:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881032.832388:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881032.832397:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881032.832413:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881032.832419:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881032.832422:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881032.832423:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881032.832425:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881032.832429:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881032.832434:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11d00 00000400:00000200:1.0:1788881032.832438:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1319 [1] + 224 00000800:00000001:1.0:1788881032.832442:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881032.832446:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881032.832447:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881032.832450:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881032.832452:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881032.832455:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881032.832463:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881032.832472:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881032.832474:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788881032.832478:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788881032.832479:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000001:2.0:1788881032.832480:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788881032.832483:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000050d7b176. 00000400:00000200:2.0:1788881032.832487:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b8cfbba8 00000400:00000010:2.0:1788881032.832488:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b8cfbba8. 00000100:00000001:2.0:1788881032.832490:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881032.832494:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788881032.832501:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:2.0:1788881032.832501:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881032.832507:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881032.832508:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881032.832510:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788881032.832512:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:1.0:1788881032.832514:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881032.832514:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788881032.832517:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881032.832518:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:2.0:1788881032.832520:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881032.832523:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788881032.832525:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=110 v=5 (5 5 5 5) 00000100:00000001:0.0:1788881032.832527:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:2.0:1788881032.832529:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=110 v=5 (1 1 1 1) 00000100:00000001:0.0:1788881032.832531:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788881032.832533:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788881032.832534:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000001:0.0:1788881032.832536:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:2.0:1788881032.832537:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788881032.832539:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00010000:00000001:2.0:1788881032.832540:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000010:0.0:1788881032.832542:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000400:00000200:1.0:1788881032.832542:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11d80 00010000:00000001:2.0:1788881032.832543:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881032.832548:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000100:00000001:2.0:1788881032.832548:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881032.832552:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000040:2.0:1788881032.832555:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881032.832556:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788881032.832562:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1321 [1] + 224 00000100:00000200:0.0:1788881032.832563:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881032.832570:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:1.0:1788881032.832573:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788881032.832576:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881032.832578:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881032.832582:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881032.832586:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:2.0:1788881032.832587:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881032.832588:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881032.832594:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881032.832601:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881032.832607:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881032.832609:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881032.832611:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:2.0:1788881032.832619:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780704896:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881032.832622:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881032.832624:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881032.832627:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881032.832640:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881032.832641:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881032.832643:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881032.832645:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881032.832651:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=100 v=5 (5 5 5 5) 00000100:00000001:2.0:1788881032.832651:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00001000:0.0:1788881032.832655:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=100 v=5 (1 1 1 1) 00000100:00000001:0.0:1788881032.832657:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881032.832658:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:2.0:1788881032.832658:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000005a8bb1de x1875776780704896/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881032.832659:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881032.832664:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:1.0:1788881032.832666:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:0.0:1788881032.832667:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881032.832672:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000800:00000001:1.0:1788881032.832672:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881032.832672:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788881032.832676:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788881032.832678:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000009e54fce. 00000100:00000040:0.0:1788881032.832681:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:2.0:1788881032.832683:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788881032.832684:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788881032.832685:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881032.832686:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881032.832686:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788881032.832689:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788881032.832690:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000001:2.0:1788881032.832692:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000040:0.0:1788881032.832693:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788881032.832695:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f9812869. 00000400:00000001:3.0:1788881032.832696:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788881032.832699:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:2.0:1788881032.832700:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788881032.832703:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788881032.832704:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000400:00000010:3.0:1788881032.832704:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004f63ce78. 00000100:00000001:0.0:1788881032.832706:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000010:2.0:1788881032.832707:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000005a8bb1de. 00000100:00000001:0.0:1788881032.832708:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881032.832711:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000100:00000001:2.0:1788881032.832712:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:0.0:1788881032.832713:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881032.832715:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788881032.832716:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000001:2.0:1788881032.832720:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881032.832722:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:0.0:1788881032.832727:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000cc13c26b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780705024:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881032.832727:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881032.832730:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788881032.832730:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881032.832736:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881032.832736:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881032.832738:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881032.832740:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881032.832745:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000cc13c26b x1875776780705024/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788881032.832747:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881032.832760:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788881032.832763:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788881032.832764:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:3.0:1788881032.832767:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788881032.832768:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000783b292f. 00000100:00000001:3.0:1788881032.832771:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881032.832772:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:3.0:1788881032.832773:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:0.0:1788881032.832775:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788881032.832775:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000040:0.0:1788881032.832776:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881032.832778:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881032.832779:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:3.0:1788881032.832780:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788881032.832781:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 02000000:00000001:0.0:1788881032.832784:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00001000:3.0:1788881032.832785:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=110 v=5 (5 5 5 5) 02000000:00000001:0.0:1788881032.832786:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881032.832787:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000cc13c26b. 00000100:00000001:0.0:1788881032.832791:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00001000:3.0:1788881032.832791:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=44 v=5 (1 1 0 0) 00000100:00000001:0.0:1788881032.832793:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881032.832796:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881032.832797:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881032.832799:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881032.832800:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00010000:00000001:3.0:1788881032.832800:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788881032.832802:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881032.832805:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00080000:3.0:1788881032.832805:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881032.832807:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881032.832811:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881032.832814:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788881032.832815:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788881032.832816:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788881032.832816:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881032.832818:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881032.832831:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881032.832832:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881032.832833:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881032.832835:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788881032.832840:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000910cb22c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780705152:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881032.832842:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881032.832843:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881032.832844:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000910cb22c x1875776780705152/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881048 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881032.832848:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788881032.832849:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788881032.832850:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a5da162d. 02000000:00000001:3.0:1788881032.832852:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881032.832852:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881032.832853:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788881032.832855:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881032.832856:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 02000000:00000001:3.0:1788881032.832857:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881032.832858:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881032.832859:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000910cb22c. 00000100:00000001:3.0:1788881032.832860:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881032.832861:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881032.832862:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881032.832865:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881032.832865:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881032.832867:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881032.832868:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881032.894293:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881032.894299:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881032.894305:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881032.894306:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881032.894345:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881032.894348:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881032.894351:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881033.918399:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881033.918407:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881033.918412:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881033.918412:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881033.918470:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881033.918474:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881033.918477:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881034.942369:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881034.942376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881034.942382:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881034.942383:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881034.942450:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881034.942469:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881034.942473:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881035.966289:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881035.966296:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881035.966298:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881035.966302:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881035.966306:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881035.966308:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881035.966311:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881035.966313:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881035.966315:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881035.966316:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881035.966395:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881035.966398:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881035.966401:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881036.990323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881036.990330:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881036.990337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881036.990338:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881036.990386:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881036.990389:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881036.990391:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788881037.950342:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306669007, at 4306669056 00000100:00000040:2.0:1788881037.950349:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306669007, at 4306669056 00000100:00000040:3.0:1788881037.950356:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306669007, at 4306669056 00000100:00000010:2.0:1788881037.950361:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b245c5fd. 00000100:00000010:1.0:1788881037.950362:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000010:3.0:1788881037.950363:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045cabc93. 00000100:00000040:0.0:1788881037.950374:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306669007, at 4306669056 00000100:00000010:0.0:1788881037.950389:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000cf4d1652. 00000100:00000040:3.0:1788881037.950389:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881037.950396:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881037.950406:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788881037.950412:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788881037.950417:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881037.950420:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881037.950426:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881037.950435:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:3.0:1788881037.950443:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881037.950453:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788881037.950463:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881037.950465:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881037.950469:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_02+2] req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881037.950488:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881037.950489:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881037.950492:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881037.950495:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788881037.950497:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881037.950499:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881037.950502:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881037.950504:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881037.950507:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:3.0:1788881037.950509:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881037.950526:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788881037.950530:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788881037.950543:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881037.950543:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881037.950551:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881037.950552:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881037.950553:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881037.950559:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788881037.950559:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_03+3] req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788881037.950572:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881037.950572:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881037.950579:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788881037.950579:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881037.950585:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881037.950591:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788881037.950592:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881037.950598:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000100:00000040:2.0:1788881037.950602:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:3.0:1788881037.950604:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881037.950613:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788881037.950614:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:3.0:1788881037.950616:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788881037.950619:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881037.950620:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881037.950621:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881037.950628:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881037.950630:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000010:3.0:1788881037.950630:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 02000000:00000001:2.0:1788881037.950633:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788881037.950639:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788881037.950639:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881037.950640:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881037.950650:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788881037.950652:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881037.950653:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881037.950665:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881037.950667:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881037.950672:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780705280:192.168.204.111@tcp:400: 00000100:00100000:2.0:1788881037.950672:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780705408:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788881037.950676:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_00+0] req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881037.950693:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788881037.950699:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788881037.950706:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788881037.950707:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881037.950713:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788881037.950713:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881037.950716:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000010:2.0:1788881037.950717:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000bef544e8. 00000100:00000040:3.0:1788881037.950718:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788881037.950720:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881037.950724:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881037.950726:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788881037.950732:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881037.950735:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788881037.950735:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788881037.950737:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:0.0:1788881037.950741:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000003222657. 00000400:00000010:2.0:1788881037.950742:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 00000100:00000040:3.0:1788881037.950742:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12006) 00000100:00000040:1.0:1788881037.950743:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881037.950754:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881037.950758:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c6ecbca9. 00000100:00000200:2.0:1788881037.950767:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780705408, portal 10 00000400:00000010:0.0:1788881037.950768:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000100:00000001:3.0:1788881037.950771:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881037.950773:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881037.950776:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881037.950777:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881037.950777:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000400:00000010:0.0:1788881037.950780:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000001:1.0:1788881037.950781:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881037.950788:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881037.950790:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:0.0:1788881037.950792:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780705280, portal 25 02000000:00000001:1.0:1788881037.950795:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881037.950800:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1788881037.950805:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881037.950806:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000001:2.0:1788881037.950811:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788881037.950819:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788881037.950822:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780705536:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788881037.950823:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788881037.950825:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f779aa21. 00000100:00000200:2.0:1788881037.950836:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780705408, offset 0 00000100:00000001:1.0:1788881037.950841:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788881037.950847:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881037.950852:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881037.950858:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788881037.950861:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788881037.950871:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:1.0:1788881037.950873:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a48931c2. 02000000:00000001:1.0:1788881037.950876:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881037.950879:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001738de44. 00000400:00000200:2.0:1788881037.950881:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:1.0:1788881037.950885:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000040:0.0:1788881037.950892:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000200:1.0:1788881037.950894:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780705536, portal 10 00000400:00000200:2.0:1788881037.950896:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:0.0:1788881037.950898:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000002aa6ba25. 00000100:00000001:1.0:1788881037.950902:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788881037.950904:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:0.0:1788881037.950907:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780705280, offset 0 00000100:00000001:1.0:1788881037.950908:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000400:00000200:2.0:1788881037.950912:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788881037.950921:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788881037.950925:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788881037.950927:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788881037.950931:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [588:588:256:4294967295] 192.168.204.111@tcp LPNI seq info [588:588:8:4294967295] 00000100:00000001:1.0:1788881037.950942:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788881037.950948:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788881037.950950:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000010:1.0:1788881037.950953:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e249bdf9. 00000800:00000200:2.0:1788881037.950957:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:1.0:1788881037.950958:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780705536, offset 0 00000800:00000010:2.0:1788881037.950962:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f9812869. 00000400:00000200:0.0:1788881037.950963:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788881037.950969:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881037.950971:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788881037.950984:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788881037.950987:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788881037.950993:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788881037.950998:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788881037.951003:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788881037.951011:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [589:589:255:4294967295] 192.168.204.111@tcp LPNI seq info [589:589:7:4294967295] 00000800:00000200:2.0:1788881037.951012:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f9812869 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788881037.951030:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788881037.951032:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788881037.951036:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:2.0:1788881037.951037:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 147687 usecs (148 jiffies) 00000800:00000010:0.0:1788881037.951041:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000005f53e16f. 00000400:00000200:1.0:1788881037.951041:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788881037.951042:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306669204 00000100:00000010:2.0:1788881037.951052:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000007e595eb4. 00000400:00000200:1.0:1788881037.951055:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:0.0:1788881037.951056:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788881037.951067:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000400:00000200:1.0:1788881037.951069:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788881037.951076:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788881037.951080:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306669204 00000800:00000200:0.0:1788881037.951082:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000005f53e16f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788881037.951083:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788881037.951091:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788881037.951093:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005c6a0f3f. 00000100:00000001:0.0:1788881037.951096:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306674127 00000400:00000200:1.0:1788881037.951097:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788881037.951100:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788881037.951111:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306669204 00000400:00000200:1.0:1788881037.951111:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [590:590:254:4294967295] 192.168.204.111@tcp LPNI seq info [590:590:6:4294967295] 00000100:00000001:2.0:1788881037.951113:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306669204 00000100:00000001:0.0:1788881037.951120:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306674127 00000100:00000010:2.0:1788881037.951123:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000076127301. 00000100:00000001:0.0:1788881037.951125:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306669204 00000100:00000001:2.0:1788881037.951130:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788881037.951134:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306674127 00000100:00000001:2.0:1788881037.951138:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306669204 00000100:00000001:0.0:1788881037.951141:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306669204 00000800:00000200:3.0:1788881037.951142:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:1.0:1788881037.951144:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788881037.951148:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306674127 00000100:00000010:2.0:1788881037.951148:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045db2575. 00000800:00000010:3.0:1788881037.951153:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f9812869. 00000800:00000200:1.0:1788881037.951156:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881037.951157:0:201288:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788881037.951159:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000010:1.0:1788881037.951160:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000131c1a55. 00000100:00000001:2.0:1788881037.951163:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881037.951167:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306669204 00000100:00000001:2.0:1788881037.951170:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881037.951172:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788881037.951172:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881037.951173:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881037.951178:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881037.951182:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881037.951186:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f779aa21 00000100:00000001:2.0:1788881037.951187:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881037.951189:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881037.951190:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:3.0:1788881037.951191:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f779aa21. 00000100:00000001:2.0:1788881037.951198:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881037.951199:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:1.0:1788881037.951201:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000131c1a55 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788881037.951202:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881037.951202:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788881037.951215:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881037.951216:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881037.951217:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881037.951218:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881037.951223:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:1.0:1788881037.951224:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11193 usecs (12 jiffies) 00000100:00000001:2.0:1788881037.951224:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881037.951255:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881037.951260:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306669069 00000100:00000001:0.0:1788881037.951261:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881037.951265:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881037.951268:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881037.951269:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306674127 00000100:00000001:0.0:1788881037.951274:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881037.951278:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306669069 00000100:00000001:1.0:1788881037.951282:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306674127 00000100:00000001:1.0:1788881037.951284:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306669069 00000100:00000001:1.0:1788881037.951286:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306674127 00000100:00000200:3.0:1788881037.951287:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881037.951288:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306669069 00000100:00000001:1.0:1788881037.951291:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306674127 00000100:00000001:1.0:1788881037.951294:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881037.951296:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881037.951298:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881037.951299:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788881037.951300:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881037.951302:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881037.951304:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881037.951305:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881037.951308:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881037.951308:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881037.951309:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881037.951310:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881037.951310:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788881037.951312:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788881037.951326:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788881037.951328:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000005f53e16f. 00000400:00000200:3.0:1788881037.951331:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881037.951334:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881037.951337:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000002aa6ba25 00000400:00000010:3.0:1788881037.951339:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000002aa6ba25. 00000100:00000001:3.0:1788881037.951341:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881037.951344:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881037.951351:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881037.951352:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881053 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881037.951356:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881037.951357:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788881037.951381:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788881037.951383:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000131c1a55. 00000400:00000200:3.0:1788881037.951385:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881037.951387:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881037.951389:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e249bdf9 00000400:00000010:3.0:1788881037.951390:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e249bdf9. 00000100:00000001:3.0:1788881037.951397:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881037.951399:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881037.951403:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881037.951404:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881037.951408:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881037.951408:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788881037.952623:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881037.952632:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881037.952634:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881037.952636:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881037.952640:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881037.952651:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11e80 00000400:00000200:2.0:1788881037.952656:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1329 [1] + 224 00000800:00000001:2.0:1788881037.952661:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881037.952665:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881037.952667:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881037.952669:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881037.952671:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881037.952674:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881037.952680:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881037.952692:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881037.952696:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881037.952697:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881037.952724:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881037.952727:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788881037.952733:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881037.952734:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788881037.952736:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000400:00000200:1.0:1788881037.952740:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c6ecbca9 00000400:00000010:1.0:1788881037.952742:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c6ecbca9. 00000100:00000001:1.0:1788881037.952745:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881037.952749:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881037.952758:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881037.952764:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881037.952766:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881037.952769:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881037.952772:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881037.952775:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788881037.952777:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788881037.952778:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881037.952782:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881037.952783:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881037.952785:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881037.952787:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788881037.952790:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=105 v=5 (5 5 5 5) 00000400:00000200:2.0:1788881037.952791:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:1.0:1788881037.952798:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=105 v=5 (1 1 1 1) 00000400:00000200:2.0:1788881037.952800:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b11e00 00000100:00000001:1.0:1788881037.952807:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881037.952810:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881037.952810:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x132d [1] + 224 00000800:00000001:2.0:1788881037.952815:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:1.0:1788881037.952816:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788881037.952818:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881037.952820:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00010000:00000001:1.0:1788881037.952821:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881037.952824:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881037.952827:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788881037.952828:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000200:2.0:1788881037.952831:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881037.952832:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788881037.952837:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788881037.952839:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881037.952843:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881037.952848:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881037.952869:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881037.952872:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881037.952874:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881037.952878:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881037.952879:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881037.952884:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881037.952888:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1788881037.952891:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780705408:192.168.204.111@tcp:400: 00000800:00000001:2.0:1788881037.952891:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881037.952894:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881037.952901:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881037.952902:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788881037.952906:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:2.0:1788881037.952914:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b11f00 00000100:00000001:0.0:1788881037.952916:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881037.952918:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780705408/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881037.952924:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788881037.952930:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1335 [1] + 224 00000100:00000001:1.0:1788881037.952933:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000800:00000001:2.0:1788881037.952934:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:1.0:1788881037.952936:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000400:00000001:0.0:1788881037.952937:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788881037.952939:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788881037.952944:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000010:1.0:1788881037.952945:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000bef544e8. 00000400:00000200:2.0:1788881037.952945:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:0.0:1788881037.952949:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 02000000:00000001:1.0:1788881037.952952:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000200:2.0:1788881037.952959:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000020:00000001:1.0:1788881037.952960:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000200:0.0:1788881037.952962:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000020:00000040:1.0:1788881037.952964:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000010:0.0:1788881037.952967:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000020:00000001:1.0:1788881037.952969:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881037.952973:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 00000100:00000001:0.0:1788881037.952974:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788881037.952975:0:201279:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:1.0:1788881037.952981:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788881037.952984:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881037.952985:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000200:0.0:1788881037.952990:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881037.952991:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881037.952992:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881037.952995:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881037.953000:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881037.953008:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881037.953012:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881037.953019:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881037.953022:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881037.953069:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881037.953069:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881037.953091:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881037.953097:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881037.953101:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881037.953104:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881037.953108:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881037.953110:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881037.953113:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881037.953113:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881037.953122:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788881037.953124:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=115 v=5 (5 5 5 5) 00000800:00000001:2.0:1788881037.953126:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881037.953132:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881037.953135:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:0.0:1788881037.953136:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=115 v=5 (1 1 1 1) 00000400:00000001:1.0:1788881037.953139:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881037.953141:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788881037.953144:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000010:1.0:1788881037.953144:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001738de44. 00000100:00000001:0.0:1788881037.953151:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881037.953153:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00010000:00000001:0.0:1788881037.953156:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000010:1.0:1788881037.953157:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00010000:00000001:0.0:1788881037.953159:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881037.953161:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788881037.953163:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881037.953170:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881037.953170:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881037.953262:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881037.953266:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881037.953276:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881037.953279:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881037.953281:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881037.953282:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881037.953283:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881037.953288:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881037.953288:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788881037.953293:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881037.953296:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881037.953301:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788881037.953303:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=115 v=5 (5 5 5 5) 00000100:00001000:1.0:1788881037.953319:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=49 v=5 (1 1 0 0) 00000100:00000001:1.0:1788881037.953331:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881037.953335:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881037.953339:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780705280:192.168.204.111@tcp:400: 00010000:00000001:1.0:1788881037.953340:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788881037.953346:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881037.953354:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:1.0:1788881037.953355:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881037.953365:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788881037.953365:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788881037.953370:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788881037.953372:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000040:0.0:1788881037.953375:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780705280/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881037.953375:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881037.953382:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881037.953428:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:1.0:1788881037.953436:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881037.953437:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788881037.953438:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881037.953442:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788881037.953444:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000003222657. 00000100:00000040:1.0:1788881037.953447:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881037.953454:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00100000:1.0:1788881037.953459:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780705536:192.168.204.111@tcp:400: 00000020:00000001:0.0:1788881037.953461:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:1.0:1788881037.953464:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:0.0:1788881037.953465:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788881037.953469:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881037.953473:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780705536/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881053 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788881037.953474:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881037.953479:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 02000000:00000001:0.0:1788881037.953487:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788881037.953488:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881037.953490:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:1.0:1788881037.953491:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000010:0.0:1788881037.953492:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 02000000:00000010:1.0:1788881037.953493:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a48931c2. 00000100:00000001:0.0:1788881037.953502:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:1.0:1788881037.953503:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788881037.953510:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:1.0:1788881037.953511:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788881037.953518:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:1.0:1788881037.953519:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788881037.953529:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881037.953542:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 00000100:00000001:0.0:1788881037.953547:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:1.0:1788881037.953547:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788881037.953551:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881037.953551:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881037.953559:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:0.0:1788881037.953560:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881037.953568:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881037.953569:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881037.953572:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881037.953575:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881037.953578:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881037.953580:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881037.953584:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881037.953587:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881038.014265:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881038.014270:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881038.014276:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881038.014277:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881038.014319:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881038.014323:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881038.014327:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881039.038342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881039.038349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881039.038354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881039.038358:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881039.038360:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881039.038363:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881039.038366:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881039.038368:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881039.038370:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881039.038371:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881039.038424:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881039.038429:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881039.038432:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881040.062352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881040.062357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881040.062362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881040.062363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881040.062413:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881040.062416:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881040.062418:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881041.086368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881041.086376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881041.086383:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881041.086383:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881041.086443:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881041.086449:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881041.086452:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881042.110332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881042.110341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881042.110347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881042.110348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881042.110406:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881042.110410:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881042.110413:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881043.070302:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306674127, at 4306674176 00000100:00000010:0.0:1788881043.070317:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000007e595eb4. 00000100:00000040:1.0:1788881043.070371:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306674127, at 4306674176 00000100:00000040:3.0:1788881043.070371:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306674127, at 4306674176 00000100:00000040:2.0:1788881043.070374:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306674127, at 4306674176 00000100:00000040:0.0:1788881043.070375:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788881043.070394:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881043.070396:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000076127301. 00000100:00000010:0.0:1788881043.070401:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d4e901df. 00000020:00000040:0.0:1788881043.070406:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788881043.070408:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005c6a0f3f. 00000100:00000010:3.0:1788881043.070410:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045db2575. 00000100:00000001:0.0:1788881043.070411:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788881043.070416:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788881043.070421:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788881043.070427:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000fc8f9b7c. 00000100:00000001:0.0:1788881043.070442:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881043.070452:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788881043.070460:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788881043.070462:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881043.070468:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d4e901df] to pc [ptlrpcd_00_01+1] req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881043.070487:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881043.070489:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788881043.070491:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788881043.070493:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b8846eac. 00000020:00000040:0.0:1788881043.070495:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881043.070497:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788881043.070498:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788881043.070499:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788881043.070501:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000050c5542b. 00000100:00000001:0.0:1788881043.070504:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881043.070506:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788881043.070511:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881043.070511:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881043.070516:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881043.070518:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788881043.070524:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881043.070525:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881043.070527:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788881043.070534:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b8846eac] to pc [ptlrpcd_00_02+2] req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881043.070535:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881043.070543:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881043.070546:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881043.070552:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788881043.070554:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788881043.070557:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788881043.070560:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780705664:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881043.070563:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788881043.070568:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881043.070570:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1788881043.070571:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788881043.070573:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881043.070574:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788881043.070579:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000802a821. 00000100:00000001:1.0:1788881043.070579:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788881043.070579:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000020:00000040:0.0:1788881043.070589:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881043.070590:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:3.0:1788881043.070591:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a5da162d. 00000100:00000001:1.0:1788881043.070595:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788881043.070596:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788881043.070602:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881043.070602:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881043.070605:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881043.070609:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788881043.070610:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e494c472. 00000100:00000001:2.0:1788881043.070614:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881043.070621:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881043.070623:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881043.070626:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004f63ce78. 00000100:00000040:0.0:1788881043.070635:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:3.0:1788881043.070643:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000100:00000200:3.0:1788881043.070654:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780705664, portal 25 00000100:00000001:1.0:1788881043.070657:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788881043.070666:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881043.070667:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881043.070667:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881043.070674:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849369472 : -126822860182144 : ffff8ca7c0782d80) 00000100:00000001:0.0:1788881043.070677:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881043.070678:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1788881043.070684:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881059 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881043.070686:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788881043.070687:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000802a821] to pc [ptlrpcd_00_03+3] req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881043.070697:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788881043.070712:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780705792:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881043.070733:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788881043.070734:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788881043.070739:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788881043.070740:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788881043.070741:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881043.070741:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000999bcd04. 02000000:00000001:1.0:1788881043.070749:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881043.070750:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:3.0:1788881043.070750:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780705664, offset 0 02000000:00000001:1.0:1788881043.070760:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788881043.070761:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:3.0:1788881043.070765:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788881043.070766:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12011) 02000000:00000010:1.0:1788881043.070768:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002b77e71a. 00000100:00000001:2.0:1788881043.070768:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788881043.070772:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881043.070778:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001738de44. 00000400:00000200:3.0:1788881043.070780:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:1.0:1788881043.070789:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000400:00000200:3.0:1788881043.070789:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788881043.070797:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:1.0:1788881043.070800:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780705792, portal 10 00000400:00000200:3.0:1788881043.070801:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788881043.070802:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788881043.070811:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788881043.070815:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788881043.070817:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849364096 : -126822860187520 : ffff8ca7c0781880) 00000100:00000040:2.0:1788881043.070819:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881043.070827:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788881043.070835:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881043.070841:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [591:591:256:4294967295] 192.168.204.111@tcp LPNI seq info [591:591:8:4294967295] 00000400:00000200:3.0:1788881043.070867:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788881043.070871:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881043.070888:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788881043.070888:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788881043.070900:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000001:1.0:1788881043.070902:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:2.0:1788881043.070903:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788881043.070908:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788881043.070916:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881043.070921:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000200:3.0:1788881043.070921:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00100000:2.0:1788881043.070926:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780705920:192.168.204.111@tcp:400: 00000400:00000010:1.0:1788881043.070932:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000161fe575. 00000100:00000200:1.0:1788881043.070940:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780705792, offset 0 00000100:00000001:2.0:1788881043.070940:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:3.0:1788881043.070943:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:2.0:1788881043.070947:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000800:00000200:3.0:1788881043.070951:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 02000000:00000001:2.0:1788881043.070952:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881043.070961:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788881043.070962:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788881043.070968:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000feb2a015. 02000000:00000001:2.0:1788881043.070971:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881043.070975:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:2.0:1788881043.070975:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000100:00000001:3.0:1788881043.070977:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881043.070983:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a3b2cf4a. 00000400:00000200:1.0:1788881043.070987:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788881043.070989:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306679247 00000100:00000200:2.0:1788881043.070990:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780705920, portal 10 00000400:00000200:1.0:1788881043.070994:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788881043.070998:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000010:3.0:1788881043.070998:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014c076ca. 00000400:00000200:1.0:1788881043.071000:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788881043.071005:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849370368 : -126822860181248 : ffff8ca7c0783100) 00000400:00000200:1.0:1788881043.071011:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788881043.071011:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000040:2.0:1788881043.071019:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881043.071022:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306679247 00000400:00000200:1.0:1788881043.071030:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [592:592:255:4294967295] 192.168.204.111@tcp LPNI seq info [592:592:7:4294967295] 00000100:00000010:3.0:1788881043.071033:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000dc3cc638. 00000100:00000001:3.0:1788881043.071041:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788881043.071051:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306679247 00000100:00000010:3.0:1788881043.071061:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005869c8be. 00000100:00000001:2.0:1788881043.071071:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788881043.071071:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:0.0:1788881043.071076:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:1.0:1788881043.071076:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788881043.071082:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306679247 00000100:00000040:2.0:1788881043.071085:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788881043.071089:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000010:3.0:1788881043.071095:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e83229b9. 00000400:00000010:2.0:1788881043.071100:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000006bff322d. 00000400:00000200:0.0:1788881043.071102:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788881043.071102:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:1.0:1788881043.071112:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788881043.071112:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780705920, offset 0 00000100:00000001:3.0:1788881043.071115:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788881043.071122:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000010:1.0:1788881043.071122:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000004e0f6846. 00000100:00000001:3.0:1788881043.071129:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881043.071138:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:1.0:1788881043.071140:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788881043.071141:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881043.071145:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000999bcd04 00000400:00000200:2.0:1788881043.071150:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:0.0:1788881043.071151:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000999bcd04. 00000100:00000001:0.0:1788881043.071159:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:1.0:1788881043.071161:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788881043.071166:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000004e0f6846 type 1, nob 320 niov 1 nkiov 1 00000100:00000200:0.0:1788881043.071170:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881059 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881043.071173:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881043.071186:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 146868 usecs (147 jiffies) 00000400:00000200:2.0:1788881043.071187:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:1.0:1788881043.071195:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306674323 00000400:00000200:2.0:1788881043.071199:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788881043.071200:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788881043.071203:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306679247 00000400:00000200:2.0:1788881043.071208:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:0.0:1788881043.071210:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881059 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881043.071211:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306674323 00000100:00000001:3.0:1788881043.071219:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881043.071220:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788881043.071224:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306679247 00000100:00000001:0.0:1788881043.071256:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881043.071258:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:2.0:1788881043.071263:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [593:593:254:4294967295] 192.168.204.111@tcp LPNI seq info [593:593:6:4294967295] 00000100:00000001:1.0:1788881043.071275:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306674324 00000100:00000001:3.0:1788881043.071275:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881043.071279:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881043.071281:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306679247 00000100:00000001:3.0:1788881043.071283:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881043.071289:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881043.071291:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306674324 00000100:00000001:3.0:1788881043.071294:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881043.071298:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306679247 00000100:00000001:3.0:1788881043.071301:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881043.071305:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788881043.071311:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788881043.071313:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881043.071319:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881043.071323:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881043.071324:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:2.0:1788881043.071331:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 00000100:00000001:1.0:1788881043.071334:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881043.071336:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881043.071341:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881043.071345:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:2.0:1788881043.071345:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788881043.071348:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788881043.071350:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788881043.071353:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788881043.071356:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000004e0f6846. 00000800:00000200:2.0:1788881043.071357:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788881043.071363:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788881043.071364:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788881043.071375:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881043.071376:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788881043.071386:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000161fe575 00000100:00000040:2.0:1788881043.071387:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11147 usecs (12 jiffies) 00000400:00000010:0.0:1788881043.071388:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000161fe575. 00000100:00000001:0.0:1788881043.071394:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788881043.071395:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306674189 00000100:00000200:0.0:1788881043.071399:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881043.071404:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306679247 00000100:00000001:2.0:1788881043.071415:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306674189 00000100:00000001:0.0:1788881043.071416:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881043.071421:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881043.071423:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306679247 00000100:00000001:2.0:1788881043.071431:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306674189 00000100:00000001:0.0:1788881043.071444:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881043.071446:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788881043.071457:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788881043.071457:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306679247 00000800:00000010:0.0:1788881043.071463:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000100:00000001:2.0:1788881043.071466:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306674189 00000400:00000200:0.0:1788881043.071468:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788881043.071475:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306679247 00000400:00000200:0.0:1788881043.071483:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881043.071484:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788881043.071486:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000006bff322d 00000400:00000010:0.0:1788881043.071489:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000006bff322d. 00000100:00000001:2.0:1788881043.071492:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881043.071493:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788881043.071497:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881043.071498:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881043.071510:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881043.071512:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881043.071518:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881043.071527:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881043.071529:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881043.071531:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788881043.071538:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788881043.071543:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881043.071545:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881043.071547:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881043.071549:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881043.072425:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881043.072448:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881043.072451:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881043.072454:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881043.072459:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881043.072469:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b11f80 00000400:00000200:2.0:1788881043.072476:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1341 [1] + 224 00000800:00000001:2.0:1788881043.072490:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881043.072500:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881043.072503:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881043.072509:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881043.072516:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881043.072520:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881043.072530:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881043.072545:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881043.072552:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881043.072555:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881043.072641:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881043.072645:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788881043.072649:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788881043.072651:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788881043.072653:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004f63ce78. 00000400:00000200:3.0:1788881043.072656:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000400:00000010:3.0:1788881043.072657:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000800:00000001:2.0:1788881043.072663:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788881043.072663:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:2.0:1788881043.072674:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788881043.072674:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881043.072680:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881043.072681:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881043.072685:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881043.072690:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12000 00000400:00000200:2.0:1788881043.072693:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1349 [1] + 224 00000800:00000001:2.0:1788881043.072696:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881043.072698:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881043.072699:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881043.072702:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881043.072704:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881043.072709:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881043.072758:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788881043.072768:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881043.072772:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881043.072776:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881043.072781:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881043.072781:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788881043.072786:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881043.072792:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788881043.072793:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788881043.072798:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000800:00000001:2.0:1788881043.072799:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881043.072803:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881043.072807:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881043.072810:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788881043.072814:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788881043.072816:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788881043.072817:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788881043.072823:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788881043.072825:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12080 00000400:00000001:1.0:1788881043.072827:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788881043.072832:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001738de44. 00000100:00001000:3.0:1788881043.072835:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=121 v=5 (5 5 5 5) 00000100:00001000:3.0:1788881043.072839:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=121 v=5 (1 1 1 1) 00000100:00000001:3.0:1788881043.072841:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881043.072842:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881043.072844:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788881043.072847:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881043.072848:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000400:00000010:1.0:1788881043.072854:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000100:00000001:3.0:1788881043.072855:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881043.072859:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788881043.072864:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1351 [1] + 224 00000100:00000040:3.0:1788881043.072865:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881043.072868:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881043.072878:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881043.072887:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881043.072894:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881043.072909:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881043.072922:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881043.072933:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881043.072936:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788881043.072942:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881043.072949:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881043.072959:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788881043.072961:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881043.072990:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881043.072993:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881043.072997:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881043.073003:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881043.073006:0:201287:0:(client.c:1580:after_reply()) Process entered 00000800:00000001:2.0:1788881043.073011:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:1.0:1788881043.073012:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881043.073017:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788881043.073017:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1788881043.073018:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d4e901df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780705664:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881043.073021:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881043.073033:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881043.073038:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788881043.073043:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=111 v=5 (5 5 5 5) 00000100:00000001:3.0:1788881043.073043:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881043.073045:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881043.073051:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1788881043.073051:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d4e901df x1875776780705664/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788881043.073055:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=111 v=5 (1 1 1 1) 00000400:00000001:0.0:1788881043.073058:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881043.073063:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788881043.073068:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000010:0.0:1788881043.073070:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000100:00000001:1.0:1788881043.073075:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881043.073077:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000200:0.0:1788881043.073078:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a3b2cf4a 00010000:00000001:1.0:1788881043.073081:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000010:0.0:1788881043.073082:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a3b2cf4a. 02000000:00000001:3.0:1788881043.073082:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00010000:00000001:1.0:1788881043.073087:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881043.073087:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a5da162d. 00000100:00000001:0.0:1788881043.073088:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788881043.073096:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000200:0.0:1788881043.073097:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881043.073097:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:1.0:1788881043.073102:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000020:00000001:3.0:1788881043.073103:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881043.073107:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788881043.073109:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788881043.073115:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881043.073116:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881043.073121:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000fc8f9b7c. 02000000:00000001:3.0:1788881043.073128:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000040:1.0:1788881043.073129:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881043.073131:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881043.073132:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881043.073136:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d4e901df. 00000100:00000001:3.0:1788881043.073147:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881043.073150:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881043.073157:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881043.073160:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881043.073162:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881043.073172:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881043.073175:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881043.073175:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881043.073177:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788881043.073182:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881043.073183:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881043.073183:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881043.073191:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881043.073194:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881043.073196:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881043.073198:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788881043.073199:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=121 v=5 (5 5 5 5) 00000100:00000001:3.0:1788881043.073200:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881043.073213:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=55 v=5 (1 1 0 0) 00000100:00000040:1.0:1788881043.073217:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881043.073218:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881043.073222:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881043.073228:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788881043.073260:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788881043.073266:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881043.073273:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788881043.073276:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881043.073282:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881043.073287:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788881043.073287:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b8846eac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780705792:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881043.073304:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881043.073306:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881043.073312:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881043.073318:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881043.073320:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881043.073326:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881043.073326:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b8846eac x1875776780705792/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788881043.073335:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881043.073393:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780705920:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881043.073395:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788881043.073399:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788881043.073400:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788881043.073402:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000010:1.0:1788881043.073406:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002b77e71a. 00000100:00000040:0.0:1788881043.073407:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000802a821 x1875776780705920/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881059 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881043.073416:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788881043.073418:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000020:00000001:1.0:1788881043.073423:0:201287:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000001:0.0:1788881043.073424:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000020:00000040:1.0:1788881043.073428:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000010:0.0:1788881043.073429:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000feb2a015. 02000000:00000001:0.0:1788881043.073447:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788881043.073448:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881043.073452:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000050c5542b. 00000020:00000001:0.0:1788881043.073453:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881043.073456:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:1.0:1788881043.073461:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000020:00000001:0.0:1788881043.073462:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881043.073466:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e494c472. 02000000:00000001:1.0:1788881043.073466:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881043.073470:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b8846eac. 02000000:00000001:0.0:1788881043.073473:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881043.073477:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788881043.073477:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881043.073481:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:0.0:1788881043.073482:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000802a821. 00000100:00000001:1.0:1788881043.073492:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881043.073493:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881043.073497:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881043.073501:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881043.073504:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881043.073505:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881043.073507:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881043.073508:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881043.073516:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881043.073520:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881043.073520:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881043.073527:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881043.134351:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881043.134359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881043.134366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881043.134367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881043.134450:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881043.134458:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881043.134462:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881044.158314:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881044.158321:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881044.158327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881044.158328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881044.158410:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881044.158425:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881044.158428:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881045.182382:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881045.182390:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881045.182411:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881045.182412:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881045.182483:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881045.182491:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881045.182496:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881046.206348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881046.206357:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881046.206359:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881046.206362:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881046.206367:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881046.206369:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881046.206371:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881046.206384:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881046.206391:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881046.206392:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881046.206483:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881046.206487:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881046.206490:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881047.230372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881047.230380:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881047.230387:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881047.230388:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881047.230443:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881047.230448:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881047.230451:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788881048.190370:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306679247, at 4306679296 00000100:00000040:3.0:1788881048.190370:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306679247, at 4306679296 00000100:00000040:2.0:1788881048.190372:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306679247, at 4306679296 00000100:00000010:1.0:1788881048.190392:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000dc3cc638. 00000100:00000010:2.0:1788881048.190394:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005869c8be. 00000100:00000010:3.0:1788881048.190397:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e83229b9. 00000100:00000040:0.0:1788881048.190400:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306679247, at 4306679296 00000100:00000010:0.0:1788881048.190420:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014c076ca. 00000100:00000040:1.0:1788881048.190458:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881048.190464:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881048.190470:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788881048.190476:0:147371:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788881048.190481:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881048.190485:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881048.190491:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881048.190500:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f0a0057b. 00000100:00000001:1.0:1788881048.190508:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881048.190519:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881048.190528:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881048.190531:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881048.190538:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000008b25f66] to pc [ptlrpcd_00_00+0] req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881048.190561:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881048.190563:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881048.190566:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881048.190569:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002dcbd5d0. 00000020:00000040:1.0:1788881048.190571:0:147371:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881048.190573:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881048.190574:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881048.190576:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881048.190577:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788881048.190580:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 00000100:00000001:1.0:1788881048.190584:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881048.190584:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881048.190589:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788881048.190592:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:2.0:1788881048.190597:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881048.190608:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881048.190611:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881048.190614:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881048.190617:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881048.190619:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002dcbd5d0] to pc [ptlrpcd_00_01+1] req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881048.190629:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788881048.190631:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881048.190634:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:2.0:1788881048.190634:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881048.190637:0:147371:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:2.0:1788881048.190641:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788881048.190642:0:147371:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000070db75b2. 00000020:00000040:1.0:1788881048.190646:0:147371:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881048.190647:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:2.0:1788881048.190648:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780706048:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881048.190651:0:147371:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788881048.190652:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788881048.190654:0:147371:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788881048.190656:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788881048.190657:0:147371:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881048.190657:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000010:1.0:1788881048.190661:0:147371:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000a371c7dc. 00000100:00000040:3.0:1788881048.190663:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881048.190665:0:147371:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881048.190671:0:147371:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788881048.190680:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788881048.190685:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881048.190686:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881048.190689:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881048.190690:0:147371:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788881048.190691:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788881048.190692:0:147371:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881048.190693:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000010:2.0:1788881048.190696:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008b528dd3. 02000000:00000001:3.0:1788881048.190697:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788881048.190698:0:147371:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000070db75b2] to pc [ptlrpcd_00_02+2] req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881048.190701:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881048.190703:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788881048.190711:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780706176:192.168.204.111@tcp:400: 00000400:00000010:2.0:1788881048.190712:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000038fe0168. 00000100:00000001:3.0:1788881048.190717:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788881048.190720:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:2.0:1788881048.190721:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000023cb0718. 02000000:00000001:3.0:1788881048.190722:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881048.190726:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788881048.190730:0:147371:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881048.190730:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000004080df20. 00000100:00000200:2.0:1788881048.190731:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780706048, portal 25 00000100:00000040:1.0:1788881048.190735:0:147371:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881048.190736:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788881048.190739:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 00000100:00000040:1.0:1788881048.190740:0:147371:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788881048.190748:0:147371:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12016) 00000100:00000040:2.0:1788881048.190748:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881064 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881048.190757:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788881048.190760:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788881048.190763:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000005071f3f0. 00000100:00000001:0.0:1788881048.190765:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881048.190765:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:2.0:1788881048.190766:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780706048, offset 0 00000100:00100000:1.0:1788881048.190772:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:0.0:1788881048.190777:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881048.190777:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881048.190778:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881048.190800:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:2.0:1788881048.190804:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788881048.190806:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881048.190812:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881048.190814:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881048.190817:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788881048.190817:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788881048.190822:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881048.190824:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881048.190826:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788881048.190833:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:3.0:1788881048.190833:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004f63ce78. 00000400:00000200:2.0:1788881048.190835:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788881048.190836:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881048.190840:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 02000000:00000001:1.0:1788881048.190842:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788881048.190845:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788881048.190845:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:3.0:1788881048.190846:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780706176, portal 10 02000000:00000001:1.0:1788881048.190850:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788881048.190853:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780706304:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788881048.190856:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [594:594:256:4294967295] 192.168.204.111@tcp LPNI seq info [594:594:8:4294967295] 00000100:00000001:1.0:1788881048.190860:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788881048.190860:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881048.190865:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938047872 : -126823771503744 : ffff8ca78a268380) 02000000:00000001:1.0:1788881048.190866:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881048.190868:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881048.190871:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788881048.190877:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000063883087. 00000400:00000200:2.0:1788881048.190877:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:3.0:1788881048.190881:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881048.190882:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881048.190888:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001738de44. 00000800:00000200:2.0:1788881048.190892:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881048.190895:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000876f9a21. 00000100:00000200:1.0:1788881048.190898:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780706304, portal 10 00000100:00000001:1.0:1788881048.190900:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881048.190900:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938046976 : -126823771504640 : ffff8ca78a268000) 00000800:00000010:2.0:1788881048.190905:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000045330cf1. 00000100:00000040:1.0:1788881048.190907:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881048.190913:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1788881048.190917:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:1.0:1788881048.190918:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881048.190923:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000002fa6ff6. 00000100:00000200:1.0:1788881048.190925:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780706304, offset 0 00000800:00000200:2.0:1788881048.190928:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:1.0:1788881048.190934:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788881048.190935:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000045330cf1 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788881048.190940:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788881048.190943:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788881048.190945:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788881048.190948:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788881048.190948:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881048.190953:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788881048.190955:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306684367 00000400:00000200:1.0:1788881048.190956:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [595:595:255:4294967295] 192.168.204.111@tcp LPNI seq info [595:595:7:4294967295] 00000100:00000010:2.0:1788881048.190962:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000076127301. 00000400:00000200:1.0:1788881048.190965:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788881048.190971:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881048.190974:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000010:1.0:1788881048.190977:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000004e0f6846. 00000100:00000001:2.0:1788881048.190980:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306684367 00000800:00000200:1.0:1788881048.190983:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:2.0:1788881048.190985:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a3bf26bf. 00000100:00000001:2.0:1788881048.190989:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:1.0:1788881048.190990:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788881048.190995:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000004e0f6846 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788881048.190995:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306684367 00000100:00000001:1.0:1788881048.191001:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788881048.191001:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000077b36062. 00000100:00000040:1.0:1788881048.191004:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11101 usecs (12 jiffies) 00000100:00000001:2.0:1788881048.191005:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788881048.191011:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306679308 00000100:00000001:2.0:1788881048.191011:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306684367 00000100:00000001:1.0:1788881048.191015:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306684367 00000100:00000010:2.0:1788881048.191017:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000c61ceeab. 00000100:00000001:1.0:1788881048.191020:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306679308 00000100:00000001:2.0:1788881048.191022:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788881048.191024:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306684367 00000100:00000001:1.0:1788881048.191028:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306679308 00000100:00000001:2.0:1788881048.191029:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788881048.191030:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788881048.191032:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306684367 00000100:00000001:2.0:1788881048.191039:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881048.191041:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306679308 00000800:00000010:0.0:1788881048.191042:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000045330cf1. 00000100:00000001:2.0:1788881048.191046:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881048.191047:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306684367 00000100:00000001:3.0:1788881048.191055:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788881048.191056:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788881048.191059:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:3.0:1788881048.191063:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881048.191064:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881048.191064:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881048.191071:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881048.191071:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000010:3.0:1788881048.191072:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000a165c7f9. 00000100:00000001:1.0:1788881048.191078:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881048.191078:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881048.191083:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788881048.191085:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881048.191085:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881048.191086:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881048.191095:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881048.191105:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000005071f3f0 00000400:00000010:0.0:1788881048.191111:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000005071f3f0. 00000100:00000001:0.0:1788881048.191126:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881048.191127:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780706176, offset 0 00000400:00000200:3.0:1788881048.191142:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000200:0.0:1788881048.191143:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881064 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881048.191150:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788881048.191158:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788881048.191168:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788881048.191176:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788881048.191179:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788881048.191182:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:3.0:1788881048.191189:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [596:596:254:4294967295] 192.168.204.111@tcp LPNI seq info [596:596:6:4294967295] 00000100:00000040:0.0:1788881048.191195:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881064 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881048.191218:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788881048.191219:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881048.191226:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788881048.191269:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:3.0:1788881048.191275:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000800:00000200:3.0:1788881048.191281:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788881048.191288:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788881048.191291:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788881048.191295:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881048.191298:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 146061 usecs (147 jiffies) 00000100:00000001:3.0:1788881048.191300:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306679444 00000100:00000001:3.0:1788881048.191302:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306684367 00000100:00000001:3.0:1788881048.191304:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306679444 00000100:00000001:3.0:1788881048.191307:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306684367 00000100:00000001:3.0:1788881048.191310:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306679444 00000100:00000001:3.0:1788881048.191312:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306684367 00000100:00000001:3.0:1788881048.191322:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306679444 00000100:00000001:3.0:1788881048.191325:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306684367 00000100:00000001:3.0:1788881048.191328:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881048.191330:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881048.191332:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881048.191335:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881048.191350:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881048.191352:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881048.191355:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881048.191357:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881048.191360:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788881048.191362:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788881048.191364:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:0.0:1788881048.191367:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000004e0f6846. 00000400:00000200:0.0:1788881048.191375:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788881048.191381:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788881048.191385:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000002fa6ff6 00000400:00000010:0.0:1788881048.191387:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000002fa6ff6. 00000100:00000001:0.0:1788881048.191391:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788881048.191394:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881048.191404:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881048.191407:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881048.191432:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881048.191435:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788881048.191481:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788881048.191484:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000400:00000200:0.0:1788881048.191488:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788881048.191492:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788881048.191496:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000a165c7f9 00000400:00000010:0.0:1788881048.191498:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000a165c7f9. 00000100:00000001:0.0:1788881048.191501:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788881048.191505:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881048.191514:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881048.191516:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881048.191525:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881048.191527:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788881048.192716:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881048.192724:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881048.192727:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881048.192729:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881048.192736:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881048.192747:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b12100 00000400:00000200:2.0:1788881048.192761:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1359 [1] + 224 00000800:00000001:2.0:1788881048.192770:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881048.192775:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881048.192777:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881048.192780:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881048.192789:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881048.192793:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881048.192808:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881048.192822:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881048.192830:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881048.192833:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881048.192834:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881048.192836:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881048.192839:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788881048.192842:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881048.192846:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788881048.192846:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12200 00000400:00000200:2.0:1788881048.192852:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1365 [1] + 224 00000800:00000001:2.0:1788881048.192856:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881048.192859:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881048.192860:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881048.192863:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:1.0:1788881048.192864:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881048.192867:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788881048.192868:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000038fe0168. 00000400:00000200:1.0:1788881048.192872:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000023cb0718 00000400:00000010:1.0:1788881048.192874:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000023cb0718. 00000100:00000001:1.0:1788881048.192876:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788881048.192877:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881048.192881:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881048.192884:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881048.192900:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881048.192916:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:2.0:1788881048.192917:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881048.192919:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881048.192924:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881048.192928:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881048.192931:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788881048.192932:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881048.192934:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881048.192935:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881048.192939:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:1.0:1788881048.192941:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=126 v=5 (5 5 5 5) 00000800:00000001:2.0:1788881048.192941:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788881048.192948:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=126 v=5 (1 1 1 1) 00000100:00000001:1.0:1788881048.192951:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881048.192952:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881048.192955:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788881048.192957:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881048.192960:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881048.192964:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881048.192969:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881048.192971:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881048.192973:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881048.192977:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788881048.192983:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780706048:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881048.192986:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881048.192988:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881048.192991:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881048.192991:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881048.192996:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780706048/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788881048.193002:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881048.193010:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788881048.193017:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000010:0.0:1788881048.193018:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001738de44. 02000000:00000001:1.0:1788881048.193020:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788881048.193024:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008b528dd3. 02000000:00000001:1.0:1788881048.193030:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000200:0.0:1788881048.193031:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000876f9a21 00000020:00000001:1.0:1788881048.193033:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788881048.193036:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000010:0.0:1788881048.193038:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000876f9a21. 00000020:00000001:1.0:1788881048.193040:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881048.193044:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f0a0057b. 00000100:00000001:0.0:1788881048.193048:0:201288:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:1.0:1788881048.193049:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788881048.193052:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881048.193053:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000200:0.0:1788881048.193058:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881048.193063:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881048.193065:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881048.193072:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881048.193080:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881048.193086:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881048.193090:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:0.0:1788881048.193094:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881048.193096:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881048.193113:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881048.193116:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881048.193119:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881048.193122:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881048.193132:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788881048.193135:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881048.193136:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881048.193144:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881048.193146:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881048.193149:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881048.193152:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788881048.193154:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=126 v=5 (5 5 5 5) 00000400:00000200:2.0:1788881048.193160:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:0.0:1788881048.193173:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=60 v=5 (1 1 0 0) 00000400:00000200:2.0:1788881048.193177:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12180 00000100:00000001:0.0:1788881048.193189:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:2.0:1788881048.193189:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1361 [1] + 224 00000100:00000001:0.0:1788881048.193199:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881048.193199:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:0.0:1788881048.193206:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1788881048.193212:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881048.193216:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00010000:00080000:0.0:1788881048.193218:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788881048.193224:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881048.193255:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881048.193259:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881048.193268:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788881048.193316:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881048.193321:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788881048.193321:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:0.0:1788881048.193325:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:2.0:1788881048.193328:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881048.193330:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000800:00000001:2.0:1788881048.193332:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881048.193334:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881048.193355:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881048.193364:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881048.193367:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881048.193368:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881048.193371:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881048.193372:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788881048.193376:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788881048.193379:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:0.0:1788881048.193383:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788881048.193385:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004f63ce78. 00000400:00000200:3.0:1788881048.193391:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000400:00000010:3.0:1788881048.193395:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000001:3.0:1788881048.193404:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788881048.193411:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881048.193415:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780706304:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881048.193437:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788881048.193440:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881048.193446:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788881048.193456:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:0.0:1788881048.193459:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000070db75b2 x1875776780706304/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881048.193460:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881048.193465:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788881048.193468:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881048.193471:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881048.193476:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881048.193480:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881048.193490:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00001000:3.0:1788881048.193490:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=116 v=5 (5 5 5 5) 02000000:00000001:0.0:1788881048.193496:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00001000:3.0:1788881048.193497:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=116 v=5 (1 1 1 1) 00000100:00000001:3.0:1788881048.193501:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000010:0.0:1788881048.193502:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000063883087. 00000100:00000001:3.0:1788881048.193503:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881048.193509:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00010000:00000001:3.0:1788881048.193512:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788881048.193516:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881048.193521:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788881048.193521:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788881048.193524:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000020:00000040:0.0:1788881048.193527:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881048.193528:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788881048.193532:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881048.193540:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000040:3.0:1788881048.193541:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788881048.193547:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000a371c7dc. 00000100:00000001:3.0:1788881048.193558:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881048.193560:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788881048.193561:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881048.193564:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881048.193569:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:3.0:1788881048.193570:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788881048.193574:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000070db75b2. 00000100:00000001:0.0:1788881048.193585:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00100000:3.0:1788881048.193585:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780706176:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881048.193591:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881048.193594:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881048.193596:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881048.193597:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881048.193602:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002dcbd5d0 x1875776780706176/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881064 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881048.193608:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881048.193609:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788881048.193614:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788881048.193615:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881048.193617:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000004080df20. 02000000:00000001:3.0:1788881048.193622:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788881048.193624:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000001:3.0:1788881048.193624:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881048.193629:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881048.193631:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881048.193632:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881048.193634:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 02000000:00000001:3.0:1788881048.193641:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881048.193643:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881048.193644:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002dcbd5d0. 00000100:00000001:3.0:1788881048.193648:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881048.193649:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881048.193651:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881048.193654:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881048.193660:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881048.193662:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881048.193663:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881048.254359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881048.254366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881048.254373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881048.254376:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881048.254378:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881048.254381:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881048.254384:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881048.254386:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881048.254388:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881048.254390:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881048.254477:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881048.254484:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881048.254488:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881049.278384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881049.278392:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881049.278399:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881049.278400:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881049.278462:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881049.278466:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881049.278468:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881050.302387:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881050.302396:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881050.302401:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881050.302402:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881050.302474:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881050.302479:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881050.302484:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881051.326364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881051.326376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881051.326384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881051.326385:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881051.326441:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881051.326446:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881051.326448:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881052.350374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881052.350382:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881052.350390:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881052.350392:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881052.350458:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881052.350462:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881052.350465:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881053.310347:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306684367, at 4306684416 00000100:00000040:1.0:1788881053.310347:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306684367, at 4306684416 00000100:00000040:2.0:1788881053.310348:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306684367, at 4306684416 00000100:00000040:3.0:1788881053.310348:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881053.310360:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881053.310365:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000077b36062. 00000100:00000010:0.0:1788881053.310367:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000076127301. 00000100:00000010:3.0:1788881053.310369:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000020:00000040:3.0:1788881053.310377:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:1.0:1788881053.310378:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a3bf26bf. 00000100:00000001:3.0:1788881053.310383:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881053.310386:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881053.310390:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881053.310399:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:3.0:1788881053.310405:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881053.310414:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788881053.310423:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881053.310424:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881053.310429:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_03+3] req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881053.310453:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881053.310455:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881053.310458:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881053.310460:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788881053.310462:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881053.310464:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881053.310466:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881053.310467:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881053.310469:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:2.0:1788881053.310480:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881053.310483:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881053.310485:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788881053.310486:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000001:0.0:1788881053.310488:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881053.310490:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881053.310491:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788881053.310501:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881053.310501:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881053.310502:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881053.310505:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881053.310508:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_00+0] req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881053.310510:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881053.310513:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881053.310517:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788881053.310523:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881053.310529:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881053.310530:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788881053.310532:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881053.310534:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881053.310538:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:2.0:1788881053.310540:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000010:3.0:1788881053.310541:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 02000000:00000001:2.0:1788881053.310543:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000020:00000040:3.0:1788881053.310546:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:2.0:1788881053.310547:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881053.310549:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788881053.310551:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881053.310553:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:2.0:1788881053.310554:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780706432:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788881053.310554:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00100000:0.0:1788881053.310556:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:1.0:1788881053.310559:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881053.310559:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881053.310562:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881053.310565:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000010:3.0:1788881053.310566:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:0.0:1788881053.310570:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788881053.310571:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788881053.310571:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881053.310575:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788881053.310576:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881053.310580:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788881053.310581:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788881053.310582:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881053.310586:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881053.310588:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788881053.310593:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000010:2.0:1788881053.310595:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003deec3bc. 00000100:00000040:0.0:1788881053.310596:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881053.310596:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881053.310602:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881053.310605:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_01+1] req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788881053.310612:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000038fe0168. 00000400:00000010:2.0:1788881053.310620:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000acd25b9c. 00000100:00000200:2.0:1788881053.310627:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780706432, portal 25 00000100:00000001:3.0:1788881053.310627:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881053.310633:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881053.310634:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788881053.310638:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000100:00000001:0.0:1788881053.310641:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881053.310643:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881053.310644:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788881053.310646:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881069 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788881053.310649:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000040:3.0:1788881053.310651:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788881053.310656:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881053.310661:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12022) 02000000:00000001:0.0:1788881053.310662:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788881053.310665:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:0.0:1788881053.310666:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881053.310667:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788881053.310670:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881053.310671:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881053.310676:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881053.310677:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788881053.310680:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780706560:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881053.310682:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:2.0:1788881053.310683:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ac9b6c42. 00000100:00000040:3.0:1788881053.310683:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306684367, at 4306684416 00000100:00000001:0.0:1788881053.310685:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788881053.310687:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000200:2.0:1788881053.310689:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780706432, offset 0 02000000:00000001:0.0:1788881053.310690:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881053.310695:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788881053.310695:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000c61ceeab. 00000100:00000040:1.0:1788881053.310697:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881053.310699:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788881053.310700:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000010:0.0:1788881053.310704:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002b77e71a. 02000000:00000001:0.0:1788881053.310708:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881053.310714:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000007a5329cd. 00000400:00000200:2.0:1788881053.310715:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788881053.310716:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:0.0:1788881053.310719:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000046e672b3. 00000100:00000001:1.0:1788881053.310719:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881053.310723:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788881053.310724:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:0.0:1788881053.310726:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780706560, portal 10 00000100:00000001:0.0:1788881053.310729:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:1.0:1788881053.310729:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788881053.310732:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788881053.310733:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 02000000:00000001:1.0:1788881053.310734:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881053.310739:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00100000:1.0:1788881053.310741:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780706688:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788881053.310744:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788881053.310745:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788881053.310750:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788881053.310755:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [597:597:256:4294967295] 192.168.204.111@tcp LPNI seq info [597:597:8:4294967295] 02000000:00000001:1.0:1788881053.310756:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881053.310759:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881053.310761:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:1.0:1788881053.310764:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788881053.310767:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:1.0:1788881053.310769:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000fdf771c7. 00000400:00000010:0.0:1788881053.310771:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000eda3020e. 02000000:00000001:1.0:1788881053.310774:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881053.310777:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780706560, offset 0 00000400:00000010:1.0:1788881053.310780:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000007389ff7. 00000400:00000200:2.0:1788881053.310781:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:0.0:1788881053.310788:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:2.0:1788881053.310792:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881053.310797:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000800:00000010:2.0:1788881053.310798:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000400:00000200:0.0:1788881053.310799:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788881053.310806:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:1.0:1788881053.310806:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780706688, portal 10 00000800:00000200:2.0:1788881053.310811:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788881053.310812:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788881053.310816:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788881053.310816:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000400:00000200:0.0:1788881053.310820:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1788881053.310825:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788881053.310826:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788881053.310826:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788881053.310832:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788881053.310835:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [598:598:255:4294967295] 192.168.204.111@tcp LPNI seq info [598:598:7:4294967295] 00000100:00000001:1.0:1788881053.310846:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788881053.310851:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881053.310853:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881053.310856:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788881053.310858:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306689487 00000400:00000010:1.0:1788881053.310859:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000030b8c48c. 00000100:00000200:1.0:1788881053.310866:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780706688, offset 0 00000100:00000010:2.0:1788881053.310866:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000077b36062. 00000800:00000200:0.0:1788881053.310872:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881053.310874:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000010:0.0:1788881053.310876:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e494c472. 00000100:00000001:2.0:1788881053.310880:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306689487 00000400:00000200:1.0:1788881053.310881:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788881053.310885:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:2.0:1788881053.310893:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ca0fac21. 00000400:00000200:1.0:1788881053.310894:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:0.0:1788881053.310896:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788881053.310899:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:0.0:1788881053.310901:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e494c472 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788881053.310904:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306689487 00000400:00000200:1.0:1788881053.310905:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788881053.310906:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788881053.310910:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ea9971ca. 00000100:00000040:0.0:1788881053.310913:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 145265 usecs (146 jiffies) 00000100:00000001:2.0:1788881053.310914:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:1.0:1788881053.310915:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788881053.310919:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306684562 00000100:00000001:2.0:1788881053.310920:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306689487 00000100:00000001:0.0:1788881053.310924:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306689487 00000400:00000200:1.0:1788881053.310924:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:2.0:1788881053.310926:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c99cc67. 00000100:00000001:0.0:1788881053.310930:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306684562 00000400:00000200:1.0:1788881053.310931:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788881053.310932:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1788881053.310937:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306689487 00000100:00000001:2.0:1788881053.310939:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788881053.310942:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [599:599:254:4294967295] 192.168.204.111@tcp LPNI seq info [599:599:6:4294967295] 00000100:00000001:0.0:1788881053.310943:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306684562 00000100:00000001:2.0:1788881053.310944:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881053.310948:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306689487 00000100:00000001:2.0:1788881053.310951:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881053.310953:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306684562 00000800:00000200:3.0:1788881053.310954:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788881053.310955:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881053.310957:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306689487 00000100:00000001:0.0:1788881053.310962:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:3.0:1788881053.310963:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000100:00000001:0.0:1788881053.310968:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881053.310968:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:3.0:1788881053.310971:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881053.310973:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881053.310978:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881053.310981:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881053.310983:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881053.310986:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:1.0:1788881053.310986:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881053.310989:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881053.310990:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ac9b6c42 00000100:00000001:0.0:1788881053.310992:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788881053.310994:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f0a0057b. 00000400:00000010:3.0:1788881053.310994:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ac9b6c42. 00000100:00000001:3.0:1788881053.311001:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:1.0:1788881053.311004:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:3.0:1788881053.311009:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881069 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788881053.311017:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788881053.311024:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f0a0057b type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788881053.311029:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788881053.311031:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881053.311036:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881069 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881053.311037:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11056 usecs (12 jiffies) 00000100:00000001:1.0:1788881053.311041:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306684428 00000100:00000001:1.0:1788881053.311044:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306689487 00000100:00000001:1.0:1788881053.311046:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306684428 00000100:00000001:1.0:1788881053.311048:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306689487 00000100:00000001:1.0:1788881053.311050:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306684428 00000100:00000001:1.0:1788881053.311053:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306689487 00000100:00000001:1.0:1788881053.311055:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306684428 00000100:00000001:1.0:1788881053.311057:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306689487 00000100:00000001:1.0:1788881053.311060:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881053.311062:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881053.311064:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881053.311067:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881053.311098:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881053.311099:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788881053.311113:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788881053.311115:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e494c472. 00000400:00000200:3.0:1788881053.311117:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881053.311120:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881053.311122:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000eda3020e 00000400:00000010:3.0:1788881053.311123:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000eda3020e. 00000100:00000001:3.0:1788881053.311125:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881053.311127:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881053.311132:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881053.311133:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881053.311138:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881053.311139:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788881053.311165:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788881053.311168:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f0a0057b. 00000400:00000200:3.0:1788881053.311171:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881053.311175:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881053.311178:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000030b8c48c 00000400:00000010:3.0:1788881053.311185:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000030b8c48c. 00000100:00000001:3.0:1788881053.311189:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881053.311192:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881053.311201:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881053.311203:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881053.311210:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881053.311211:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788881053.312722:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881053.312732:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881053.312734:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881053.312735:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881053.312741:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881053.312751:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b12280 00000400:00000200:2.0:1788881053.312758:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1371 [1] + 224 00000800:00000001:2.0:1788881053.312764:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881053.312767:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881053.312770:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881053.312773:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881053.312778:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881053.312781:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881053.312791:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881053.312810:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881053.312816:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881053.312820:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881053.312828:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881053.312831:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881053.312835:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881053.312840:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12300 00000100:00000001:0.0:1788881053.312844:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788881053.312845:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1379 [1] + 224 00000100:00000001:0.0:1788881053.312847:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788881053.312851:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788881053.312854:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:2.0:1788881053.312855:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788881053.312857:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788881053.312859:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:0.0:1788881053.312860:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000038fe0168. 00000400:00000200:2.0:1788881053.312865:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788881053.312866:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000acd25b9c 00000400:00000010:0.0:1788881053.312869:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000acd25b9c. 00000100:00000001:0.0:1788881053.312872:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788881053.312876:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881053.312884:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881053.312885:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881053.312909:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881053.312917:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881053.312926:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881053.312930:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881053.312935:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881053.312939:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881053.312942:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881053.312944:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881053.312945:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1788881053.312949:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881053.312953:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1788881053.312956:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=131 v=5 (5 5 5 5) 00000800:00000001:2.0:1788881053.312960:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788881053.312961:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=131 v=5 (1 1 1 1) 00000800:00000001:2.0:1788881053.312966:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881053.312967:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881053.312970:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881053.312971:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1788881053.312975:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788881053.312976:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:0.0:1788881053.312980:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881053.312981:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788881053.312981:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788881053.312985:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788881053.312987:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788881053.312988:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788881053.312992:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000007a5329cd. 00000400:00000200:2.0:1788881053.312993:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12380 00000400:00000200:3.0:1788881053.312996:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000046e672b3 00000100:00000040:0.0:1788881053.312998:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788881053.312999:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000046e672b3. 00000100:00000001:3.0:1788881053.313006:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1788881053.313012:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1381 [1] + 224 00000100:00000200:3.0:1788881053.313013:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881053.313020:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000800:00000001:2.0:1788881053.313022:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881053.313028:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000800:00000001:2.0:1788881053.313029:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788881053.313031:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881053.313032:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881053.313034:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881053.313042:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881053.313042:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788881053.313044:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881053.313045:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881053.313046:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:2.0:1788881053.313047:0:201280:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:3.0:1788881053.313050:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000200:2.0:1788881053.313053:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881053.313053:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881053.313055:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881053.313058:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788881053.313066:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=121 v=5 (5 5 5 5) 00000100:00001000:3.0:1788881053.313074:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=121 v=5 (1 1 1 1) 00000100:00100000:0.0:1788881053.313077:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780706432:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788881053.313078:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881053.313078:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881053.313081:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881053.313085:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788881053.313089:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881053.313090:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881053.313093:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788881053.313095:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881053.313100:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000040:0.0:1788881053.313101:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780706432/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881053.313105:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788881053.313108:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881053.313117:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881053.313124:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881053.313128:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:2.0:1788881053.313130:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788881053.313132:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881053.313134:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000010:0.0:1788881053.313137:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003deec3bc. 00000800:00000001:2.0:1788881053.313137:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788881053.313137:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881053.313140:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881053.313143:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788881053.313147:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000040:3.0:1788881053.313147:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1788881053.313150:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788881053.313155:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881053.313158:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 02000000:00000001:0.0:1788881053.313163:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788881053.313163:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788881053.313167:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00100000:3.0:1788881053.313168:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780706560:192.168.204.111@tcp:400: 00000100:00000010:0.0:1788881053.313169:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000001:0.0:1788881053.313174:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881053.313175:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881053.313177:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881053.313180:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881053.313180:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788881053.313182:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881053.313187:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:3.0:1788881053.313187:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881053.313189:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788881053.313192:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780706560/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881053.313193:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881053.313194:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000007389ff7. 00000100:00000001:0.0:1788881053.313200:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788881053.313201:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000100:00000001:0.0:1788881053.313203:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881053.313206:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00000100:00000001:1.0:1788881053.313212:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788881053.313212:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788881053.313215:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788881053.313217:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002b77e71a. 00000100:00000200:1.0:1788881053.313219:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881053.313227:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881053.313228:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881053.313261:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881053.313263:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788881053.313267:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881053.313269:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 00000100:00000001:1.0:1788881053.313274:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:3.0:1788881053.313275:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881053.313277:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788881053.313279:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:3.0:1788881053.313279:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:3.0:1788881053.313284:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881053.313285:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788881053.313286:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881053.313290:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788881053.313291:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881053.313293:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881053.313294:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881053.313296:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881053.313298:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881053.313299:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881053.313301:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881053.313303:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788881053.313309:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=131 v=5 (5 5 5 5) 00000100:00001000:1.0:1788881053.313315:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=65 v=5 (1 1 0 0) 00000100:00000001:1.0:1788881053.313318:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881053.313320:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881053.313321:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788881053.313326:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788881053.313336:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881053.313337:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788881053.313338:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788881053.313339:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788881053.313340:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881053.313342:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881053.313346:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881053.313347:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881053.313348:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881053.313350:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788881053.313355:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780706688:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881053.313357:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881053.313358:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881053.313360:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780706688/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881069 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881053.313363:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788881053.313365:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788881053.313366:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000fdf771c7. 02000000:00000001:1.0:1788881053.313368:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788881053.313369:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788881053.313370:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788881053.313372:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881053.313373:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:1.0:1788881053.313375:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788881053.313376:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881053.313376:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:1.0:1788881053.313378:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881053.313379:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881053.313380:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881053.313383:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881053.313384:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881053.313386:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881053.313387:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881053.374362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881053.374370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881053.374376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881053.374378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881053.374451:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881053.374455:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881053.374459:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881054.398342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881054.398347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881054.398353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881054.398354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881054.398402:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881054.398405:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881054.398408:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881055.422352:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881055.422358:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881055.422363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881055.422364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881055.422429:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881055.422433:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881055.422436:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881056.446386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881056.446399:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881056.446402:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881056.446416:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881056.446422:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881056.446425:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881056.446428:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881056.446430:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881056.446436:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881056.446438:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881056.446504:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881056.446510:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881056.446514:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881057.470355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881057.470363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881057.470370:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881057.470371:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881057.470433:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881057.470439:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881057.470442:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881058.430314:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306689487, at 4306689536 00000100:00000040:1.0:1788881058.430319:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306689487, at 4306689536 00000100:00000040:2.0:1788881058.430320:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306689487, at 4306689536 00000100:00000040:3.0:1788881058.430329:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000010:0.0:1788881058.430335:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000077b36062. 00000100:00000010:2.0:1788881058.430338:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ea9971ca. 00000100:00000001:3.0:1788881058.430340:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881058.430341:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ca0fac21. 00000100:00000010:3.0:1788881058.430355:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000aa33757e. 00000020:00000040:3.0:1788881058.430362:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788881058.430367:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881058.430377:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881058.430383:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881058.430393:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:3.0:1788881058.430401:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881058.430414:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788881058.430427:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881058.430430:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881058.430437:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa33757e] to pc [ptlrpcd_00_02+2] req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881058.430469:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881058.430473:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881058.430478:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788881058.430482:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:3.0:1788881058.430483:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e9e280b6. 00000100:00000001:0.0:1788881058.430487:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000020:00000040:3.0:1788881058.430487:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881058.430490:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788881058.430492:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881058.430495:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:0.0:1788881058.430498:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881058.430498:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788881058.430499:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881058.430504:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881058.430506:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:0.0:1788881058.430513:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881058.430515:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881058.430519:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788881058.430522:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881058.430524:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881058.430526:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780706816:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881058.430533:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788881058.430536:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881058.430542:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788881058.430551:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788881058.430554:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788881058.430556:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881058.430557:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881058.430563:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:3.0:1788881058.430564:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e9e280b6] to pc [ptlrpcd_00_03+3] req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788881058.430570:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000099f3ac40. 02000000:00000001:0.0:1788881058.430575:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881058.430583:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000100:00000001:3.0:1788881058.430584:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881058.430588:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:0.0:1788881058.430592:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000001:3.0:1788881058.430592:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881058.430597:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000100:00000200:0.0:1788881058.430602:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780706816, portal 25 00000020:00000040:3.0:1788881058.430602:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881058.430605:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881058.430606:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788881058.430608:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881058.430610:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788881058.430610:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788881058.430612:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643442560 : -126823066109056 : ffff8ca7b431fb80) 02000000:00000001:3.0:1788881058.430614:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881058.430615:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:3.0:1788881058.430622:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000040:1.0:1788881058.430626:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788881058.430627:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881074 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881058.430628:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881058.430637:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788881058.430662:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788881058.430665:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881058.430667:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000040:0.0:1788881058.430670:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881058.430670:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881058.430671:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881058.430676:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e73139ca. 02000000:00000001:1.0:1788881058.430678:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:0.0:1788881058.430680:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780706816, offset 0 00000100:00000040:3.0:1788881058.430681:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_00+0] req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881058.430682:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788881058.430689:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881058.430694:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:1.0:1788881058.430698:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780706944:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881058.430706:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788881058.430709:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881058.430711:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881058.430714:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788881058.430715:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788881058.430721:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881058.430722:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008f22742c. 00000100:00000040:3.0:1788881058.430725:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788881058.430727:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881058.430729:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788881058.430734:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:1.0:1788881058.430735:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000007389ff7. 00000400:00000200:0.0:1788881058.430738:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788881058.430742:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12027) 00000400:00000200:0.0:1788881058.430743:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:1.0:1788881058.430744:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000400:00000200:0.0:1788881058.430750:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:1.0:1788881058.430751:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780706944, portal 10 00000400:00000200:0.0:1788881058.430756:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [600:600:256:4294967295] 192.168.204.111@tcp LPNI seq info [600:600:8:4294967295] 00000100:00000001:1.0:1788881058.430756:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881058.430759:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881058.430761:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643430016 : -126823066121600 : ffff8ca7b431ca80) 00000100:00100000:3.0:1788881058.430764:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:2.0:1788881058.430766:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881058.430770:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881058.430772:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881058.430780:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788881058.430784:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788881058.430796:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881058.430797:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788881058.430797:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881058.430802:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:0.0:1788881058.430805:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000087d24527. 00000100:00000040:1.0:1788881058.430806:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881058.430808:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881058.430811:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:1.0:1788881058.430813:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000007dbd5e. 00000100:00000001:3.0:1788881058.430814:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:0.0:1788881058.430818:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:1.0:1788881058.430820:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780706944, offset 0 00000100:00000040:3.0:1788881058.430822:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881058.430833:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881058.430835:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788881058.430837:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:3.0:1788881058.430840:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1788881058.430844:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788881058.430844:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000800:00000200:0.0:1788881058.430845:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000087d24527 type 1, nob 320 niov 1 nkiov 1 02000000:00000001:3.0:1788881058.430849:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788881058.430854:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780707072:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788881058.430858:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788881058.430862:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881058.430862:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:1.0:1788881058.430865:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:3.0:1788881058.430866:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788881058.430870:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881058.430871:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306694607 00000400:00000200:1.0:1788881058.430872:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788881058.430875:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788881058.430879:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:0.0:1788881058.430880:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009a4dee8c. 02000000:00000010:3.0:1788881058.430881:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000030b982bf. 00000400:00000200:1.0:1788881058.430884:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:3.0:1788881058.430886:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881058.430894:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [601:601:255:4294967295] 192.168.204.111@tcp LPNI seq info [601:601:7:4294967295] 00000400:00000010:3.0:1788881058.430894:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004f63ce78. 00000100:00000001:0.0:1788881058.430896:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000010:3.0:1788881058.430902:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000100:00000001:0.0:1788881058.430904:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306694607 00000100:00000200:3.0:1788881058.430909:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780707072, portal 10 00000100:00000010:0.0:1788881058.430911:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000a4f85af. 00000100:00000001:3.0:1788881058.430915:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881058.430917:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788881058.430918:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000400:00000200:1.0:1788881058.430919:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788881058.430924:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306694607 00000100:00000040:3.0:1788881058.430930:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788881058.430931:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006bd48548. 00000800:00000200:1.0:1788881058.430936:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881058.430937:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788881058.430943:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306694607 00000100:00000001:3.0:1788881058.430949:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788881058.430950:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306689487 00000800:00000010:1.0:1788881058.430951:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000131c1a55. 00000100:00000040:3.0:1788881058.430956:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881058.430957:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881058.430961:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881058.430962:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e78d7264. 00000100:00000001:0.0:1788881058.430965:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881058.430966:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:3.0:1788881058.430968:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780707072, offset 0 00000100:00000001:0.0:1788881058.430969:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881058.430977:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:1.0:1788881058.430978:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788881058.430980:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:3.0:1788881058.430982:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881058.430986:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000200:1.0:1788881058.430986:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000131c1a55 type 1, nob 320 niov 1 nkiov 1 00000800:00000200:2.0:1788881058.430987:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788881058.430989:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788881058.430993:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788881058.430994:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881058.430996:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788881058.430996:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000087d24527. 00000100:00000001:0.0:1788881058.431000:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881058.431001:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788881058.431004:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:1.0:1788881058.431005:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 144475 usecs (145 jiffies) 00000100:00000001:1.0:1788881058.431013:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306689681 00000400:00000200:2.0:1788881058.431013:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881058.431016:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788881058.431020:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306694607 00000400:00000200:3.0:1788881058.431023:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788881058.431027:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306689681 00000400:00000200:3.0:1788881058.431029:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788881058.431033:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306694607 00000400:00000200:3.0:1788881058.431037:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [602:602:254:4294967295] 192.168.204.111@tcp LPNI seq info [602:602:6:4294967295] 00000100:00000001:1.0:1788881058.431038:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306689681 00000100:00000001:1.0:1788881058.431043:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306694607 00000100:00000001:1.0:1788881058.431047:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306689681 00000100:00000001:1.0:1788881058.431052:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306694607 00000400:00000200:3.0:1788881058.431057:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788881058.431058:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881058.431064:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881058.431069:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788881058.431069:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788881058.431070:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e73139ca 00000400:00000010:2.0:1788881058.431073:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e73139ca. 00000100:00000001:1.0:1788881058.431075:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881058.431079:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1788881058.431082:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:2.0:1788881058.431085:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881074 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881058.431086:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881058.431091:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881058.431093:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881058.431095:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881058.431100:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881058.431110:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881058.431114:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881074 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881058.431124:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881058.431127:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788881058.431158:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788881058.431161:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000131c1a55. 00000400:00000200:2.0:1788881058.431167:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881058.431172:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881058.431176:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000007dbd5e 00000400:00000010:2.0:1788881058.431178:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000007dbd5e. 00000100:00000001:2.0:1788881058.431182:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881058.431187:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881058.431196:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881058.431200:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000010:3.0:1788881058.431206:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000800:00000200:3.0:1788881058.431216:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788881058.431218:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881058.431222:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788881058.431225:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788881058.431229:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788881058.431267:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881058.431309:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 11012 usecs (12 jiffies) 00000100:00000001:3.0:1788881058.431313:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306689549 00000100:00000001:3.0:1788881058.431316:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306694607 00000100:00000001:3.0:1788881058.431318:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306689549 00000100:00000001:3.0:1788881058.431321:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306694607 00000100:00000001:3.0:1788881058.431324:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306689549 00000100:00000001:3.0:1788881058.431326:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306694607 00000100:00000001:3.0:1788881058.431328:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306689549 00000100:00000001:3.0:1788881058.431330:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306694607 00000100:00000001:3.0:1788881058.431333:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881058.431335:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881058.431337:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881058.431340:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788881058.431368:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788881058.431372:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:0.0:1788881058.431375:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788881058.431380:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788881058.431382:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e78d7264 00000400:00000010:0.0:1788881058.431384:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e78d7264. 00000100:00000001:0.0:1788881058.431386:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788881058.431389:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881058.431396:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881058.431397:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881058.431402:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881058.431408:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788881058.432448:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881058.432463:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881058.432466:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881058.432467:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881058.432473:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881058.432483:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b12400 00000400:00000200:2.0:1788881058.432489:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1389 [1] + 224 00000800:00000001:2.0:1788881058.432494:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881058.432499:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881058.432501:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881058.432503:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881058.432507:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881058.432510:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881058.432516:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881058.432528:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881058.432534:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881058.432537:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881058.432561:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881058.432564:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788881058.432569:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881058.432571:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788881058.432572:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000400:00000200:0.0:1788881058.432575:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:0.0:1788881058.432577:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881058.432579:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788881058.432582:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881058.432593:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881058.432601:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881058.432603:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881058.432606:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881058.432609:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881058.432612:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881058.432615:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881058.432618:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881058.432625:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=136 v=5 (5 5 5 5) 00000100:00001000:0.0:1788881058.432630:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=136 v=5 (1 1 1 1) 00000100:00000001:0.0:1788881058.432633:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881058.432634:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881058.432637:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788881058.432639:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881058.432641:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881058.432644:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881058.432650:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881058.432652:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881058.432654:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881058.432657:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881058.432661:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000aa33757e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780706816:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881058.432672:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881058.432673:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881058.432675:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000aa33757e x1875776780706816/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881058.432681:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881058.432683:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881058.432686:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000099f3ac40. 02000000:00000001:0.0:1788881058.432688:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788881058.432690:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881058.432692:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:0.0:1788881058.432693:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881058.432696:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:0.0:1788881058.432700:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881058.432701:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881058.432703:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000aa33757e. 00000100:00000001:0.0:1788881058.432705:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881058.432706:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881058.432708:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881058.432712:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881058.432713:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881058.432714:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881058.432715:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881058.432798:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881058.432806:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881058.432809:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881058.432833:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881058.432840:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881058.432849:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12480 00000400:00000200:2.0:1788881058.432854:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1391 [1] + 224 00000800:00000001:2.0:1788881058.432857:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881058.432859:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881058.432861:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881058.432868:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881058.432871:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881058.432874:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881058.432879:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881058.432891:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881058.432895:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881058.432898:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881058.432899:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881058.432901:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881058.432905:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881058.432909:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12500 00000400:00000200:2.0:1788881058.432912:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1399 [1] + 224 00000800:00000001:2.0:1788881058.432915:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881058.432919:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881058.432921:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881058.432924:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881058.432926:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881058.432928:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881058.432933:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788881058.432935:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881058.432938:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788881058.432944:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881058.432948:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788881058.432948:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000010:1.0:1788881058.432953:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000007389ff7. 00000400:00000200:1.0:1788881058.432958:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000400:00000010:1.0:1788881058.432960:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00000100:00000001:1.0:1788881058.432964:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:2.0:1788881058.432967:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788881058.432971:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881058.432972:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788881058.432986:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881058.432994:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881058.432997:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881058.432999:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881058.433003:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788881058.433004:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788881058.433005:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881058.433008:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788881058.433010:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881058.433013:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788881058.433016:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00001000:1.0:1788881058.433020:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=126 v=5 (5 5 5 5) 00000400:00000001:3.0:1788881058.433021:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788881058.433027:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004f63ce78. 00000100:00001000:1.0:1788881058.433028:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=126 v=5 (1 1 1 1) 00000100:00000001:1.0:1788881058.433032:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881058.433033:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881058.433035:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788881058.433038:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881058.433041:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788881058.433044:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788881058.433051:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000200:3.0:1788881058.433054:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000100:00000001:1.0:1788881058.433055:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881058.433061:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000040:1.0:1788881058.433066:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881058.433072:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788881058.433080:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881058.433085:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:3.0:1788881058.433085:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881058.433088:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881058.433103:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788881058.433130:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e9e280b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780706944:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881058.433152:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788881058.433153:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881058.433158:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881058.433168:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e9e280b6 x1875776780706944/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881058.433189:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881058.433191:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788881058.433195:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788881058.433196:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788881058.433200:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008f22742c. 00000100:00000001:3.0:1788881058.433203:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881058.433206:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:3.0:1788881058.433208:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000020:00000001:1.0:1788881058.433210:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788881058.433213:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000040:1.0:1788881058.433214:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881058.433218:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788881058.433220:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881058.433224:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 02000000:00000001:3.0:1788881058.433226:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881058.433231:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788881058.433264:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881058.433265:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e9e280b6. 00000100:00000001:1.0:1788881058.433270:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881058.433272:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881058.433274:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881058.433281:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00001000:3.0:1788881058.433281:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=136 v=5 (5 5 5 5) 00000100:00000001:1.0:1788881058.433284:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881058.433292:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:3.0:1788881058.433293:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=70 v=5 (1 1 0 0) 00000100:00000001:1.0:1788881058.433296:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881058.433306:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881058.433309:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881058.433312:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788881058.433319:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788881058.433328:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881058.433331:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788881058.433333:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788881058.433335:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788881058.433337:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881058.433341:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881058.433348:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881058.433352:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881058.433354:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881058.433357:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788881058.433379:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780707072:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881058.433385:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881058.433387:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881058.433390:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780707072/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881074 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881058.433397:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788881058.433400:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788881058.433403:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000030b982bf. 02000000:00000001:3.0:1788881058.433407:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881058.433410:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881058.433412:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788881058.433414:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881058.433417:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 02000000:00000001:3.0:1788881058.433420:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881058.433422:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881058.433424:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000001:3.0:1788881058.433427:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881058.433428:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881058.433430:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881058.433438:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881058.433439:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881058.433442:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881058.433444:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881058.494372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881058.494381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881058.494386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881058.494390:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881058.494392:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881058.494394:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881058.494398:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881058.494401:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881058.494404:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881058.494406:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881058.494479:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881058.494485:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881058.494488:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881059.518360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881059.518367:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881059.518373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881059.518374:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881059.518453:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881059.518458:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881059.518462:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881060.542347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881060.542354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881060.542362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881060.542363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881060.542432:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881060.542437:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881060.542440:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881061.566320:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881061.566327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881061.566331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881061.566332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881061.566373:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881061.566381:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881061.566384:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881062.590366:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881062.590372:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881062.590379:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881062.590381:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881062.590433:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881062.590437:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881062.590453:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788881063.550297:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306694607, at 4306694656 00000100:00000040:0.0:1788881063.550310:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306694607, at 4306694656 00000100:00000010:1.0:1788881063.550316:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000a4f85af. 00000100:00000040:3.0:1788881063.550316:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306694607, at 4306694656 00000100:00000010:0.0:1788881063.550319:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009a4dee8c. 00000100:00000010:3.0:1788881063.550328:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c99cc67. 00000100:00000040:3.0:1788881063.550373:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788881063.550375:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306694607, at 4306694656 00000100:00000001:3.0:1788881063.550377:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881063.550384:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000020:00000040:3.0:1788881063.550388:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000010:2.0:1788881063.550389:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000006bd48548. 00000100:00000001:3.0:1788881063.550396:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881063.550399:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881063.550405:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881063.550426:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 00000100:00000001:3.0:1788881063.550433:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881063.550443:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788881063.550454:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881063.550456:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881063.550461:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_01+1] req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881063.550495:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881063.550497:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881063.550501:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881063.550505:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000020:00000040:3.0:1788881063.550508:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881063.550510:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881063.550511:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788881063.550513:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788881063.550513:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881063.550519:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:3.0:1788881063.550520:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000007660b3e. 00000100:00000001:0.0:1788881063.550523:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788881063.550526:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881063.550532:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881063.550535:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788881063.550536:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881063.550545:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881063.550566:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881063.550567:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788881063.550570:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881063.550570:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881063.550578:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:3.0:1788881063.550579:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_02+2] req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881063.550582:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881063.550587:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881063.550594:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780707200:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881063.550600:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788881063.550606:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881063.550608:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881063.550610:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788881063.550615:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000948807ec. 02000000:00000001:0.0:1788881063.550618:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881063.550618:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881063.550623:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:0.0:1788881063.550627:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000100:00000001:3.0:1788881063.550628:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881063.550633:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000100:00000001:2.0:1788881063.550636:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:0.0:1788881063.550637:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000020:00000040:3.0:1788881063.550637:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881063.550642:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:1.0:1788881063.550643:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:0.0:1788881063.550647:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780707200, portal 25 00000100:00100000:1.0:1788881063.550647:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:1.0:1788881063.550652:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881063.550653:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881063.550658:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000100:00000001:2.0:1788881063.550660:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881063.550660:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:1.0:1788881063.550664:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:3.0:1788881063.550664:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881063.550666:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881063.550668:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:3.0:1788881063.550672:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:1.0:1788881063.550673:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881063.550676:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881063.550678:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881063.550679:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788881063.550685:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788881063.550685:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881063.550687:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:0.0:1788881063.550694:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788881063.550699:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ff62fd67. 00000100:00000200:0.0:1788881063.550704:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780707200, offset 0 00000100:00000001:1.0:1788881063.550712:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881063.550713:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881063.550715:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881063.550716:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881063.550717:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:1.0:1788881063.550719:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788881063.550722:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788881063.550723:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_03+3] req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881063.550725:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788881063.550733:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780707328:192.168.204.111@tcp:400: 00000400:00000200:0.0:1788881063.550734:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788881063.550738:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788881063.550741:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788881063.550742:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881063.550745:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881063.550747:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:1.0:1788881063.550749:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788881063.550750:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:1.0:1788881063.550755:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000026532f47. 00000400:00000200:0.0:1788881063.550757:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:3.0:1788881063.550758:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:1.0:1788881063.550761:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881063.550764:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1788881063.550769:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000007389ff7. 00000100:00100000:2.0:1788881063.550770:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000040:3.0:1788881063.550771:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12032) 00000400:00000200:0.0:1788881063.550776:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:1.0:1788881063.550780:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000100:00000001:2.0:1788881063.550781:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788881063.550789:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788881063.550791:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881063.550795:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788881063.550798:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [603:603:256:4294967295] 192.168.204.111@tcp LPNI seq info [603:603:8:4294967295] 00000100:00000001:2.0:1788881063.550800:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:1.0:1788881063.550803:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780707328, portal 10 00000100:00000001:2.0:1788881063.550805:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881063.550811:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788881063.550812:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881063.550815:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000040:2.0:1788881063.550823:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881063.550825:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881063.550843:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788881063.550875:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788881063.550889:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000001c4b7888. 00000100:00000001:2.0:1788881063.550893:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788881063.550897:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788881063.550897:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881063.550906:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000800:00000200:0.0:1788881063.550907:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000040:1.0:1788881063.550907:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788881063.550912:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788881063.550913:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000065ad0e62. 02000000:00000001:2.0:1788881063.550920:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881063.550921:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780707328, offset 0 00000800:00000200:0.0:1788881063.550924:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00100000:2.0:1788881063.550929:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780707456:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788881063.550935:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000800:00000200:0.0:1788881063.550936:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000001c4b7888 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788881063.550943:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788881063.550949:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1788881063.550950:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:2.0:1788881063.550956:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881063.550958:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788881063.550959:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881063.550964:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788881063.550969:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306699727 00000400:00000200:1.0:1788881063.550969:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000010:2.0:1788881063.550971:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000115c4ce2. 02000000:00000001:2.0:1788881063.550981:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881063.550982:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000010:0.0:1788881063.550984:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009a4dee8c. 00000400:00000010:2.0:1788881063.550989:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000005398250a. 00000100:00000001:0.0:1788881063.550991:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000010:2.0:1788881063.551001:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001e963647. 00000100:00000001:0.0:1788881063.551002:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306699727 00000400:00000200:1.0:1788881063.551004:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:2.0:1788881063.551010:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780707456, portal 10 00000100:00000010:0.0:1788881063.551012:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009f22a9a0. 00000100:00000001:2.0:1788881063.551017:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:1.0:1788881063.551018:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [604:604:255:4294967295] 192.168.204.111@tcp LPNI seq info [604:604:7:4294967295] 00000100:00000001:2.0:1788881063.551020:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000001:0.0:1788881063.551021:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000040:2.0:1788881063.551038:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881063.551039:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306699727 00000100:00000010:0.0:1788881063.551063:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000bc13d1e4. 00000400:00000200:1.0:1788881063.551064:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788881063.551076:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788881063.551087:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306699727 00000100:00000001:2.0:1788881063.551094:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:0.0:1788881063.551096:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e602733. 00000800:00000200:1.0:1788881063.551099:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881063.551104:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:3.0:1788881063.551104:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:2.0:1788881063.551105:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788881063.551107:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000131c1a55. 00000400:00000010:2.0:1788881063.551117:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000015cc594c. 00000800:00000010:3.0:1788881063.551117:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000001c4b7888. 00000100:00000001:0.0:1788881063.551118:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788881063.551118:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:2.0:1788881063.551127:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780707456, offset 0 00000100:00000001:0.0:1788881063.551128:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881063.551129:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000200:1.0:1788881063.551131:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788881063.551134:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881063.551135:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000131c1a55 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788881063.551143:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881063.551145:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881063.551149:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881063.551153:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788881063.551154:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 143693 usecs (144 jiffies) 00000400:00000200:2.0:1788881063.551161:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881063.551163:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881063.551163:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306694800 00000100:00000001:1.0:1788881063.551166:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306699727 00000400:00000200:3.0:1788881063.551168:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ff62fd67 00000100:00000001:1.0:1788881063.551169:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306694800 00000400:00000010:3.0:1788881063.551173:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ff62fd67. 00000100:00000001:1.0:1788881063.551174:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306699727 00000400:00000200:2.0:1788881063.551175:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788881063.551178:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881063.551184:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306694800 00000100:00000001:0.0:1788881063.551189:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881063.551189:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306699727 00000400:00000200:2.0:1788881063.551192:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788881063.551192:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788881063.551193:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881063.551193:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306694800 00000100:00000001:0.0:1788881063.551202:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881063.551202:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306699727 00000400:00000200:2.0:1788881063.551208:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:1.0:1788881063.551209:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881063.551213:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881063.551215:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881063.551217:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1788881063.551219:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881063.551221:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881063.551222:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881063.551223:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881063.551224:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881063.551232:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788881063.551265:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [605:605:254:4294967295] 192.168.204.111@tcp LPNI seq info [605:605:6:4294967295] 00000400:00000200:2.0:1788881063.551283:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:3.0:1788881063.551291:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788881063.551294:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788881063.551309:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e53a8f34. 00000100:00000001:3.0:1788881063.551311:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881063.551316:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881079 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788881063.551325:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788881063.551336:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881063.551342:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788881063.551342:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788881063.551347:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e53a8f34 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788881063.551353:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881063.551357:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10968 usecs (11 jiffies) 00000100:00000001:2.0:1788881063.551360:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306694668 00000100:00000001:2.0:1788881063.551363:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306699727 00000800:00000200:3.0:1788881063.551363:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788881063.551368:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306694668 00000800:00000010:3.0:1788881063.551370:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000131c1a55. 00000100:00000001:2.0:1788881063.551374:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306699727 00000400:00000200:3.0:1788881063.551375:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788881063.551377:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306694668 00000100:00000001:2.0:1788881063.551378:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306699727 00000100:00000001:2.0:1788881063.551379:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306694668 00000100:00000001:2.0:1788881063.551381:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306699727 00000100:00000001:2.0:1788881063.551386:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881063.551391:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881063.551394:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881063.551398:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881063.551400:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000065ad0e62 00000400:00000010:3.0:1788881063.551403:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000065ad0e62. 00000100:00000001:2.0:1788881063.551404:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881063.551418:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881063.551423:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881063.551429:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881063.551433:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881063.551437:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881063.551438:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788881063.551462:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788881063.551464:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e53a8f34. 00000400:00000200:3.0:1788881063.551465:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881063.551467:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881063.551469:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000015cc594c 00000400:00000010:3.0:1788881063.551470:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000015cc594c. 00000100:00000001:3.0:1788881063.551471:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881063.551473:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881063.551477:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881063.551478:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881063.551482:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881063.551482:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788881063.552611:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881063.552633:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881063.552637:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881063.552640:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881063.552648:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881063.552659:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12600 00000400:00000200:2.0:1788881063.552665:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x13a9 [1] + 224 00000800:00000001:2.0:1788881063.552671:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881063.552674:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881063.552676:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881063.552679:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881063.552684:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881063.552687:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881063.552695:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881063.552716:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881063.552723:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881063.552725:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881063.552726:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881063.552728:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881063.552730:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881063.552733:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b12580 00000400:00000200:2.0:1788881063.552737:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x13a1 [1] + 224 00000800:00000001:2.0:1788881063.552739:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881063.552741:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881063.552742:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881063.552744:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881063.552746:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881063.552748:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881063.552754:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881063.552759:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881063.552763:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788881063.552769:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881063.552771:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788881063.552771:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000010:1.0:1788881063.552774:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000007389ff7. 00000400:00000200:1.0:1788881063.552777:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000400:00000010:1.0:1788881063.552779:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00000100:00000001:1.0:1788881063.552781:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:2.0:1788881063.552785:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:1.0:1788881063.552786:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881063.552789:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881063.552793:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881063.552797:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788881063.552799:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788881063.552808:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788881063.552812:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881063.552814:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788881063.552815:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12680 00000100:00000001:1.0:1788881063.552816:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881063.552821:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788881063.552823:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:2.0:1788881063.552824:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x13b1 [1] + 224 02000000:00000001:1.0:1788881063.552825:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881063.552829:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:2.0:1788881063.552831:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788881063.552833:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788881063.552835:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 02000000:00000001:1.0:1788881063.552837:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881063.552839:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788881063.552840:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:1.0:1788881063.552846:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=131 v=5 (5 5 5 5) 00000400:00000200:2.0:1788881063.552846:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:0.0:1788881063.552847:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000400:00000200:2.0:1788881063.552855:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00001000:1.0:1788881063.552860:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=131 v=5 (1 1 1 1) 00000400:00000200:0.0:1788881063.552861:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000100:00000001:1.0:1788881063.552865:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000010:0.0:1788881063.552866:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:1.0:1788881063.552868:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881063.552874:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788881063.552876:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788881063.552877:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:1.0:1788881063.552879:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881063.552885:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000200:2.0:1788881063.552885:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881063.552888:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000200:0.0:1788881063.552891:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881063.552892:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788881063.552897:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881063.552908:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881063.552920:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881063.552965:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788881063.552971:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881063.552971:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881063.552973:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881063.552979:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881063.552980:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881063.552985:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881063.552994:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881063.552997:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881063.553003:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881063.553010:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000001:3.0:1788881063.553010:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788881063.553015:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788881063.553017:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788881063.553019:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000005398250a. 00000100:00100000:1.0:1788881063.553020:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780707328:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881063.553027:0:201286:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:3.0:1788881063.553028:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001e963647 00000100:00000001:1.0:1788881063.553030:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881063.553031:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000400:00000010:3.0:1788881063.553034:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001e963647. 00000100:00000001:0.0:1788881063.553038:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788881063.553038:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788881063.553040:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788881063.553054:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881063.553056:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780707328/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788881063.553059:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881063.553062:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881063.553074:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=141 v=5 (5 5 5 5) 00000100:00000001:1.0:1788881063.553082:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788881063.553088:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00001000:0.0:1788881063.553094:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=141 v=5 (1 1 1 1) 02000000:00000010:1.0:1788881063.553095:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000026532f47. 00000100:00000200:3.0:1788881063.553101:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881063.553105:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788881063.553109:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000020:00000001:1.0:1788881063.553110:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:0.0:1788881063.553114:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788881063.553115:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:1.0:1788881063.553123:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00010000:00000001:0.0:1788881063.553124:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788881063.553127:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000010:1.0:1788881063.553128:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000007660b3e. 00010000:00000001:0.0:1788881063.553129:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881063.553132:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881063.553139:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788881063.553140:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881063.553140:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881063.553143:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000001:1.0:1788881063.553146:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:3.0:1788881063.553148:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000010:1.0:1788881063.553151:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:3.0:1788881063.553152:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881063.553157:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881063.553160:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:3.0:1788881063.553160:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881063.553165:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788881063.553172:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=141 v=5 (5 5 5 5) 00000100:00000001:1.0:1788881063.553177:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788881063.553184:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=75 v=5 (1 1 0 0) 00000100:00000001:1.0:1788881063.553188:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881063.553192:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881063.553195:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881063.553195:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881063.553204:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788881063.553206:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881063.553208:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881063.553211:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:3.0:1788881063.553213:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881063.553221:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881063.553227:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881063.553260:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788881063.553270:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881063.553283:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780707200:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881063.553285:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788881063.553290:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881063.553292:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881063.553293:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881063.553299:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788881063.553299:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881063.553305:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780707200/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881063.553309:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881063.553359:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881063.553360:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788881063.553364:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881063.553367:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881063.553368:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881063.553377:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000948807ec. 00000100:00000040:3.0:1788881063.553379:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881063.553386:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00100000:3.0:1788881063.553391:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780707456:192.168.204.111@tcp:400: 00000020:00000001:0.0:1788881063.553392:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881063.553398:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788881063.553401:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:0.0:1788881063.553403:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881063.553405:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 02000000:00000001:0.0:1788881063.553410:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788881063.553411:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:0.0:1788881063.553416:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000040:3.0:1788881063.553417:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780707456/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881079 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788881063.553421:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000001:0.0:1788881063.553432:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881063.553432:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788881063.553439:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788881063.553439:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788881063.553442:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000115c4ce2. 00000100:00000001:0.0:1788881063.553446:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788881063.553447:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881063.553449:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881063.553451:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:3.0:1788881063.553454:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788881063.553455:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000010:3.0:1788881063.553456:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 00000100:00000001:0.0:1788881063.553459:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881063.553460:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881063.553464:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881063.553466:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000100:00000001:0.0:1788881063.553467:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881063.553476:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881063.553476:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881063.553478:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881063.553483:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881063.553488:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881063.553491:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881063.553493:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881063.553494:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881063.614323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881063.614330:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881063.614336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881063.614337:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881063.614422:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881063.614428:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881063.614431:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881064.638322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881064.638327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881064.638332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881064.638333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881064.638379:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881064.638388:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881064.638391:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881065.662337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881065.662343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881065.662346:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881065.662347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881065.662402:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881065.662408:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881065.662411:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881066.686322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881066.686330:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881066.686332:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881066.686335:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881066.686340:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881066.686342:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881066.686343:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881066.686345:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881066.686348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881066.686349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881066.686397:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881066.686402:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881066.686419:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881067.710347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881067.710354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881067.710359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881067.710360:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881067.710415:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881067.710418:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881067.710420:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788881068.670387:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306699727, at 4306699776 00000100:00000040:0.0:1788881068.670388:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306699727, at 4306699776 00000100:00000040:2.0:1788881068.670388:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306699727, at 4306699776 00000100:00000040:3.0:1788881068.670391:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306699727, at 4306699776 00000100:00000010:3.0:1788881068.670413:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e602733. 00000100:00000010:1.0:1788881068.670416:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009f22a9a0. 00000100:00000010:0.0:1788881068.670422:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009a4dee8c. 00000100:00000010:2.0:1788881068.670426:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000bc13d1e4. 00000100:00000040:3.0:1788881068.670448:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881068.670455:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881068.670467:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788881068.670472:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788881068.670477:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881068.670480:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881068.670486:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881068.670495:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:3.0:1788881068.670504:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881068.670516:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788881068.670550:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881068.670553:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881068.670559:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_00+0] req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881068.670587:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881068.670589:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881068.670592:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881068.670595:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788881068.670597:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881068.670599:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881068.670601:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881068.670603:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881068.670606:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:3.0:1788881068.670610:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881068.670614:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788881068.670618:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881068.670623:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788881068.670624:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881068.670625:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881068.670626:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881068.670629:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881068.670632:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_01+1] req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881068.670634:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788881068.670643:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881068.670644:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881068.670648:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788881068.670652:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881068.670657:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881068.670657:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881068.670661:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000100:00000001:0.0:1788881068.670662:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788881068.670663:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881068.670667:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1788881068.670667:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:0.0:1788881068.670672:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881068.670673:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881068.670675:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788881068.670676:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881068.670679:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881068.670684:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:0.0:1788881068.670686:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:2.0:1788881068.670687:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881068.670688:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881068.670693:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881068.670694:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788881068.670696:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881068.670697:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881068.670701:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788881068.670702:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780707584:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788881068.670704:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788881068.670707:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780707712:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881068.670711:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788881068.670715:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788881068.670719:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:2.0:1788881068.670720:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881068.670721:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788881068.670721:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881068.670725:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881068.670725:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881068.670728:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_02+2] req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881068.670731:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:2.0:1788881068.670731:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788881068.670738:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a6d90e67. 02000000:00000010:2.0:1788881068.670738:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000c0495ef7. 02000000:00000001:0.0:1788881068.670742:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881068.670742:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881068.670749:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000400:00000010:2.0:1788881068.670749:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 00000100:00000001:3.0:1788881068.670754:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881068.670757:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000010:0.0:1788881068.670758:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000400:00000010:2.0:1788881068.670759:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000040:3.0:1788881068.670765:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000200:2.0:1788881068.670769:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780707584, portal 25 00000100:00000200:0.0:1788881068.670772:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780707712, portal 10 00000100:00000001:2.0:1788881068.670772:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:3.0:1788881068.670772:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12037) 00000100:00000001:2.0:1788881068.670775:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000001:1.0:1788881068.670776:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881068.670777:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881068.670785:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000040:2.0:1788881068.670788:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881068.670791:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788881068.670795:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881068.670801:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788881068.670803:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788881068.670806:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788881068.670810:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881068.670811:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881068.670813:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788881068.670815:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788881068.670818:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c3a5d4ae. 00000100:00000200:2.0:1788881068.670827:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780707584, offset 0 00000100:00000040:0.0:1788881068.670830:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788881068.670836:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000096cf0c4a. 00000400:00000200:2.0:1788881068.670845:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881068.670848:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788881068.670850:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881068.670852:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780707712, offset 0 02000000:00000001:1.0:1788881068.670853:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788881068.670858:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:0.0:1788881068.670864:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788881068.670867:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:1.0:1788881068.670871:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881068.670875:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788881068.670884:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00100000:1.0:1788881068.670885:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780707840:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788881068.670890:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788881068.670895:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788881068.670898:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:2.0:1788881068.670903:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [606:606:256:4294967295] 192.168.204.111@tcp LPNI seq info [606:606:8:4294967295] 02000000:00000001:1.0:1788881068.670907:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881068.670911:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881068.670920:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:2.0:1788881068.670921:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788881068.670932:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881068.670935:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000010:1.0:1788881068.670935:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000760cd5b8. 00000800:00000010:2.0:1788881068.670940:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e53a8f34. 00000400:00000200:0.0:1788881068.670943:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:1.0:1788881068.670956:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881068.670957:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788881068.670967:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1788881068.670969:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788881068.670972:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:1.0:1788881068.670974:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000007389ff7. 00000400:00000200:0.0:1788881068.670978:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [607:607:255:4294967295] 192.168.204.111@tcp LPNI seq info [607:607:7:4294967295] 00000800:00000200:2.0:1788881068.670984:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:1.0:1788881068.670988:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000400:00000200:0.0:1788881068.670997:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:1.0:1788881068.670998:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780707840, portal 10 00000800:00000200:2.0:1788881068.671002:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e53a8f34 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788881068.671006:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881068.671010:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000800:00000200:0.0:1788881068.671014:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788881068.671019:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000b7399401. 00000100:00000040:1.0:1788881068.671019:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881068.671020:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788881068.671034:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788881068.671034:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306704847 00000100:00000010:2.0:1788881068.671048:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ea9971ca. 00000800:00000200:0.0:1788881068.671057:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788881068.671058:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788881068.671060:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:0.0:1788881068.671065:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000b7399401 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:1.0:1788881068.671067:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881068.671067:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306704847 00000400:00000010:1.0:1788881068.671078:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000ae22117. 00000100:00000010:2.0:1788881068.671078:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000043ac658c. 00000100:00000001:0.0:1788881068.671079:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788881068.671082:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788881068.671089:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000010:3.0:1788881068.671091:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e53a8f34. 00000100:00000040:0.0:1788881068.671094:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 142920 usecs (143 jiffies) 00000100:00000200:1.0:1788881068.671095:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780707840, offset 0 00000100:00000001:2.0:1788881068.671096:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306704847 00000400:00000200:3.0:1788881068.671097:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000010:2.0:1788881068.671104:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b63100a0. 00000100:00000001:0.0:1788881068.671106:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306699919 00000100:00000001:2.0:1788881068.671110:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:3.0:1788881068.671113:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881068.671114:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306704847 00000100:00000001:2.0:1788881068.671116:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306704847 00000100:00000001:0.0:1788881068.671123:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306699919 00000400:00000200:1.0:1788881068.671123:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788881068.671126:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001e508d7a. 00000100:00000001:0.0:1788881068.671133:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306704847 00000100:00000001:2.0:1788881068.671135:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788881068.671136:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c3a5d4ae 00000100:00000001:0.0:1788881068.671139:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306699919 00000400:00000010:3.0:1788881068.671141:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c3a5d4ae. 00000100:00000001:0.0:1788881068.671143:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306704847 00000100:00000001:2.0:1788881068.671143:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788881068.671147:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788881068.671149:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788881068.671151:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306699919 00000100:00000001:2.0:1788881068.671151:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881068.671159:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306704847 00000100:00000200:3.0:1788881068.671159:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881068.671161:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881068.671164:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788881068.671165:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881068.671168:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881068.671172:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881068.671179:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788881068.671179:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881068.671181:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881068.671181:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788881068.671190:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788881068.671192:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881068.671192:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881084 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881068.671193:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788881068.671196:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881068.671201:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788881068.671203:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788881068.671205:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881068.671207:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881068.671216:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788881068.671216:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881068.671219:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881068.671221:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [608:608:254:4294967295] 192.168.204.111@tcp LPNI seq info [608:608:6:4294967295] 00000100:00000001:0.0:1788881068.671231:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881068.671261:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881068.671263:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788881068.671264:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881068.671269:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788881068.671279:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788881068.671285:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ac392bfe. 00000100:00000001:0.0:1788881068.671289:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881068.671294:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788881068.671303:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788881068.671306:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ac392bfe type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788881068.671311:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881068.671314:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10923 usecs (11 jiffies) 00000100:00000001:1.0:1788881068.671317:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306699788 00000100:00000001:1.0:1788881068.671319:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306704847 00000100:00000001:1.0:1788881068.671320:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306699788 00000100:00000001:1.0:1788881068.671322:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306704847 00000100:00000001:1.0:1788881068.671323:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306699788 00000100:00000001:1.0:1788881068.671324:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306704847 00000100:00000001:1.0:1788881068.671325:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306699788 00000100:00000001:1.0:1788881068.671326:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306704847 00000100:00000001:1.0:1788881068.671327:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881068.671328:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881068.671332:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881068.671334:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881068.671339:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881068.671341:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881068.671342:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881068.671343:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881068.671344:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881068.671345:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788881068.671359:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788881068.671363:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000b7399401. 00000400:00000200:3.0:1788881068.671368:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881068.671373:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881068.671382:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000096cf0c4a 00000400:00000010:3.0:1788881068.671384:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000096cf0c4a. 00000100:00000001:3.0:1788881068.671387:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881068.671390:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881068.671399:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881068.671400:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881068.671407:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881068.671408:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788881068.671422:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788881068.671526:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ac392bfe. 00000400:00000200:3.0:1788881068.671529:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881068.671532:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881068.671534:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000ae22117 00000400:00000010:3.0:1788881068.671536:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000ae22117. 00000100:00000001:3.0:1788881068.671539:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881068.671542:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881068.671549:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881068.671551:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881068.671557:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881068.671559:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788881068.672844:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881068.672968:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881068.672971:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881068.672972:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881068.672978:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788881068.672987:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b12700 00000400:00000200:3.0:1788881068.672993:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x13b9 [1] + 224 00000800:00000001:3.0:1788881068.673025:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881068.673033:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881068.673035:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881068.673037:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881068.673040:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788881068.673043:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881068.673049:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881068.673060:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788881068.673065:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881068.673067:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881068.673068:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881068.673069:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881068.673072:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788881068.673077:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12780 00000400:00000200:3.0:1788881068.673080:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x13bd [1] + 224 00000100:00000001:2.0:1788881068.673084:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1788881068.673084:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881068.673086:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881068.673087:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788881068.673088:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881068.673092:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000001:2.0:1788881068.673093:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788881068.673096:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788881068.673097:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000400:00000200:2.0:1788881068.673100:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788881068.673101:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:2.0:1788881068.673103:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788881068.673105:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881068.673108:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788881068.673112:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881068.673127:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881068.673135:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881068.673151:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881068.673152:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881068.673154:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788881068.673156:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788881068.673158:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788881068.673159:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881068.673159:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:2.0:1788881068.673163:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788881068.673167:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:2.0:1788881068.673168:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=146 v=5 (5 5 5 5) 00000100:00001000:2.0:1788881068.673173:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=146 v=5 (1 1 1 1) 00000800:00000001:3.0:1788881068.673174:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881068.673175:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788881068.673178:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788881068.673179:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:2.0:1788881068.673181:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:3.0:1788881068.673182:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788881068.673184:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881068.673188:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881068.673189:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788881068.673192:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788881068.673194:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881068.673195:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788881068.673199:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12800 00000400:00000001:0.0:1788881068.673200:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881068.673204:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788881068.673204:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788881068.673207:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000100:00000001:2.0:1788881068.673210:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000200:0.0:1788881068.673212:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000100:00000001:2.0:1788881068.673212:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788881068.673215:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881068.673219:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:2.0:1788881068.673219:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881068.673224:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x13c9 [1] + 224 00000100:00000200:0.0:1788881068.673226:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881068.673254:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788881068.673257:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780707584:192.168.204.111@tcp:400: 00000800:00000001:3.0:1788881068.673257:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881068.673267:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881068.673269:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881068.673270:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881068.673270:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881068.673272:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881068.673274:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881068.673276:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881068.673280:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:2.0:1788881068.673280:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780707584/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881068.673280:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881068.673284:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881068.673286:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788881068.673288:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881068.673293:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788881068.673294:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881068.673296:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788881068.673299:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00001000:0.0:1788881068.673301:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=136 v=5 (5 5 5 5) 02000000:00000010:2.0:1788881068.673302:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000c0495ef7. 00000100:00001000:0.0:1788881068.673310:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=136 v=5 (1 1 1 1) 00000100:00000040:3.0:1788881068.673313:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881068.673318:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788881068.673319:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881068.673322:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881068.673322:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788881068.673325:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00010000:00000001:0.0:1788881068.673329:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000001:2.0:1788881068.673330:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788881068.673333:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 00010000:00000001:0.0:1788881068.673334:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881068.673336:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788881068.673336:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881068.673339:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:2.0:1788881068.673340:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000800:00000001:3.0:1788881068.673343:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1788881068.673344:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000010:2.0:1788881068.673344:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000800:00000001:3.0:1788881068.673347:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881068.673349:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788881068.673349:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788881068.673352:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881068.673354:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881068.673359:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881068.673360:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881068.673364:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881068.673364:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881068.673370:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881068.673372:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881068.673374:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881068.673379:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788881068.673381:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881068.673387:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788881068.673391:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000007389ff7. 00000100:00000001:0.0:1788881068.673393:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788881068.673396:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000100:00000001:0.0:1788881068.673398:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000400:00000010:1.0:1788881068.673400:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00000100:00000001:0.0:1788881068.673403:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881068.673405:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881068.673414:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788881068.673415:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881068.673463:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881068.673471:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780707712:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881068.673475:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881068.673477:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881068.673484:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881068.673487:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881068.673488:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881068.673489:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788881068.673492:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881068.673495:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:1.0:1788881068.673496:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881068.673504:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780707712/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788881068.673505:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=146 v=5 (5 5 5 5) 00000100:00001000:1.0:1788881068.673523:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=80 v=5 (1 1 0 0) 00000100:00000001:0.0:1788881068.673524:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881068.673527:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881068.673530:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a6d90e67. 00000100:00000001:1.0:1788881068.673533:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000001:0.0:1788881068.673537:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:1.0:1788881068.673537:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881068.673541:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00010000:00000001:1.0:1788881068.673543:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000040:0.0:1788881068.673544:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881068.673549:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881068.673551:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 00010000:00080000:1.0:1788881068.673552:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881068.673556:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00010000:00000001:1.0:1788881068.673562:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881068.673565:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:1.0:1788881068.673566:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000010:0.0:1788881068.673567:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000040:1.0:1788881068.673570:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881068.673572:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881068.673573:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881068.673577:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881068.673577:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881068.673591:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881068.673596:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881068.673599:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881068.673603:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881068.673611:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881068.673616:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881068.673617:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881068.673626:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881068.673627:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881068.673630:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788881068.673639:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780707840:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881068.673643:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881068.673645:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881068.673647:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780707840/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881084 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881068.673653:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788881068.673655:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788881068.673657:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000760cd5b8. 02000000:00000001:1.0:1788881068.673661:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788881068.673663:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788881068.673665:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:1.0:1788881068.673667:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881068.673669:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:1.0:1788881068.673673:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788881068.673674:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881068.673675:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:1.0:1788881068.673679:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881068.673680:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881068.673681:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881068.673685:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881068.673686:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881068.673688:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881068.673689:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881068.734318:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881068.734324:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881068.734328:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881068.734332:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881068.734335:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881068.734337:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881068.734339:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881068.734341:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881068.734342:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881068.734343:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881068.734389:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881068.734393:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881068.734409:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881069.758353:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881069.758361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881069.758365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881069.758366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881069.758426:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881069.758431:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881069.758434:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881070.782322:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881070.782328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881070.782333:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881070.782334:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881070.782373:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881070.782377:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881070.782379:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881071.806303:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881071.806308:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881071.806312:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881071.806313:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881071.806362:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881071.806366:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881071.806369:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881072.830363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881072.830370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881072.830376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881072.830378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881072.830438:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881072.830445:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881072.830450:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881073.790330:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306704847, at 4306704896 00000100:00000040:1.0:1788881073.790330:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306704847, at 4306704896 00000100:00000040:2.0:1788881073.790334:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306704847, at 4306704896 00000100:00000010:1.0:1788881073.790350:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000043ac658c. 00000100:00000010:0.0:1788881073.790353:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ea9971ca. 00000100:00000010:2.0:1788881073.790359:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b63100a0. 00000100:00000040:1.0:1788881073.790385:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881073.790389:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881073.790394:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000008b25f66. 00000020:00000040:1.0:1788881073.790398:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788881073.790402:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000040:3.0:1788881073.790403:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306704847, at 4306704896 02000000:00000001:1.0:1788881073.790404:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881073.790409:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788881073.790411:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001e508d7a. 02000000:00000010:1.0:1788881073.790415:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f0a0057b. 00000100:00000001:1.0:1788881073.790421:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881073.790430:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881073.790438:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881073.790440:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881073.790444:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000008b25f66] to pc [ptlrpcd_00_03+3] req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881073.790465:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881073.790467:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881073.790470:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881073.790473:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002dcbd5d0. 00000020:00000040:1.0:1788881073.790475:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881073.790476:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881073.790478:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881073.790479:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881073.790480:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 00000100:00000001:1.0:1788881073.790482:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881073.790484:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881073.790489:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881073.790491:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881073.790493:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002dcbd5d0] to pc [ptlrpcd_00_00+0] req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881073.790504:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881073.790508:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788881073.790511:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00000040:1.0:1788881073.790512:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881073.790517:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881073.790518:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881073.790526:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000070db75b2. 00000020:00000040:1.0:1788881073.790530:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881073.790531:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788881073.790533:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:1.0:1788881073.790534:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788881073.790540:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788881073.790541:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881073.790541:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881073.790543:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881073.790550:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881073.790552:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881073.790555:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881073.790556:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:1.0:1788881073.790556:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000a371c7dc. 00000100:00000001:2.0:1788881073.790559:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881073.790563:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881073.790565:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788881073.790566:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881073.790571:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881073.790572:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:2.0:1788881073.790578:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881073.790579:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881073.790622:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788881073.790624:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788881073.790629:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881073.790629:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881073.790641:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000040:1.0:1788881073.790644:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000070db75b2] to pc [ptlrpcd_00_01+1] req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881073.790644:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:0.0:1788881073.790645:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881073.790650:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881073.790653:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881073.790661:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780708096:192.168.204.111@tcp:400: 02000000:00000001:2.0:1788881073.790665:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788881073.790671:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881073.790673:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788881073.790682:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881073.790687:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881073.790695:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788881073.790696:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780707968:192.168.204.111@tcp:400: 02000000:00000001:0.0:1788881073.790707:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788881073.790707:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788881073.790713:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788881073.790714:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000c9708566. 02000000:00000001:2.0:1788881073.790714:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788881073.790720:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881073.790721:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788881073.790722:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881073.790728:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:0.0:1788881073.790731:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000100:00000040:1.0:1788881073.790733:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12042) 00000400:00000010:0.0:1788881073.790741:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000001:3.0:1788881073.790746:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788881073.790751:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788881073.790752:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:0.0:1788881073.790755:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780708096, portal 10 00000100:00000001:3.0:1788881073.790756:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:2.0:1788881073.790760:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000001363d535. 00000100:00000001:0.0:1788881073.790761:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881073.790765:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938047872 : -126823771503744 : ffff8ca78a268380) 00000100:00000040:3.0:1788881073.790765:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881073.790768:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881073.790780:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 00000100:00000001:3.0:1788881073.790782:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881073.790786:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881073.790789:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 02000000:00000001:3.0:1788881073.790793:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:2.0:1788881073.790796:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780707968, portal 25 00000100:00000040:0.0:1788881073.790797:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881073.790798:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881073.790803:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788881073.790807:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938048768 : -126823771502848 : ffff8ca78a268700) 02000000:00000001:3.0:1788881073.790810:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881073.790818:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788881073.790823:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780708224:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881073.790841:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788881073.790858:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:0.0:1788881073.790863:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788881073.790864:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788881073.790868:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881073.790869:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:0.0:1788881073.790872:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008c00e34d. 02000000:00000001:3.0:1788881073.790874:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:2.0:1788881073.790881:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:3.0:1788881073.790881:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b405d9c4. 00000100:00000200:0.0:1788881073.790883:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780708096, offset 0 00000400:00000010:2.0:1788881073.790887:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000005509e00. 02000000:00000001:3.0:1788881073.790887:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881073.790898:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788881073.790899:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004f63ce78. 00000100:00000200:2.0:1788881073.790903:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780707968, offset 0 00000400:00000010:3.0:1788881073.790906:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000100:00000200:3.0:1788881073.790911:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780708224, portal 10 00000400:00000200:0.0:1788881073.790913:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788881073.790920:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788881073.790921:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881073.790926:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938046976 : -126823771504640 : ffff8ca78a268000) 00000400:00000200:0.0:1788881073.790932:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788881073.790934:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881073.790940:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788881073.790945:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788881073.790947:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1788881073.790952:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881073.790955:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000010:3.0:1788881073.790955:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000003539bb5a. 00000100:00000200:3.0:1788881073.790959:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780708224, offset 0 00000400:00000200:0.0:1788881073.790963:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [609:609:256:4294967295] 192.168.204.111@tcp LPNI seq info [609:609:8:4294967295] 00000400:00000200:3.0:1788881073.790966:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881073.790982:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788881073.790989:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788881073.790995:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788881073.790996:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e494c472. 00000400:00000200:2.0:1788881073.791004:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788881073.791010:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788881073.791013:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788881073.791015:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788881073.791018:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788881073.791027:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [610:610:255:4294967295] 192.168.204.111@tcp LPNI seq info [610:610:7:4294967295] 00000800:00000200:0.0:1788881073.791039:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788881073.791046:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e494c472 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788881073.791054:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788881073.791062:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881073.791067:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:2.0:1788881073.791068:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c54a5c6f. 00000400:00000200:3.0:1788881073.791068:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:3.0:1788881073.791079:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:0.0:1788881073.791080:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 142156 usecs (143 jiffies) 00000400:00000200:3.0:1788881073.791088:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788881073.791089:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788881073.791092:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306705039 00000400:00000200:3.0:1788881073.791096:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788881073.791101:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000010:0.0:1788881073.791102:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000085563e6f. 00000800:00000200:2.0:1788881073.791109:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788881073.791111:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [611:611:254:4294967295] 192.168.204.111@tcp LPNI seq info [611:611:6:4294967295] 00000100:00000001:0.0:1788881073.791113:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 0 00000800:00000200:2.0:1788881073.791119:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c54a5c6f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788881073.791122:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306705039 00000100:00000001:2.0:1788881073.791129:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788881073.791130:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000004e64ede4. 00000400:00000200:3.0:1788881073.791137:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788881073.791138:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306709967 00000100:00000001:0.0:1788881073.791139:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788881073.791145:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306705039 00000800:00000200:1.0:1788881073.791149:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000010:0.0:1788881073.791152:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001a2ff840. 00000800:00000010:1.0:1788881073.791154:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e494c472. 00000800:00000200:3.0:1788881073.791155:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881073.791157:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306705039 00000400:00000200:1.0:1788881073.791160:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000800:00000010:3.0:1788881073.791162:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:0.0:1788881073.791167:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788881073.791168:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306709967 00000400:00000200:1.0:1788881073.791171:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000800:00000200:3.0:1788881073.791171:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788881073.791176:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306705039 00000400:00000200:1.0:1788881073.791177:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008c00e34d 00000800:00000200:3.0:1788881073.791179:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:1.0:1788881073.791182:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008c00e34d. 00000800:00000200:3.0:1788881073.791187:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788881073.791189:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306705039 00000100:00000010:0.0:1788881073.791190:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009a4dee8c. 00000100:00000001:1.0:1788881073.791190:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788881073.791191:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881073.791198:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881073.791199:0:201285:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788881073.791201:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306709967 00000100:00000040:3.0:1788881073.791202:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10882 usecs (11 jiffies) 00000100:00000001:0.0:1788881073.791207:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881073.791210:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306705039 00000100:00000001:0.0:1788881073.791214:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881073.791221:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306709967 00000100:00000001:0.0:1788881073.791222:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881073.791225:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881073.791229:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881073.791231:0:201287:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4306705039 00000100:00000040:1.0:1788881073.791257:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881073.791266:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881073.791268:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788881073.791272:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306704908 00000100:00000001:0.0:1788881073.791276:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881073.791277:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306709967 00000100:00000001:2.0:1788881073.791279:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881073.791280:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881073.791284:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306704908 00000800:00000200:1.0:1788881073.791288:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788881073.791289:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881073.791292:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:1.0:1788881073.791292:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c54a5c6f. 00000100:00000001:3.0:1788881073.791292:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306709967 00000100:00000001:0.0:1788881073.791296:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881073.791297:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881073.791297:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306704908 00000400:00000200:1.0:1788881073.791298:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881073.791302:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881073.791307:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881073.791307:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306709967 00000100:00000001:0.0:1788881073.791308:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881073.791314:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881073.791315:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306704908 00000100:00000001:3.0:1788881073.791319:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306709967 00000400:00000200:1.0:1788881073.791320:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000005509e00 00000100:00000001:3.0:1788881073.791322:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:1.0:1788881073.791323:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000005509e00. 00000100:00000001:3.0:1788881073.791325:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881073.791328:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881073.791330:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788881073.791331:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788881073.791334:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881073.791335:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881073.791337:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881073.791342:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788881073.791342:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881073.791345:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788881073.791347:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881089 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881073.791347:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881073.791348:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881073.791358:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881073.791359:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788881073.791391:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881073.791394:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:1.0:1788881073.791396:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881073.791400:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881073.791402:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000003539bb5a 00000400:00000010:1.0:1788881073.791404:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000003539bb5a. 00000100:00000001:1.0:1788881073.791406:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881073.791408:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881073.791414:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881073.791415:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881073.791418:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881073.791419:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788881073.792979:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881073.792986:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881073.792988:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881073.792989:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881073.792994:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788881073.793001:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12900 00000400:00000200:3.0:1788881073.793006:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x13d1 [1] + 224 00000800:00000001:3.0:1788881073.793034:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881073.793039:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881073.793040:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881073.793042:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881073.793045:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788881073.793052:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881073.793058:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881073.793068:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788881073.793072:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881073.793074:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881073.793099:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881073.793102:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788881073.793106:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881073.793108:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788881073.793110:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000400:00000200:0.0:1788881073.793113:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:0.0:1788881073.793115:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881073.793118:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788881073.793121:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881073.793129:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881073.793136:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881073.793138:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881073.793140:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881073.793143:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881073.793145:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000800:00000001:3.0:1788881073.793147:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881073.793148:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881073.793152:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788881073.793152:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881073.793155:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881073.793157:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788881073.793161:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=141 v=5 (5 5 5 5) 00000400:00000200:3.0:1788881073.793162:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00001000:0.0:1788881073.793168:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=141 v=5 (1 1 1 1) 00000400:00000200:3.0:1788881073.793170:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b12880 00000100:00000001:0.0:1788881073.793180:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881073.793182:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881073.793184:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788881073.793187:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881073.793191:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788881073.793192:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881073.793194:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881073.793195:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881073.793199:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881073.793206:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881073.793208:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881073.793209:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881073.793213:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881073.793219:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x13d5 [1] + 224 00000100:00100000:0.0:1788881073.793220:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002dcbd5d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780708096:192.168.204.111@tcp:400: 00000800:00000001:3.0:1788881073.793227:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881073.793228:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881073.793232:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:3.0:1788881073.793249:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881073.793258:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002dcbd5d0 x1875776780708096/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881073.793265:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881073.793267:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881073.793270:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000c9708566. 02000000:00000001:0.0:1788881073.793273:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788881073.793276:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881073.793277:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881073.793280:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881073.793282:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 02000000:00000001:0.0:1788881073.793286:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881073.793287:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881073.793288:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002dcbd5d0. 00000100:00000001:0.0:1788881073.793291:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881073.793292:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881073.793299:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881073.793303:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881073.793304:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881073.793307:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788881073.793307:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881073.793310:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881073.793314:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881073.793316:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788881073.793319:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881073.793325:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881073.793335:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788881073.793338:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881073.793340:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881073.793341:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881073.793342:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881073.793345:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788881073.793349:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12980 00000400:00000200:3.0:1788881073.793352:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x13e1 [1] + 224 00000800:00000001:3.0:1788881073.793360:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881073.793364:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881073.793365:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788881073.793372:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788881073.793372:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881073.793379:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881073.793382:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000001:2.0:1788881073.793386:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788881073.793388:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788881073.793390:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000400:00000200:2.0:1788881073.793394:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788881073.793396:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:2.0:1788881073.793399:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881073.793404:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881073.793413:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788881073.793417:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881073.793439:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881073.793443:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881073.793450:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788881073.793453:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788881073.793455:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788881073.793457:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881073.793459:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881073.793461:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788881073.793466:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=151 v=5 (5 5 5 5) 00000100:00001000:2.0:1788881073.793477:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=151 v=5 (1 1 1 1) 00000100:00000001:2.0:1788881073.793481:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788881073.793483:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788881073.793484:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:3.0:1788881073.793489:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:2.0:1788881073.793495:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881073.793498:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788881073.793498:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:2.0:1788881073.793503:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788881073.793505:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881073.793516:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881073.793519:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881073.793521:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881073.793522:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788881073.793529:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881073.793531:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00100000:2.0:1788881073.793539:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000008b25f66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780707968:192.168.204.111@tcp:400: 00000400:00000001:0.0:1788881073.793540:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881073.793544:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:2.0:1788881073.793545:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881073.793553:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000010:0.0:1788881073.793554:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004f63ce78. 00000100:00000040:2.0:1788881073.793559:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000008b25f66 x1875776780707968/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881073.793562:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000400:00000010:0.0:1788881073.793570:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000001:2.0:1788881073.793572:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788881073.793577:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788881073.793579:0:201286:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000010:2.0:1788881073.793580:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000001363d535. 02000000:00000001:2.0:1788881073.793586:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788881073.793589:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000200:0.0:1788881073.793591:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:2.0:1788881073.793592:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788881073.793598:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788881073.793602:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f0a0057b. 02000000:00000001:2.0:1788881073.793611:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788881073.793613:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788881073.793618:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000008b25f66. 00000100:00000200:0.0:1788881073.793624:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881073.793625:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788881073.793627:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881073.793634:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881073.793645:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881073.793647:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881073.793648:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881073.793653:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881073.793654:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881073.793655:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881073.793663:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881073.793666:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881073.793668:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881073.793672:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881073.793676:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881073.793680:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=151 v=5 (5 5 5 5) 00000100:00001000:0.0:1788881073.793684:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=85 v=5 (1 1 0 0) 00000100:00000001:0.0:1788881073.793687:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881073.793688:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881073.793690:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788881073.793693:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788881073.793699:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881073.793701:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788881073.793702:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881073.793704:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881073.793712:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881073.793716:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881073.793721:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881073.793723:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881073.793724:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881073.793727:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881073.793733:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000070db75b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780708224:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881073.793737:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881073.793739:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881073.793741:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000070db75b2 x1875776780708224/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881089 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881073.793746:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881073.793748:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881073.793750:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b405d9c4. 02000000:00000001:0.0:1788881073.793753:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788881073.793754:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881073.793755:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881073.793758:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881073.793759:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000a371c7dc. 02000000:00000001:0.0:1788881073.793762:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881073.793763:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881073.793764:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000070db75b2. 00000100:00000001:0.0:1788881073.793766:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881073.793767:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881073.793769:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881073.793773:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881073.793784:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881073.793786:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881073.793788:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881073.854347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881073.854354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881073.854362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881073.854363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881073.854423:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881073.854426:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881073.854429:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881074.878346:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881074.878351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881074.878355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881074.878356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881074.878405:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881074.878408:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881074.878424:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881075.902394:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881075.902402:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881075.902410:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881075.902412:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881075.902469:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881075.902478:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881075.902483:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881076.926360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881076.926368:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881076.926370:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881076.926373:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881076.926376:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881076.926379:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881076.926382:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881076.926383:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881076.926386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881076.926386:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881076.926447:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881076.926452:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881076.926455:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881077.950361:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881077.950368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881077.950375:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881077.950376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881077.950579:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881077.950583:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881077.950585:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788881078.910376:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306709967, at 4306710016 00000100:00000040:3.0:1788881078.910376:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306709967, at 4306710016 00000100:00000040:1.0:1788881078.910377:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306709967, at 4306710016 00000100:00000010:2.0:1788881078.910402:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001a2ff840. 00000100:00000010:1.0:1788881078.910406:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000004e64ede4. 00000100:00000010:3.0:1788881078.910408:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009a4dee8c. 00000100:00000040:3.0:1788881078.910430:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788881078.910434:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306709967, at 4306710016 00000100:00000001:3.0:1788881078.910436:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881078.910443:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002a42fbe1. 00000100:00000010:0.0:1788881078.910447:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000085563e6f. 00000020:00000040:3.0:1788881078.910449:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788881078.910455:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881078.910458:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881078.910463:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881078.910470:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00000001:3.0:1788881078.910478:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881078.910554:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788881078.910562:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881078.910564:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881078.910569:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002a42fbe1] to pc [ptlrpcd_00_02+2] req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881078.910587:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881078.910589:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881078.910592:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881078.910595:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000028715c64. 00000020:00000040:3.0:1788881078.910597:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881078.910599:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881078.910600:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881078.910602:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881078.910605:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:3.0:1788881078.910605:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:1.0:1788881078.910610:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881078.910611:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881078.910613:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881078.910614:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881078.910619:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788881078.910620:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:2.0:1788881078.910630:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881078.910637:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881078.910642:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881078.910644:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881078.910651:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881078.910655:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000028715c64] to pc [ptlrpcd_00_03+3] req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881078.910656:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788881078.910662:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788881078.910672:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881078.910689:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788881078.910694:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780708352:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788881078.910694:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881078.910703:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881078.910709:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881078.910709:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1788881078.910712:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788881078.910713:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000010:3.0:1788881078.910713:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca4fc672. 00000020:00000040:3.0:1788881078.910718:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:1.0:1788881078.910721:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881078.910722:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881078.910723:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788881078.910726:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881078.910730:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881078.910730:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881078.910732:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:2.0:1788881078.910738:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788881078.910739:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881078.910740:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000d5454cc1. 02000000:00000001:1.0:1788881078.910745:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788881078.910745:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881078.910757:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000003deec3bc. 02000000:00000001:1.0:1788881078.910758:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:3.0:1788881078.910765:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788881078.910767:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881078.910768:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788881078.910781:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780708480:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881078.910784:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:2.0:1788881078.910784:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 02000000:00000001:1.0:1788881078.910794:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:2.0:1788881078.910796:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 02000000:00000001:1.0:1788881078.910797:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881078.910803:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:2.0:1788881078.910807:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780708352, portal 25 02000000:00000010:1.0:1788881078.910808:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008e3e56e7. 00000100:00000001:2.0:1788881078.910813:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788881078.910815:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643432704 : -126823066118912 : ffff8ca7b431d500) 02000000:00000001:1.0:1788881078.910820:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881078.910823:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881094 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788881078.910833:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000007389ff7. 00000400:00000010:1.0:1788881078.910844:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000100:00000200:1.0:1788881078.910859:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780708480, portal 10 00000100:00000001:1.0:1788881078.910867:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881078.910871:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438080 : -126823066113536 : ffff8ca7b431ea00) 00000100:00000040:1.0:1788881078.910887:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881078.910893:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:3.0:1788881078.910894:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881078.910899:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881078.910907:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881078.910914:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788881078.910914:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca4fc672] to pc [ptlrpcd_00_00+0] req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788881078.910916:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001c5b8d0f. 00000100:00000040:1.0:1788881078.910926:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788881078.910927:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780708352, offset 0 00000400:00000010:1.0:1788881078.910931:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000100:00000200:1.0:1788881078.910940:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780708480, offset 0 00000400:00000200:2.0:1788881078.910947:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881078.910949:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881078.910952:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788881078.910955:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:1.0:1788881078.910959:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788881078.910962:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12047) 00000400:00000200:2.0:1788881078.910968:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788881078.910973:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788881078.910976:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788881078.910981:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788881078.910984:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881078.910986:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:2.0:1788881078.910986:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:0.0:1788881078.910992:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881078.910999:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788881078.911002:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788881078.911005:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881078.911015:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [612:612:256:4294967295] 192.168.204.111@tcp LPNI seq info [612:612:8:4294967295] 00000100:00000001:0.0:1788881078.911030:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881078.911032:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881078.911036:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788881078.911042:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788881078.911050:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788881078.911052:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881078.911059:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780708608:192.168.204.111@tcp:400: 00000800:00000200:2.0:1788881078.911061:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788881078.911066:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788881078.911075:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000010:2.0:1788881078.911076:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000045330cf1. 02000000:00000001:0.0:1788881078.911079:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1788881078.911079:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:0.0:1788881078.911083:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881078.911089:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788881078.911092:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:0.0:1788881078.911093:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:1.0:1788881078.911100:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000010:0.0:1788881078.911101:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000ee42a892. 02000000:00000001:0.0:1788881078.911104:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881078.911107:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:2.0:1788881078.911113:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000010:0.0:1788881078.911115:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000400:00000200:1.0:1788881078.911122:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [613:613:255:4294967295] 192.168.204.111@tcp LPNI seq info [613:613:7:4294967295] 00000800:00000200:2.0:1788881078.911124:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000045330cf1 type 1, nob 320 niov 1 nkiov 1 00000400:00000010:0.0:1788881078.911126:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000200:0.0:1788881078.911135:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780708608, portal 10 00000100:00000001:0.0:1788881078.911140:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881078.911141:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643437184 : -126823066114432 : ffff8ca7b431e680) 00000100:00000040:0.0:1788881078.911147:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881078.911151:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788881078.911153:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881078.911165:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306715087 00000100:00000001:0.0:1788881078.911171:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:1.0:1788881078.911174:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788881078.911178:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788881078.911178:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000488d1158. 00000800:00000010:1.0:1788881078.911179:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000004e0f6846. 00000400:00000010:0.0:1788881078.911188:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000fbbca8a. 00000100:00000001:2.0:1788881078.911189:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000800:00000200:1.0:1788881078.911193:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:0.0:1788881078.911202:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780708608, offset 0 00000100:00000001:2.0:1788881078.911203:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306715087 00000800:00000200:1.0:1788881078.911208:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788881078.911211:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788881078.911212:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb1a14f4. 00000800:00000200:1.0:1788881078.911215:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000004e0f6846 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788881078.911219:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1788881078.911222:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881078.911223:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:3.0:1788881078.911224:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:0.0:1788881078.911254:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788881078.911257:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788881078.911258:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306715088 00000800:00000010:3.0:1788881078.911261:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000045330cf1. 00000400:00000200:0.0:1788881078.911262:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000040:1.0:1788881078.911262:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 141401 usecs (142 jiffies) 00000100:00000010:2.0:1788881078.911269:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a0959d87. 00000400:00000200:0.0:1788881078.911270:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:1.0:1788881078.911271:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306710159 00000400:00000200:3.0:1788881078.911272:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788881078.911280:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [614:614:254:4294967295] 192.168.204.111@tcp LPNI seq info [614:614:6:4294967295] 00000100:00000001:1.0:1788881078.911281:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306715087 00000100:00000001:2.0:1788881078.911281:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:1.0:1788881078.911291:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306710159 00000100:00000001:2.0:1788881078.911294:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306715088 00000100:00000001:1.0:1788881078.911295:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306715087 00000400:00000200:0.0:1788881078.911300:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788881078.911306:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306710159 00000100:00000010:2.0:1788881078.911306:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b63100a0. 00000800:00000200:0.0:1788881078.911313:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881078.911318:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306715088 00000800:00000010:0.0:1788881078.911319:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000050c5542b. 00000100:00000001:2.0:1788881078.911319:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:3.0:1788881078.911327:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881078.911329:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306710159 00000100:00000001:2.0:1788881078.911331:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881078.911332:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306715088 00000400:00000200:3.0:1788881078.911335:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001c5b8d0f 00000800:00000200:0.0:1788881078.911339:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788881078.911342:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881078.911343:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1788881078.911344:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000001c5b8d0f. 00000800:00000200:0.0:1788881078.911354:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788881078.911354:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881078.911356:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881078.911357:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788881078.911360:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:0.0:1788881078.911362:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000050c5542b type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788881078.911364:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788881078.911367:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881094 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881078.911371:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881078.911371:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881078.911372:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881078.911376:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:0.0:1788881078.911380:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10843 usecs (11 jiffies) 00000100:00000001:1.0:1788881078.911385:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881078.911387:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881078.911390:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306710028 00000100:00000001:1.0:1788881078.911393:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881078.911393:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788881078.911394:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881078.911396:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306715087 00000100:00000001:2.0:1788881078.911401:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881078.911403:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306710028 00000100:00000001:1.0:1788881078.911403:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788881078.911406:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881094 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881078.911409:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881078.911410:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306715087 00000100:00000001:1.0:1788881078.911410:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881078.911414:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881078.911416:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306710028 00000100:00000001:1.0:1788881078.911421:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881078.911426:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306715088 00000100:00000001:0.0:1788881078.911432:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306710028 00000100:00000001:3.0:1788881078.911432:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881078.911435:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306715088 00000100:00000001:3.0:1788881078.911436:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788881078.911440:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881078.911443:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881078.911447:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881078.911450:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:3.0:1788881078.911460:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788881078.911461:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881078.911464:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:3.0:1788881078.911464:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000004e0f6846. 00000100:00000001:0.0:1788881078.911467:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881078.911470:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788881078.911470:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881078.911472:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881078.911474:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881078.911480:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881078.911483:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:3.0:1788881078.911484:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:3.0:1788881078.911486:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881078.911488:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881078.911494:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881078.911495:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881078.911499:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881078.911500:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788881078.911524:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788881078.911526:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000050c5542b. 00000400:00000200:3.0:1788881078.911528:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881078.911530:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881078.911532:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000fbbca8a 00000400:00000010:3.0:1788881078.911532:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000fbbca8a. 00000100:00000001:3.0:1788881078.911534:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881078.911536:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881078.911539:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881078.911541:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881078.911544:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881078.911545:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788881078.912460:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881078.912469:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881078.912472:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881078.912474:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881078.912479:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788881078.912485:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b12a00 00000400:00000200:3.0:1788881078.912517:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x13e9 [1] + 224 00000800:00000001:3.0:1788881078.912527:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881078.912531:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881078.912532:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881078.912534:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881078.912536:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788881078.912539:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881078.912544:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881078.912558:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881078.912599:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881078.912603:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788881078.912608:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000800:00000001:3.0:1788881078.912608:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:2.0:1788881078.912611:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:3.0:1788881078.912613:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:2.0:1788881078.912615:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000800:00000001:3.0:1788881078.912616:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881078.912618:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881078.912621:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788881078.912624:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000400:00000200:3.0:1788881078.912625:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788881078.912629:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881078.912636:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881078.912638:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12a80 00000400:00000200:3.0:1788881078.912650:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x13ed [1] + 224 00000100:00000200:2.0:1788881078.912653:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788881078.912656:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788881078.912662:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881078.912663:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881078.912666:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788881078.912668:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788881078.912671:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788881078.912673:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881078.912675:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788881078.912680:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=6 v=5 (5 5 5 5) 00000100:00001000:2.0:1788881078.912684:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=6 v=5 (1 1 1 1) 00000100:00000001:2.0:1788881078.912686:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788881078.912687:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788881078.912690:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788881078.912692:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881078.912694:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881078.912698:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881078.912705:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881078.912709:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881078.912711:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881078.912711:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881078.912717:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:2.0:1788881078.912718:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881078.912725:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881078.912733:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:2.0:1788881078.912737:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002a42fbe1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780708352:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881078.912762:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881078.912764:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881078.912768:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002a42fbe1 x1875776780708352/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881078.912774:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788881078.912776:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788881078.912779:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000003deec3bc. 02000000:00000001:2.0:1788881078.912782:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788881078.912784:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788881078.912786:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788881078.912788:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788881078.912789:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 02000000:00000001:2.0:1788881078.912794:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788881078.912795:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788881078.912797:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002a42fbe1. 00000100:00000001:2.0:1788881078.912800:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788881078.912801:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881078.912803:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788881078.912803:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881078.912809:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1788881078.912815:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881078.912821:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881078.912828:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881078.912829:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881078.912832:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788881078.912838:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881078.912843:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881078.912844:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881078.912846:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881078.912867:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881078.912870:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788881078.912873:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881078.912875:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788881078.912876:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000007389ff7. 00000400:00000200:1.0:1788881078.912881:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000400:00000010:1.0:1788881078.912882:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00000100:00000001:1.0:1788881078.912885:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881078.912892:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881078.912894:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000200:1.0:1788881078.912906:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881078.912906:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12b00 00000400:00000200:3.0:1788881078.912916:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x13f9 [1] + 224 00000100:00000001:1.0:1788881078.912918:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881078.912922:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881078.912923:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788881078.912924:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881078.912927:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881078.912928:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788881078.912930:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788881078.912930:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788881078.912935:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881078.912936:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00001000:1.0:1788881078.912940:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=146 v=5 (5 5 5 5) 00000400:00000200:3.0:1788881078.912942:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00001000:1.0:1788881078.912946:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=146 v=5 (1 1 1 1) 00000100:00000001:3.0:1788881078.912948:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788881078.912949:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881078.912952:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788881078.912954:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788881078.912955:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788881078.912959:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881078.912964:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788881078.912968:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000040:3.0:1788881078.912969:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881078.912971:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788881078.912974:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881078.912981:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881078.912989:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881078.912991:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881078.912992:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881078.912995:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881078.912997:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:1.0:1788881078.912998:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788881078.913001:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1788881078.913010:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000028715c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780708480:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881078.913016:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881078.913017:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881078.913020:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000028715c64 x1875776780708480/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881078.913024:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881078.913026:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788881078.913027:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788881078.913029:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788881078.913032:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008e3e56e7. 00000400:00000001:0.0:1788881078.913033:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881078.913036:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:1.0:1788881078.913037:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000010:0.0:1788881078.913038:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000020:00000001:1.0:1788881078.913040:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000200:0.0:1788881078.913042:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000020:00000040:1.0:1788881078.913043:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000010:0.0:1788881078.913044:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881078.913049:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000020:00000001:1.0:1788881078.913049:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1788881078.913051:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 00000100:00000200:0.0:1788881078.913055:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881078.913056:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788881078.913059:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881078.913061:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000028715c64. 00000100:00000001:1.0:1788881078.913066:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000200:0.0:1788881078.913067:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881078.913069:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881078.913073:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881078.913078:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881078.913079:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881078.913079:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881078.913083:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788881078.913083:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881078.913085:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881078.913087:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881078.913088:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881078.913091:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881078.913092:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881078.913093:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881078.913098:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=6 v=5 (5 5 5 5) 00000100:00001000:0.0:1788881078.913099:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=90 v=5 (1 1 0 0) 00000100:00000001:0.0:1788881078.913101:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881078.913102:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881078.913103:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788881078.913106:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788881078.913111:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881078.913112:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788881078.913113:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881078.913114:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881078.913119:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881078.913122:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881078.913126:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881078.913127:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881078.913128:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881078.913129:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881078.913133:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca4fc672 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780708608:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881078.913136:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881078.913137:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881078.913138:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca4fc672 x1875776780708608/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881094 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881078.913141:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881078.913142:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881078.913143:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000ee42a892. 02000000:00000001:0.0:1788881078.913146:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788881078.913147:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881078.913147:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881078.913149:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881078.913150:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000d5454cc1. 02000000:00000001:0.0:1788881078.913151:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881078.913152:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881078.913153:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca4fc672. 00000100:00000001:0.0:1788881078.913154:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881078.913155:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881078.913156:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881078.913158:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881078.913159:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881078.913160:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881078.913161:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881078.974373:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881078.974380:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881078.974386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881078.974389:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881078.974391:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881078.974394:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881078.974398:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881078.974400:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881078.974402:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881078.974406:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881078.974449:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881078.974452:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881078.974470:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881079.998398:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881079.998410:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881079.998420:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881079.998421:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881079.998475:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881079.998479:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881079.998481:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881081.022364:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881081.022374:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881081.022382:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881081.022383:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881081.022439:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881081.022443:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881081.022446:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881082.046354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881082.046362:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881082.046368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881082.046369:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881082.046439:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881082.046444:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881082.046447:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881083.070314:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881083.070320:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881083.070325:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881083.070327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881083.070380:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881083.070384:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881083.070387:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788881084.030422:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306715088, at 4306715136 00000100:00000040:0.0:1788881084.030423:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306715087, at 4306715136 00000100:00000040:1.0:1788881084.030423:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788881084.030423:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306715088, at 4306715136 00000100:00000001:1.0:1788881084.030444:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881084.030450:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a0959d87. 00000100:00000010:3.0:1788881084.030453:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b63100a0. 00000100:00000010:1.0:1788881084.030457:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000001cf64abe. 00000100:00000010:0.0:1788881084.030461:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000488d1158. 00000020:00000040:1.0:1788881084.030463:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788881084.030470:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881084.030474:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881084.030481:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881084.030493:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000004e0f6846. 00000100:00000001:1.0:1788881084.030502:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881084.030516:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881084.030532:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881084.030534:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881084.030539:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001cf64abe] to pc [ptlrpcd_00_01+1] req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881084.030561:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881084.030563:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881084.030567:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881084.030570:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006a9c2e02. 00000020:00000040:1.0:1788881084.030572:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881084.030575:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881084.030576:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881084.030578:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881084.030582:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000a371c7dc. 00000100:00000001:1.0:1788881084.030585:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881084.030588:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881084.030589:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788881084.030592:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788881084.030593:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:3.0:1788881084.030596:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881084.030597:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881084.030605:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881084.030606:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881084.030609:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788881084.030609:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006a9c2e02] to pc [ptlrpcd_00_02+2] req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881084.030612:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881084.030619:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881084.030622:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881084.030625:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788881084.030630:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881084.030637:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881084.030639:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881084.030641:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881084.030642:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788881084.030646:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788881084.030646:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881084.030660:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 02000000:00000001:0.0:1788881084.030662:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788881084.030665:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000dc99d6ba. 00000100:00100000:0.0:1788881084.030669:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780708736:192.168.204.111@tcp:400: 00000020:00000040:1.0:1788881084.030670:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881084.030676:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788881084.030678:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788881084.030679:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788881084.030681:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881084.030682:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788881084.030684:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788881084.030684:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 02000000:00000001:0.0:1788881084.030688:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788881084.030689:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788881084.030690:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f0a0057b. 00000100:00000001:2.0:1788881084.030690:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881084.030693:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881084.030696:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881084.030703:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881084.030704:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788881084.030707:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881084.030708:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881084.030710:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:0.0:1788881084.030712:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000826ae8f4. 00000100:00000001:3.0:1788881084.030713:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788881084.030715:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881084.030715:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:0.0:1788881084.030725:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000100:00000001:1.0:1788881084.030726:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881084.030730:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881084.030734:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000040:3.0:1788881084.030734:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881084.030735:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000dc99d6ba] to pc [ptlrpcd_00_03+3] req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881084.030742:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780708736, portal 25 00000100:00000001:0.0:1788881084.030747:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881084.030747:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881084.030750:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881084.030753:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788881084.030755:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881084.030756:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881084.030757:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938054144 : -126823771497472 : ffff8ca78a269c00) 00000100:00000040:1.0:1788881084.030758:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:3.0:1788881084.030761:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881084.030764:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1788881084.030766:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006a9c2e02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780708864:192.168.204.111@tcp:400: 00000100:00000040:0.0:1788881084.030767:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881100 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881084.030772:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12052) 00000100:00000001:2.0:1788881084.030775:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881084.030776:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00100000:2.0:1788881084.030783:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 02000000:00000001:3.0:1788881084.030786:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788881084.030787:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788881084.030788:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788881084.030791:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881084.030795:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788881084.030795:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306715087, at 4306715136 02000000:00000001:3.0:1788881084.030797:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788881084.030799:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000010:0.0:1788881084.030800:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f4d84419. 00000100:00000010:1.0:1788881084.030802:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb1a14f4. 02000000:00000010:3.0:1788881084.030802:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002ce43d73. 00000100:00000001:2.0:1788881084.030803:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788881084.030805:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780708736, offset 0 02000000:00000001:3.0:1788881084.030816:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881084.030817:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881084.030824:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788881084.030826:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881084.030828:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000010:3.0:1788881084.030829:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000006b419026. 00000400:00000010:3.0:1788881084.030839:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000100:00000040:2.0:1788881084.030841:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788881084.030850:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780708864, portal 10 00000400:00000200:0.0:1788881084.030852:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788881084.030862:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881084.030867:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938058624 : -126823771492992 : ffff8ca78a26ad80) 00000400:00000200:0.0:1788881084.030876:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:3.0:1788881084.030884:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881084.030888:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788881084.030895:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788881084.030900:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788881084.030905:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788881084.030915:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [615:615:256:4294967295] 192.168.204.111@tcp LPNI seq info [615:615:8:4294967295] 00000100:00000040:3.0:1788881084.030918:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788881084.030924:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 00000100:00000200:3.0:1788881084.030932:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780708864, offset 0 00000400:00000200:3.0:1788881084.030941:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881084.030943:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881084.030949:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881084.030955:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788881084.030959:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788881084.030960:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788881084.030969:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788881084.030977:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000dc99d6ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780708992:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881084.030984:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1788881084.030984:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:0.0:1788881084.030985:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788881084.030994:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:3.0:1788881084.030995:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:0.0:1788881084.030996:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000fc8f9b7c. 02000000:00000001:2.0:1788881084.031003:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881084.031007:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:2.0:1788881084.031009:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788881084.031014:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000010:2.0:1788881084.031015:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000115c4ce2. 00000800:00000200:0.0:1788881084.031016:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788881084.031022:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:2.0:1788881084.031024:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788881084.031036:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788881084.031037:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [616:616:255:4294967295] 192.168.204.111@tcp LPNI seq info [616:616:7:4294967295] 00000400:00000010:2.0:1788881084.031038:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 00000800:00000200:0.0:1788881084.031050:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000fc8f9b7c type 1, nob 320 niov 1 nkiov 1 00000400:00000010:2.0:1788881084.031053:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000200:2.0:1788881084.031064:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780708992, portal 10 00000100:00000001:2.0:1788881084.031070:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881084.031076:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881084.031076:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938052352 : -126823771499264 : ffff8ca78a269500) 00000400:00000200:3.0:1788881084.031077:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788881084.031093:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306720207 00000100:00000040:2.0:1788881084.031099:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788881084.031103:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788881084.031108:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005b8bebe2. 00000800:00000010:3.0:1788881084.031116:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000007660b3e. 00000100:00000001:0.0:1788881084.031124:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:2.0:1788881084.031128:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:3.0:1788881084.031132:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788881084.031134:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306720207 00000100:00000010:0.0:1788881084.031140:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014321469. 00000800:00000200:3.0:1788881084.031142:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788881084.031146:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:3.0:1788881084.031148:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000007660b3e type 1, nob 320 niov 1 nkiov 1 00000100:00000040:2.0:1788881084.031152:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881084.031156:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881084.031165:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c56cc487. 00000100:00000040:3.0:1788881084.031165:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 140652 usecs (141 jiffies) 00000100:00000001:0.0:1788881084.031167:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306720207 00000100:00000001:3.0:1788881084.031179:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306715277 00000100:00000010:0.0:1788881084.031180:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000e6f9f09c. 00000100:00000200:2.0:1788881084.031180:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780708992, offset 0 00000100:00000001:0.0:1788881084.031189:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788881084.031191:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306720207 00000100:00000001:3.0:1788881084.031195:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306715277 00000100:00000001:0.0:1788881084.031203:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306720207 00000400:00000200:2.0:1788881084.031204:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881084.031204:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306720207 00000100:00000010:0.0:1788881084.031212:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000085563e6f. 00000100:00000001:3.0:1788881084.031215:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306715277 00000100:00000001:0.0:1788881084.031220:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788881084.031220:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306720207 00000800:00000200:1.0:1788881084.031222:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788881084.031226:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306715277 00000800:00000010:1.0:1788881084.031227:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000fc8f9b7c. 00000100:00000001:0.0:1788881084.031228:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881084.031263:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306720207 00000400:00000200:2.0:1788881084.031265:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788881084.031266:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881084.031268:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881084.031269:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881084.031279:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881084.031279:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881084.031281:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881084.031285:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788881084.031286:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881084.031289:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881084.031291:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881084.031296:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788881084.031297:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788881084.031299:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881084.031304:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881084.031308:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788881084.031313:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788881084.031317:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [617:617:254:4294967295] 192.168.204.111@tcp LPNI seq info [617:617:6:4294967295] 00000400:00000200:2.0:1788881084.031328:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:1.0:1788881084.031343:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f4d84419 00000800:00000200:2.0:1788881084.031343:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881084.031349:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f4d84419. 00000800:00000010:2.0:1788881084.031350:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000009b8e844. 00000100:00000001:1.0:1788881084.031355:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000800:00000200:2.0:1788881084.031361:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000200:1.0:1788881084.031364:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881100 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881084.031364:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881084.031375:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788881084.031377:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000200:2.0:1788881084.031386:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000009b8e844 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:1.0:1788881084.031391:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881100 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881084.031393:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881084.031404:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10798 usecs (11 jiffies) 00000100:00000001:2.0:1788881084.031414:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306715148 00000100:00000001:1.0:1788881084.031419:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881084.031421:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306720207 00000100:00000001:1.0:1788881084.031423:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788881084.031428:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306715148 00000100:00000001:2.0:1788881084.031433:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306720207 00000100:00000001:2.0:1788881084.031435:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306715148 00000100:00000001:2.0:1788881084.031438:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306720207 00000100:00000001:2.0:1788881084.031440:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306715148 00000100:00000001:2.0:1788881084.031442:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306720207 00000100:00000001:2.0:1788881084.031445:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788881084.031447:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881084.031451:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000007660b3e. 00000100:00000001:2.0:1788881084.031451:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881084.031457:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788881084.031460:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881084.031465:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881084.031471:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 00000100:00000001:2.0:1788881084.031471:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881084.031475:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 00000100:00000001:1.0:1788881084.031483:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881084.031489:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881084.031494:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881084.031496:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881084.031500:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881084.031502:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788881084.031527:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881084.031529:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000009b8e844. 00000400:00000200:1.0:1788881084.031533:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881084.031536:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881084.031539:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c56cc487 00000400:00000010:1.0:1788881084.031540:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c56cc487. 00000100:00000001:1.0:1788881084.031543:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881084.031546:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881084.031553:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881084.031555:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881084.031560:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881084.031562:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788881084.032583:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881084.032596:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881084.032600:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881084.032603:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881084.032610:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788881084.032679:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b12b80 00000400:00000200:3.0:1788881084.032691:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1401 [1] + 224 00000800:00000001:3.0:1788881084.032697:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881084.032703:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881084.032705:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881084.032708:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881084.032713:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788881084.032717:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881084.032725:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881084.032802:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788881084.032810:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881084.032814:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881084.032816:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881084.032818:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881084.032824:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:3.0:1788881084.032831:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12c00 00000100:00000001:0.0:1788881084.032832:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881084.032836:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1788881084.032873:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1409 [1] + 224 00000800:00000001:3.0:1788881084.032879:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:0.0:1788881084.032880:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881084.032883:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000800:00000001:3.0:1788881084.032884:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788881084.032886:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000400:00000200:0.0:1788881084.032891:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:0.0:1788881084.032893:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881084.032896:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:3.0:1788881084.032897:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:0.0:1788881084.032901:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881084.032904:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881084.032911:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788881084.032918:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788881084.032919:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881084.032940:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881084.032944:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881084.032944:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881084.032950:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881084.032954:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881084.032958:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881084.032960:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881084.032963:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881084.032967:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00001000:0.0:1788881084.032969:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=12 v=5 (5 5 5 5) 00000800:00000001:3.0:1788881084.032974:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881084.032978:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788881084.032986:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788881084.032988:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881084.032993:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788881084.032995:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788881084.032996:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788881084.033001:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00001000:0.0:1788881084.033003:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=12 v=5 (1 1 1 1) 00000400:00000001:1.0:1788881084.033003:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788881084.033006:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000006b419026. 00000400:00000200:3.0:1788881084.033009:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12c80 00000100:00000001:0.0:1788881084.033010:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:1.0:1788881084.033011:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000100:00000001:0.0:1788881084.033014:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881084.033015:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00010000:00000001:0.0:1788881084.033018:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788881084.033019:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:0.0:1788881084.033022:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881084.033026:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881084.033027:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1411 [1] + 224 00000100:00000001:0.0:1788881084.033028:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788881084.033037:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:0.0:1788881084.033038:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881084.033045:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1788881084.033046:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881084.033051:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881084.033060:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881084.033066:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788881084.033069:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881084.033073:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788881084.033074:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881084.033079:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788881084.033080:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788881084.033084:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881084.033088:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881084.033088:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881084.033092:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881084.033093:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881084.033093:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881084.033099:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881084.033102:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:1.0:1788881084.033107:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=152 v=5 (5 5 5 5) 00000100:00001000:1.0:1788881084.033115:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=152 v=5 (1 1 1 1) 00000100:00000001:1.0:1788881084.033122:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881084.033125:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881084.033125:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:0.0:1788881084.033127:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000001cf64abe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780708736:192.168.204.111@tcp:400: 00010000:00000001:1.0:1788881084.033128:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788881084.033133:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788881084.033133:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881084.033135:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881084.033135:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881084.033139:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788881084.033140:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788881084.033140:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881084.033141:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881084.033141:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788881084.033146:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000040:0.0:1788881084.033149:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000001cf64abe x1875776780708736/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881084.033149:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788881084.033149:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788881084.033155:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:1.0:1788881084.033158:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788881084.033159:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000100:00000001:0.0:1788881084.033167:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000200:2.0:1788881084.033167:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000100:00000001:1.0:1788881084.033170:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:2.0:1788881084.033172:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 02000000:00000001:0.0:1788881084.033173:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788881084.033173:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788881084.033177:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788881084.033178:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000826ae8f4. 00000100:00000001:2.0:1788881084.033179:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:1.0:1788881084.033180:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881084.033186:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:2.0:1788881084.033187:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788881084.033190:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881084.033193:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00100000:1.0:1788881084.033198:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006a9c2e02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780708864:192.168.204.111@tcp:400: 00000020:00000001:0.0:1788881084.033199:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881084.033203:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000004e0f6846. 00000100:00000001:1.0:1788881084.033207:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881084.033210:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788881084.033212:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:0.0:1788881084.033214:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000200:2.0:1788881084.033217:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:0.0:1788881084.033218:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000001cf64abe. 00000100:00000040:1.0:1788881084.033218:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006a9c2e02 x1875776780708864/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881084.033224:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881084.033226:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881084.033229:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881084.033265:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881084.033269:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881084.033271:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788881084.033273:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881084.033273:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:1.0:1788881084.033274:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788881084.033277:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002ce43d73. 00000100:00000001:0.0:1788881084.033278:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881084.033278:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:1.0:1788881084.033281:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:2.0:1788881084.033284:0:201287:0:(client.c:1580:after_reply()) Process entered 00000020:00000001:1.0:1788881084.033285:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788881084.033288:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:2.0:1788881084.033289:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788881084.033292:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000001:1.0:1788881084.033293:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788881084.033296:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881084.033297:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000a371c7dc. 02000000:00000001:2.0:1788881084.033301:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881084.033302:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788881084.033305:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881084.033306:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006a9c2e02. 00000100:00001000:2.0:1788881084.033307:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=12 v=5 (5 5 5 5) 00000100:00000001:1.0:1788881084.033309:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788881084.033313:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788881084.033314:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=96 v=5 (1 1 0 0) 00000100:00000001:1.0:1788881084.033316:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881084.033321:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881084.033324:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881084.033324:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881084.033327:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788881084.033328:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788881084.033333:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00080000:2.0:1788881084.033336:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881084.033337:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788881084.033347:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881084.033350:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788881084.033351:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788881084.033354:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788881084.033355:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881084.033358:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881084.033364:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881084.033366:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881084.033368:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788881084.033370:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788881084.033378:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000dc99d6ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780708992:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881084.033383:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881084.033385:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881084.033387:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000dc99d6ba x1875776780708992/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881100 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881084.033404:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788881084.033406:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788881084.033407:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000115c4ce2. 02000000:00000001:2.0:1788881084.033411:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788881084.033413:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788881084.033414:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788881084.033416:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788881084.033418:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f0a0057b. 02000000:00000001:2.0:1788881084.033421:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788881084.033422:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788881084.033423:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000dc99d6ba. 00000100:00000001:2.0:1788881084.033426:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788881084.033427:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881084.033429:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881084.033433:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881084.033434:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881084.033436:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881084.033438:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881084.094357:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881084.094366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881084.094372:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881084.094373:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881084.094434:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881084.094440:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881084.094444:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881085.118363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881085.118370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881085.118375:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881085.118376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881085.118427:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881085.118431:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881085.118434:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881086.142324:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881086.142330:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881086.142334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881086.142334:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881086.142375:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881086.142379:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881086.142382:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881087.166367:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881087.166377:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881087.166379:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881087.166382:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881087.166386:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881087.166388:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881087.166391:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881087.166392:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881087.166395:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881087.166396:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881087.166452:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881087.166457:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881087.166482:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881088.190350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881088.190357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881088.190363:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881088.190364:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881088.190423:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881088.190428:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881088.190433:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881089.150372:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306720207, at 4306720256 00000100:00000040:2.0:1788881089.150372:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306720207, at 4306720256 00000100:00000040:3.0:1788881089.150373:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306720207, at 4306720256 00000100:00000040:1.0:1788881089.150377:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881089.150390:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788881089.150393:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005b8bebe2. 00000100:00000010:3.0:1788881089.150394:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000085563e6f. 00000100:00000010:2.0:1788881089.150397:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000e6f9f09c. 00000100:00000010:1.0:1788881089.150402:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000006a9c2e02. 00000020:00000040:1.0:1788881089.150409:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788881089.150416:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881089.150420:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881089.150426:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881089.150435:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b016e4f4. 00000100:00000001:1.0:1788881089.150444:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881089.150457:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881089.150468:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881089.150470:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881089.150477:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006a9c2e02] to pc [ptlrpcd_00_00+0] req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881089.150506:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881089.150509:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881089.150513:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881089.150516:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000003b122924. 00000100:00000001:0.0:1788881089.150517:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:1.0:1788881089.150519:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881089.150522:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881089.150523:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788881089.150526:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788881089.150532:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881089.150532:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881089.150536:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881089.150537:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788881089.150544:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000030024c2c. 00000100:00000001:3.0:1788881089.150544:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881089.150545:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881089.150548:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881089.150551:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881089.150560:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788881089.150564:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788881089.150569:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:0.0:1788881089.150570:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881089.150578:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000006a9c2e02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780709120:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881089.150583:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788881089.150586:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788881089.150587:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881089.150593:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003b122924] to pc [ptlrpcd_00_01+1] req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881089.150595:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881089.150599:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881089.150604:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788881089.150607:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788881089.150610:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000007e76baee. 00000100:00000040:1.0:1788881089.150611:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788881089.150615:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881089.150617:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881089.150621:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000b5bd1bf6. 00000400:00000010:0.0:1788881089.150625:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000020:00000040:1.0:1788881089.150626:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881089.150629:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:3.0:1788881089.150630:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788881089.150632:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788881089.150634:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:1.0:1788881089.150636:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881089.150637:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000001:3.0:1788881089.150638:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000010:1.0:1788881089.150642:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000a371c7dc. 00000100:00000001:1.0:1788881089.150646:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881089.150647:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780709120, portal 25 00000100:00000040:1.0:1788881089.150650:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881089.150657:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881089.150659:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881089.150662:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000040:1.0:1788881089.150664:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b5bd1bf6] to pc [ptlrpcd_00_02+2] req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881089.150666:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938058624 : -126823771492992 : ffff8ca78a26ad80) 00000100:00000040:3.0:1788881089.150672:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788881089.150678:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881105 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881089.150695:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881089.150697:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881089.150701:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788881089.150704:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881089.150705:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788881089.150706:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881089.150707:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881089.150712:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788881089.150713:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00100000:3.0:1788881089.150713:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780709248:192.168.204.111@tcp:400: 00000400:00000010:0.0:1788881089.150719:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000dc289a80. 00000100:00000001:3.0:1788881089.150721:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788881089.150724:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:3.0:1788881089.150725:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000200:0.0:1788881089.150726:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780709120, offset 0 02000000:00000001:3.0:1788881089.150729:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881089.150731:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12057) 00000100:00000001:2.0:1788881089.150731:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788881089.150731:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788881089.150735:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:3.0:1788881089.150737:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002ce43d73. 00000400:00000200:0.0:1788881089.150738:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881089.150739:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1788881089.150741:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881089.150745:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881089.150746:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:3.0:1788881089.150746:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ca22282b. 00000100:00000001:1.0:1788881089.150751:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881089.150757:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:3.0:1788881089.150758:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000400:00000200:0.0:1788881089.150760:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788881089.150761:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788881089.150762:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780709248, portal 10 00000100:00000040:1.0:1788881089.150763:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306720207, at 4306720256 02000000:00000001:2.0:1788881089.150764:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788881089.150765:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:2.0:1788881089.150767:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881089.150768:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938062208 : -126823771489408 : ffff8ca78a26bb80) 02000000:00000001:2.0:1788881089.150773:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788881089.150774:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000014321469. 00000100:00100000:2.0:1788881089.150779:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000b5bd1bf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780709376:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788881089.150780:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881089.150781:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788881089.150788:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788881089.150792:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:2.0:1788881089.150792:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788881089.150796:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788881089.150801:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:2.0:1788881089.150801:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881089.150803:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788881089.150807:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:3.0:1788881089.150809:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b6d0d6b5. 00000400:00000200:0.0:1788881089.150810:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000010:2.0:1788881089.150811:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e5dc9915. 02000000:00000001:2.0:1788881089.150815:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788881089.150816:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780709248, offset 0 00000400:00000200:0.0:1788881089.150820:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [618:618:256:4294967295] 192.168.204.111@tcp LPNI seq info [618:618:8:4294967295] 00000400:00000010:2.0:1788881089.150820:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 00000400:00000200:3.0:1788881089.150828:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788881089.150834:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000200:2.0:1788881089.150839:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780709376, portal 10 00000100:00000001:2.0:1788881089.150843:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788881089.150844:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788881089.150845:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938059520 : -126823771492096 : ffff8ca78a26b100) 00000100:00000040:2.0:1788881089.150853:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881089.150858:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:0.0:1788881089.150859:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788881089.150867:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788881089.150868:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000010:0.0:1788881089.150869:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000bd68c00a. 00000400:00000200:3.0:1788881089.150872:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:2.0:1788881089.150874:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788881089.150877:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000008f790e4a. 00000100:00000200:2.0:1788881089.150879:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780709376, offset 0 00000800:00000200:0.0:1788881089.150883:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788881089.150883:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788881089.150889:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788881089.150890:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788881089.150895:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [619:619:255:4294967295] 192.168.204.111@tcp LPNI seq info [619:619:7:4294967295] 00000800:00000200:0.0:1788881089.150899:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788881089.150905:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000bd68c00a type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788881089.150918:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788881089.150923:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881089.150924:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788881089.150926:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 00000400:00000200:2.0:1788881089.150927:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:0.0:1788881089.150930:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306725327 00000400:00000200:2.0:1788881089.150934:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:3.0:1788881089.150936:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000010:0.0:1788881089.150937:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000005b8bebe2. 00000400:00000200:2.0:1788881089.150940:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788881089.150945:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:2.0:1788881089.150945:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788881089.150945:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:2.0:1788881089.150951:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788881089.150952:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306725327 00000800:00000200:3.0:1788881089.150952:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788881089.150957:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [620:620:254:4294967295] 192.168.204.111@tcp LPNI seq info [620:620:6:4294967295] 00000100:00000001:3.0:1788881089.150958:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788881089.150960:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000009ac8d70b. 00000100:00000040:3.0:1788881089.150965:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 139912 usecs (140 jiffies) 00000100:00000001:0.0:1788881089.150967:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788881089.150968:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306720396 00000100:00000001:3.0:1788881089.150973:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306725327 00000100:00000001:0.0:1788881089.150976:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306725327 00000100:00000001:3.0:1788881089.150976:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306720396 00000400:00000200:2.0:1788881089.150977:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788881089.150983:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306725327 00000100:00000010:0.0:1788881089.150984:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000da3f50d1. 00000100:00000001:3.0:1788881089.150987:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306720396 00000800:00000200:2.0:1788881089.150988:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881089.150991:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1788881089.150996:0:201285:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306725327 00000100:00000001:3.0:1788881089.150996:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306725327 00000800:00000010:2.0:1788881089.150998:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000007cb951ea. 00000100:00000001:3.0:1788881089.150999:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306720396 00000100:00000010:0.0:1788881089.151004:0:201285:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c0bfe00. 00000800:00000200:2.0:1788881089.151007:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788881089.151010:0:201285:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000200:2.0:1788881089.151015:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788881089.151017:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881089.151018:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306725327 00000800:00000200:2.0:1788881089.151019:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000007cb951ea type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788881089.151023:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881089.151024:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881089.151026:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881089.151029:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881089.151031:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881089.151032:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10757 usecs (11 jiffies) 00000100:00000001:3.0:1788881089.151033:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881089.151036:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881089.151040:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306720267 00000100:00000001:3.0:1788881089.151040:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881089.151049:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881089.151049:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881089.151051:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306725327 00000100:00000001:0.0:1788881089.151054:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881089.151054:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788881089.151055:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306720267 00000100:00000001:3.0:1788881089.151059:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881089.151061:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306725327 00000100:00000001:3.0:1788881089.151063:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881089.151066:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881089.151067:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306720267 00000100:00000001:3.0:1788881089.151067:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881089.151070:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881089.151073:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306725327 00000100:00000001:0.0:1788881089.151074:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881089.151074:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881089.151078:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306720267 00000100:00000001:0.0:1788881089.151080:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881089.151081:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306725327 00000100:00000001:2.0:1788881089.151085:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881089.151086:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881089.151088:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881089.151090:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881089.151093:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881089.151094:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788881089.151096:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881089.151097:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881089.151098:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881089.151100:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881089.151110:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881089.151115:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000bd68c00a. 00000400:00000200:1.0:1788881089.151121:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881089.151126:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881089.151132:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000dc289a80 00000400:00000010:1.0:1788881089.151134:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000dc289a80. 00000100:00000001:1.0:1788881089.151138:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881089.151142:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881105 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881089.151151:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881089.151154:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881105 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881089.151161:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881089.151162:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788881089.151175:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881089.151178:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000400:00000200:1.0:1788881089.151180:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881089.151183:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881089.151186:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b6d0d6b5 00000400:00000010:1.0:1788881089.151187:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b6d0d6b5. 00000100:00000001:1.0:1788881089.151189:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881089.151191:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881089.151197:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881089.151198:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881089.151203:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881089.151205:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788881089.151304:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881089.151307:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000007cb951ea. 00000400:00000200:1.0:1788881089.151310:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881089.151313:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881089.151316:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000008f790e4a 00000400:00000010:1.0:1788881089.151318:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000008f790e4a. 00000100:00000001:1.0:1788881089.151321:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881089.151324:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881089.151331:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881089.151333:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881089.151339:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881089.151341:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788881089.152801:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881089.152813:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881089.152816:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881089.152818:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881089.152825:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881089.152837:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b12d00 00000400:00000200:1.0:1788881089.152844:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1419 [1] + 224 00000800:00000001:1.0:1788881089.152852:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881089.152855:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881089.152857:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881089.152860:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881089.152865:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881089.152870:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881089.152880:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881089.152897:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881089.152904:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881089.152907:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881089.152908:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881089.152909:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881089.152912:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881089.152917:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12d80 00000400:00000200:1.0:1788881089.152921:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1421 [1] + 224 00000800:00000001:1.0:1788881089.152924:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881089.152926:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881089.152927:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881089.152930:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788881089.152930:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881089.152932:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788881089.152936:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:1.0:1788881089.152936:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788881089.152943:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:1.0:1788881089.152944:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788881089.152946:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788881089.152948:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000400:00000200:0.0:1788881089.152953:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:0.0:1788881089.152956:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881089.152958:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788881089.152961:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881089.152964:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881089.152969:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788881089.152971:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788881089.152973:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881089.152977:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881089.152980:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881089.152981:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881089.152984:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881089.152988:0:201285:0:(client.c:1580:after_reply()) Process entered 00000400:00000200:1.0:1788881089.152988:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 02000000:00000001:0.0:1788881089.152992:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881089.152992:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788881089.152996:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12e00 00000100:00000001:0.0:1788881089.152997:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881089.152997:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881089.153001:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881089.153001:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1429 [1] + 224 02000000:00000001:0.0:1788881089.153007:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881089.153007:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:3.0:1788881089.153008:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788881089.153010:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00001000:0.0:1788881089.153017:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=17 v=5 (5 5 5 5) 00000400:00000010:3.0:1788881089.153019:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ca22282b. 00000800:00000001:1.0:1788881089.153020:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881089.153024:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000100:00001000:0.0:1788881089.153025:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=17 v=5 (1 1 1 1) 00000400:00000010:3.0:1788881089.153027:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000400:00000200:1.0:1788881089.153028:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881089.153031:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881089.153032:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788881089.153036:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881089.153037:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:3.0:1788881089.153039:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788881089.153043:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788881089.153046:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00010000:00000001:0.0:1788881089.153048:0:201285:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881089.153053:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788881089.153054:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881089.153055:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788881089.153059:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881089.153078:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788881089.153080:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881089.153085:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:0.0:1788881089.153086:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788881089.153088:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:1.0:1788881089.153089:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881089.153092:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881089.153092:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881089.153096:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881089.153096:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881089.153101:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881089.153104:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000040:0.0:1788881089.153105:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788881089.153109:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:3.0:1788881089.153111:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=7 v=5 (5 5 5 5) 00000800:00000001:1.0:1788881089.153113:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788881089.153118:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=7 v=5 (1 1 1 1) 00000100:00000001:3.0:1788881089.153125:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881089.153128:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881089.153131:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00100000:0.0:1788881089.153133:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000006a9c2e02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:fa4856d5-51a8-499e-b3f8-b477743a3dec:201285:1875776780709120:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881089.153138:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881089.153141:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881089.153142:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:3.0:1788881089.153144:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881089.153146:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000001:2.0:1788881089.153147:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:3.0:1788881089.153149:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000400:00000001:2.0:1788881089.153151:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:3.0:1788881089.153152:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000040:0.0:1788881089.153154:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000006a9c2e02 x1875776780709120/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788881089.153155:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000100:00000001:3.0:1788881089.153157:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000200:2.0:1788881089.153162:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000100:00000001:3.0:1788881089.153162:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881089.153166:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000400:00000010:2.0:1788881089.153167:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 02000000:00000001:0.0:1788881089.153170:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:3.0:1788881089.153172:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:0.0:1788881089.153173:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000007e76baee. 00000100:00000001:2.0:1788881089.153173:0:201287:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:0.0:1788881089.153180:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000200:2.0:1788881089.153181:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788881089.153183:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788881089.153184:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000020:00000040:0.0:1788881089.153187:0:201285:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788881089.153189:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000020:00000001:0.0:1788881089.153191:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788881089.153192:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:0.0:1788881089.153195:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b016e4f4. 00000100:00000040:3.0:1788881089.153200:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881089.153202:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881089.153205:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881089.153207:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000006a9c2e02. 00000100:00000200:2.0:1788881089.153210:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881089.153214:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00100000:3.0:1788881089.153215:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000003b122924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780709248:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881089.153217:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881089.153220:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881089.153226:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881089.153228:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881089.153229:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881089.153250:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881089.153254:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000003b122924 x1875776780709248/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881089.153255:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881089.153258:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881089.153259:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881089.153262:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881089.153265:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788881089.153268:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788881089.153270:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788881089.153271:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002ce43d73. 02000000:00000001:2.0:1788881089.153275:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788881089.153278:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788881089.153279:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881089.153282:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881089.153284:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881089.153284:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788881089.153286:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881089.153287:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000030024c2c. 02000000:00000001:2.0:1788881089.153289:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881089.153291:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881089.153293:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00001000:2.0:1788881089.153295:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=17 v=5 (5 5 5 5) 00000100:00000010:3.0:1788881089.153295:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000003b122924. 00000100:00000001:3.0:1788881089.153298:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881089.153299:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788881089.153302:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=101 v=5 (1 1 0 0) 00000100:00000001:3.0:1788881089.153302:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881089.153307:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881089.153308:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881089.153309:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881089.153311:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881089.153312:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881089.153315:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788881089.153316:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788881089.153321:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788881089.153329:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881089.153331:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788881089.153332:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788881089.153334:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788881089.153335:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881089.153338:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881089.153344:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881089.153346:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881089.153347:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788881089.153350:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788881089.153357:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000b5bd1bf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780709376:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881089.153361:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881089.153363:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881089.153365:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000b5bd1bf6 x1875776780709376/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881105 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881089.153371:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788881089.153372:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788881089.153374:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e5dc9915. 02000000:00000001:2.0:1788881089.153377:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788881089.153383:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788881089.153385:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788881089.153387:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788881089.153388:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000a371c7dc. 02000000:00000001:2.0:1788881089.153391:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788881089.153392:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788881089.153394:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000b5bd1bf6. 00000100:00000001:2.0:1788881089.153397:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788881089.153397:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881089.153399:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881089.153403:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881089.153404:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881089.153406:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881089.153408:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881089.214328:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881089.214335:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881089.214341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881089.214344:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881089.214346:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881089.214349:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881089.214352:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881089.214354:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881089.214356:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881089.214357:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881089.214409:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881089.214413:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881089.214415:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881090.238324:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881090.238331:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881090.238335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881090.238336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881090.238388:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881090.238394:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881090.238397:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881091.262330:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881091.262340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881091.262348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881091.262350:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881091.262396:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881091.262400:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881091.262402:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881092.286327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881092.286332:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881092.286337:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881092.286338:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881092.286384:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881092.286388:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881092.286390:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881093.310334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881093.310341:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881093.310346:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881093.310347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881093.310400:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881093.310404:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881093.310421:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881094.270327:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306725327, at 4306725376 00000100:00000040:1.0:1788881094.270327:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306725327, at 4306725376 00000100:00000040:2.0:1788881094.270327:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306725327, at 4306725376 00000100:00000010:2.0:1788881094.270342:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000da3f50d1. 00000100:00000010:0.0:1788881094.270344:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005b8bebe2. 00000100:00000010:1.0:1788881094.270346:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000009ac8d70b. 00000100:00000040:2.0:1788881094.270356:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881094.270361:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881094.270364:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000005a8bb1de. 00000020:00000040:2.0:1788881094.270368:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788881094.270372:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881094.270374:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881094.270378:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881094.270380:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306725327, at 4306725376 02000000:00000010:2.0:1788881094.270383:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000e5360691. 00000100:00000010:3.0:1788881094.270389:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c0bfe00. 00000100:00000001:2.0:1788881094.270390:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881094.270401:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788881094.270408:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788881094.270410:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881094.270415:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a8bb1de] to pc [ptlrpcd_00_03+3] req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881094.270435:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881094.270437:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881094.270447:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881094.270451:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881094.270452:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1788881094.270465:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000010:2.0:1788881094.270470:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ea101a56. 00000100:00000040:1.0:1788881094.270473:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000040:2.0:1788881094.270474:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788881094.270477:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881094.270479:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881094.270482:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881094.270486:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f47bc5fc. 00000100:00000001:1.0:1788881094.270487:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788881094.270489:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881094.270492:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788881094.270495:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881094.270496:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788881094.270500:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881094.270502:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788881094.270505:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780709504:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881094.270514:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788881094.270517:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788881094.270518:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881094.270521:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788881094.270523:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ea101a56] to pc [ptlrpcd_00_00+0] req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881094.270524:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881094.270529:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788881094.270533:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788881094.270534:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000051bfba0. 02000000:00000001:1.0:1788881094.270538:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881094.270539:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881094.270543:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 0000000007389ff7. 00000100:00000001:2.0:1788881094.270544:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881094.270547:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881094.270548:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000010:1.0:1788881094.270549:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000100:00000001:2.0:1788881094.270550:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00100000:3.0:1788881094.270553:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000010:2.0:1788881094.270554:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000071512652. 00000100:00000200:1.0:1788881094.270556:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780709504, portal 25 00000100:00000001:3.0:1788881094.270556:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:2.0:1788881094.270558:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881094.270560:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881094.270562:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788881094.270562:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:0.0:1788881094.270564:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881094.270565:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938427392 : -126823771124224 : ffff8ca78a2c4e00) 02000000:00000001:2.0:1788881094.270566:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881094.270569:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881094.270573:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881110 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:2.0:1788881094.270576:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000007cb951ea. 00000100:00000001:1.0:1788881094.270581:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788881094.270582:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881094.270585:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881094.270589:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000487fae38. 00000100:00000040:2.0:1788881094.270591:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000200:1.0:1788881094.270592:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780709504, offset 0 00000400:00000200:1.0:1788881094.270602:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881094.270606:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788881094.270608:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881094.270613:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000071512652] to pc [ptlrpcd_00_01+1] req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881094.270616:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788881094.270623:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788881094.270627:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788881094.270628:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788881094.270631:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881094.270634:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788881094.270637:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788881094.270639:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:3.0:1788881094.270641:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788881094.270647:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [621:621:256:4294967295] 192.168.204.111@tcp LPNI seq info [621:621:8:4294967295] 00000100:00000001:3.0:1788881094.270653:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788881094.270660:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881094.270662:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881094.270664:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000040:2.0:1788881094.270668:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788881094.270675:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788881094.270678:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1788881094.270679:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000800:00000200:1.0:1788881094.270683:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881094.270684:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788881094.270685:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12063) 00000100:00000001:3.0:1788881094.270685:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788881094.270690:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.270691:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000800:00000010:1.0:1788881094.270693:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000da26fd2f. 00000100:00000001:0.0:1788881094.270694:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:3.0:1788881094.270695:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788881094.270700:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788881094.270701:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881094.270703:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881094.270706:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000800:00000200:1.0:1788881094.270707:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 02000000:00000001:3.0:1788881094.270707:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881094.270712:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788881094.270714:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780709632:192.168.204.111@tcp:400: 00000800:00000200:1.0:1788881094.270723:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788881094.270724:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788881094.270727:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:3.0:1788881094.270729:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788881094.270730:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881094.270732:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000da26fd2f type 1, nob 320 niov 1 nkiov 1 02000000:00000001:3.0:1788881094.270732:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881094.270733:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788881094.270736:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788881094.270738:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:0.0:1788881094.270740:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881094.270742:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000005701f3a3. 00000100:00100000:0.0:1788881094.270743:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000071512652 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780709760:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788881094.270746:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.270748:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788881094.270751:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881094.270752:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:3.0:1788881094.270753:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ca22282b. 02000000:00000001:0.0:1788881094.270754:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881094.270758:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:1.0:1788881094.270759:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306730447 00000400:00000010:3.0:1788881094.270760:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 02000000:00000010:0.0:1788881094.270761:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008e3e56e7. 00000100:00000010:1.0:1788881094.270764:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb2ffc08. 02000000:00000001:0.0:1788881094.270765:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788881094.270766:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780709632, portal 10 00000100:00000001:1.0:1788881094.270770:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:3.0:1788881094.270770:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000010:0.0:1788881094.270771:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000100:00000001:3.0:1788881094.270774:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938436352 : -126823771115264 : ffff8ca78a2c7100) 00000100:00000001:1.0:1788881094.270776:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306730447 00000400:00000010:0.0:1788881094.270777:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000040:3.0:1788881094.270783:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881094.270784:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780709760, portal 10 00000100:00000010:1.0:1788881094.270784:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000f4742496. 00000100:00000001:1.0:1788881094.270790:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788881094.270791:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881094.270793:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938430080 : -126823771121536 : ffff8ca78a2c5880) 00000100:00000001:1.0:1788881094.270796:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306730447 00000100:00000010:1.0:1788881094.270802:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000045676c6b. 00000100:00000001:3.0:1788881094.270802:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788881094.270803:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881094.270807:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000040:3.0:1788881094.270811:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881094.270812:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306730447 00000100:00000010:1.0:1788881094.270817:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003c9d1628. 00000400:00000010:3.0:1788881094.270817:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b6d0d6b5. 00000800:00000200:2.0:1788881094.270819:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000200:3.0:1788881094.270822:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780709632, offset 0 00000100:00000001:1.0:1788881094.270824:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000010:2.0:1788881094.270824:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000da26fd2f. 00000100:00000001:0.0:1788881094.270833:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788881094.270833:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:2.0:1788881094.270833:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881094.270835:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:0.0:1788881094.270841:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881094.270841:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881094.270844:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:0.0:1788881094.270846:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 00000400:00000200:3.0:1788881094.270847:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:1.0:1788881094.270848:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881094.270851:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780709760, offset 0 00000100:00000001:1.0:1788881094.270854:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788881094.270855:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788881094.270859:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:1.0:1788881094.270860:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881094.270863:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881094.270865:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881094.270866:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881094.270867:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881094.270868:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881094.270898:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788881094.270901:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788881094.270904:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788881094.270907:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [622:622:255:4294967295] 192.168.204.111@tcp LPNI seq info [622:622:7:4294967295] 00000400:00000200:3.0:1788881094.270921:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788881094.270926:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788881094.270929:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000487fae38 00000800:00000010:3.0:1788881094.270930:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000400:00000200:0.0:1788881094.270931:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000010:2.0:1788881094.270934:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000487fae38. 00000800:00000200:3.0:1788881094.270940:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788881094.270941:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788881094.270942:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:0.0:1788881094.270948:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:3.0:1788881094.270950:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788881094.270953:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788881094.270956:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000100:00000200:2.0:1788881094.270957:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881110 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881094.270958:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788881094.270963:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [623:623:254:4294967295] 192.168.204.111@tcp LPNI seq info [623:623:6:4294967295] 00000100:00000001:3.0:1788881094.270963:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881094.270970:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 139181 usecs (140 jiffies) 00000100:00000001:3.0:1788881094.270977:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306725516 00000100:00000001:2.0:1788881094.270980:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1788881094.270982:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788881094.270983:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306730447 00000100:00000040:2.0:1788881094.270986:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881110 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881094.270990:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306725516 00000800:00000200:0.0:1788881094.270994:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881094.270996:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306730447 00000800:00000010:0.0:1788881094.271000:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000ae4abc27. 00000100:00000001:3.0:1788881094.271002:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306725516 00000100:00000001:2.0:1788881094.271004:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881094.271006:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306730447 00000100:00000001:2.0:1788881094.271007:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788881094.271009:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788881094.271011:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306725516 00000800:00000200:0.0:1788881094.271015:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788881094.271015:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306730447 00000800:00000200:0.0:1788881094.271020:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000ae4abc27 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788881094.271020:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881094.271023:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881094.271025:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881094.271027:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10716 usecs (11 jiffies) 00000100:00000001:3.0:1788881094.271028:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.271030:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306725387 00000100:00000001:3.0:1788881094.271032:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.271033:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306730447 00000100:00000001:3.0:1788881094.271036:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881094.271037:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306725387 00000800:00000200:2.0:1788881094.271039:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788881094.271040:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306730447 00000100:00000001:3.0:1788881094.271040:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000010:2.0:1788881094.271043:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000100:00000001:3.0:1788881094.271044:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.271045:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306725387 00000400:00000200:2.0:1788881094.271047:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881094.271049:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306730447 00000100:00000001:3.0:1788881094.271049:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.271054:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306725387 00000400:00000200:2.0:1788881094.271054:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881094.271059:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306730447 00000400:00000200:2.0:1788881094.271059:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b6d0d6b5 00000400:00000010:2.0:1788881094.271061:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b6d0d6b5. 00000100:00000001:2.0:1788881094.271064:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788881094.271066:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788881094.271068:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881094.271070:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.271074:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.271079:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881094.271080:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881094.271083:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881094.271095:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881094.271097:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788881094.271106:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788881094.271107:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000ae4abc27. 00000400:00000200:2.0:1788881094.271110:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881094.271112:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881094.271114:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000400:00000010:2.0:1788881094.271116:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00000001:2.0:1788881094.271117:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881094.271119:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881094.271124:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881094.271125:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881094.271128:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881094.271129:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788881094.272189:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881094.272199:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881094.272202:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881094.272204:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881094.272210:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881094.272222:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b12e80 00000400:00000200:1.0:1788881094.272228:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1431 [1] + 224 00000800:00000001:1.0:1788881094.272250:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881094.272257:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881094.272259:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881094.272263:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881094.272267:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881094.272270:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881094.272279:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881094.272294:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881094.272299:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881094.272302:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881094.272316:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881094.272319:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788881094.272323:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788881094.272324:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788881094.272325:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 0000000007389ff7. 00000400:00000200:2.0:1788881094.272328:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000400:00000010:2.0:1788881094.272330:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00000100:00000001:2.0:1788881094.272331:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881094.272334:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881094.272340:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881094.272345:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881094.272346:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881094.272348:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788881094.272349:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788881094.272351:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788881094.272361:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881094.272363:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881094.272367:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:2.0:1788881094.272368:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=22 v=5 (5 5 5 5) 00000100:00001000:2.0:1788881094.272372:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=22 v=5 (1 1 1 1) 00000100:00000001:2.0:1788881094.272374:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:1.0:1788881094.272375:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881094.272376:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788881094.272380:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:1.0:1788881094.272381:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:2.0:1788881094.272383:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881094.272385:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881094.272386:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788881094.272389:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881094.272396:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788881094.272399:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788881094.272400:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881094.272403:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788881094.272407:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881094.272414:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12f00 00000100:00100000:2.0:1788881094.272414:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000005a8bb1de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780709504:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881094.272421:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881094.272423:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881094.272426:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000005a8bb1de x1875776780709504/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881094.272427:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1439 [1] + 224 00000100:00000001:2.0:1788881094.272438:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000800:00000001:1.0:1788881094.272439:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000001:2.0:1788881094.272440:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788881094.272443:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000051bfba0. 02000000:00000001:2.0:1788881094.272445:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788881094.272447:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000800:00000001:1.0:1788881094.272448:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:2.0:1788881094.272448:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000400:00000200:1.0:1788881094.272463:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000020:00000001:2.0:1788881094.272463:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788881094.272466:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000e5360691. 00000400:00000200:1.0:1788881094.272471:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 02000000:00000001:2.0:1788881094.272472:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:1.0:1788881094.272475:0:201278:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000001:2.0:1788881094.272475:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788881094.272479:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000005a8bb1de. 00000100:00000200:1.0:1788881094.272481:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881094.272483:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:1.0:1788881094.272489:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881094.272490:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881094.272494:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881094.272500:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881094.272503:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881094.272505:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881094.272507:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881094.272510:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881094.272517:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881094.272522:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881094.272524:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881094.272527:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881094.272532:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788881094.272533:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:3.0:1788881094.272536:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788881094.272541:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788881094.272544:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:1.0:1788881094.272545:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b12f80 00000400:00000010:3.0:1788881094.272596:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ca22282b. 00000400:00000200:3.0:1788881094.272601:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000400:00000010:3.0:1788881094.272608:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:3.0:1788881094.272611:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788881094.272615:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x143d [1] + 224 00000100:00000200:3.0:1788881094.272616:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788881094.272622:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881094.272629:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1788881094.272629:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881094.272633:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881094.272641:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881094.272646:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1788881094.272649:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:1.0:1788881094.272651:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881094.272652:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881094.272657:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788881094.272660:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881094.272662:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000040:1.0:1788881094.272665:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881094.272665:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881094.272670:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788881094.272676:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=12 v=5 (5 5 5 5) 00000100:00001000:3.0:1788881094.272680:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=12 v=5 (1 1 1 1) 00000100:00000001:3.0:1788881094.272682:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881094.272683:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881094.272685:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788881094.272686:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:3.0:1788881094.272688:0:201286:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881094.272692:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:3.0:1788881094.272692:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788881094.272696:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000800:00000001:1.0:1788881094.272697:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881094.272699:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788881094.272702:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881094.272707:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881094.272714:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881094.272716:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881094.272717:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881094.272718:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881094.272720:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788881094.272724:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:3.0:1788881094.272724:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788881094.272726:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788881094.272729:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000400:00000200:0.0:1788881094.272733:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:0.0:1788881094.272735:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881094.272740:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:3.0:1788881094.272740:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ea101a56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780709632:192.168.204.111@tcp:400: 00000100:00000200:0.0:1788881094.272745:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881094.272747:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788881094.272751:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881094.272757:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881094.272759:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881094.272762:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881094.272763:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:3.0:1788881094.272764:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ea101a56 x1875776780709632/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881094.272765:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881094.272768:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881094.272769:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881094.272772:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881094.272778:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788881094.272779:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788881094.272782:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000005701f3a3. 00000100:00001000:0.0:1788881094.272784:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=22 v=5 (5 5 5 5) 02000000:00000001:3.0:1788881094.272786:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881094.272789:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00001000:0.0:1788881094.272790:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=106 v=5 (1 1 0 0) 00000020:00000040:3.0:1788881094.272791:0:201286:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881094.272794:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000020:00000001:3.0:1788881094.272796:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788881094.272797:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881094.272801:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 02000000:00000010:3.0:1788881094.272801:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f47bc5fc. 02000000:00000001:3.0:1788881094.272806:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00010000:00080000:0.0:1788881094.272807:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881094.272810:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881094.272812:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ea101a56. 00000100:00000001:3.0:1788881094.272817:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00010000:00000001:0.0:1788881094.272819:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.272821:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788881094.272821:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881094.272824:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788881094.272825:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881094.272826:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788881094.272830:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881094.272832:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.272834:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881094.272836:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788881094.272838:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881094.272839:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.272847:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881094.272848:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881094.272850:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881094.272853:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881094.272857:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000071512652 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780709760:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881094.272860:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881094.272861:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881094.272862:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000071512652 x1875776780709760/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881110 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881094.272865:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881094.272866:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881094.272867:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008e3e56e7. 02000000:00000001:0.0:1788881094.272869:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788881094.272870:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881094.272871:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881094.272872:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881094.272874:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000007cb951ea. 02000000:00000001:0.0:1788881094.272877:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881094.272878:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881094.272879:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000071512652. 00000100:00000001:0.0:1788881094.272882:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881094.272883:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881094.272885:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881094.272889:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881094.272890:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881094.272892:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881094.272894:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881094.334343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881094.334349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881094.334355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881094.334356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881094.334413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881094.334418:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881094.334435:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881095.358335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881095.358340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881095.358346:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881095.358348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881095.358403:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881095.358407:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881095.358410:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881096.382315:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881096.382322:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881096.382325:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881096.382328:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881096.382332:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881096.382334:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881096.382336:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881096.382338:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881096.382342:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881096.382343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881096.382395:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881096.382399:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881096.382402:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881097.406344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881097.406352:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881097.406358:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881097.406359:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881097.406429:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881097.406434:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881097.406437:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881098.430338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881098.430348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881098.430355:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881098.430356:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881098.430429:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881098.430434:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881098.430437:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788881099.390307:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306730447, at 4306730496 00000100:00000040:2.0:1788881099.390327:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306730447, at 4306730496 00000100:00000010:1.0:1788881099.390328:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f4742496. 00000100:00000040:3.0:1788881099.390333:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306730447, at 4306730496 00000100:00000010:2.0:1788881099.390338:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000045676c6b. 00000100:00000010:3.0:1788881099.390342:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003c9d1628. 00000100:00000040:1.0:1788881099.390391:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881099.390397:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000040:0.0:1788881099.390403:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306730447, at 4306730496 00000100:00000010:1.0:1788881099.390408:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000094565713. 00000020:00000040:1.0:1788881099.390416:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788881099.390423:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881099.390426:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000010:0.0:1788881099.390427:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb2ffc08. 02000000:00000001:1.0:1788881099.390433:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881099.390443:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000ac392bfe. 00000100:00000001:1.0:1788881099.390451:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881099.390465:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881099.390475:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881099.390478:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881099.390483:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000094565713] to pc [ptlrpcd_00_02+2] req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881099.390501:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881099.390503:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881099.390506:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881099.390508:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000077a1dd13. 00000020:00000040:1.0:1788881099.390510:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881099.390512:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881099.390513:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788881099.390513:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788881099.390516:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788881099.390518:0:201288:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00000001:3.0:1788881099.390521:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881099.390522:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881099.390525:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881099.390532:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881099.390535:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881099.390537:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881099.390539:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881099.390541:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788881099.390546:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881099.390553:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881099.390555:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881099.390558:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788881099.390560:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788881099.390562:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788881099.390564:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780709888:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881099.390567:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788881099.390573:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788881099.390575:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881099.390577:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788881099.390581:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000002974b115. 02000000:00000001:2.0:1788881099.390583:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881099.390589:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 00000400:00000010:2.0:1788881099.390595:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000200:2.0:1788881099.390602:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780709888, portal 25 00000100:00000001:2.0:1788881099.390604:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788881099.390606:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055040 : -126823771496576 : ffff8ca78a269f80) 02000000:00000010:1.0:1788881099.390629:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000131c1a55. 00000100:00000040:2.0:1788881099.390632:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881115 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881099.390641:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788881099.390643:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881099.390648:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788881099.390654:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000841b943f. 00000100:00000200:2.0:1788881099.390658:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780709888, offset 0 00000400:00000200:2.0:1788881099.390666:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788881099.390667:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881099.390678:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881099.390680:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881099.390682:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:2.0:1788881099.390694:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788881099.390700:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000077a1dd13] to pc [ptlrpcd_00_03+3] req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788881099.390707:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788881099.390720:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788881099.390726:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:2.0:1788881099.390730:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [624:624:256:4294967295] 192.168.204.111@tcp LPNI seq info [624:624:8:4294967295] 00000100:00000001:1.0:1788881099.390733:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881099.390736:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881099.390739:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881099.390743:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c09231ec. 00000020:00000040:1.0:1788881099.390752:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000400:00000200:2.0:1788881099.390758:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:1.0:1788881099.390759:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881099.390763:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:3.0:1788881099.390767:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788881099.390769:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788881099.390772:0:201287:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000800:00000200:2.0:1788881099.390779:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881099.390781:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788881099.390783:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000da26fd2f. 00000800:00000010:2.0:1788881099.390786:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000e5360691. 00000100:00000001:1.0:1788881099.390788:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881099.390791:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:1.0:1788881099.390796:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000200:2.0:1788881099.390798:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788881099.390800:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881099.390810:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881099.390811:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881099.390817:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881099.390817:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788881099.390819:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881099.390828:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788881099.390830:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c09231ec] to pc [ptlrpcd_00_00+0] req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788881099.390836:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000e5360691 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:3.0:1788881099.390837:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881099.390856:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788881099.390857:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881099.390860:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881099.390862:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881099.390864:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:2.0:1788881099.390870:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306735567 00000100:00000001:3.0:1788881099.390872:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881099.390873:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12068) 00000100:00000010:2.0:1788881099.390877:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ce680e22. 00000100:00000001:0.0:1788881099.390878:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881099.390888:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 02000000:00000001:3.0:1788881099.390894:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788881099.390896:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306735567 00000100:00000001:0.0:1788881099.390899:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1788881099.390899:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000010:2.0:1788881099.390901:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003053b935. 00000100:00000001:0.0:1788881099.390906:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788881099.390908:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 02000000:00000001:3.0:1788881099.390909:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881099.390915:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306735567 00000100:00000040:0.0:1788881099.390920:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788881099.390920:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000780e164. 00000100:00100000:3.0:1788881099.390922:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780710016:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881099.390929:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788881099.390941:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306735567 00000100:00000001:3.0:1788881099.390942:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788881099.390943:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881099.390946:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881099.390950:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881099.390952:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000010:2.0:1788881099.390958:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000a0959d87. 02000000:00000001:0.0:1788881099.390959:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788881099.390959:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881099.390970:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 02000000:00000001:0.0:1788881099.390971:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881099.390971:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788881099.390981:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000010:3.0:1788881099.390984:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008545eb31. 00000100:00100000:0.0:1788881099.390985:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780710144:192.168.204.111@tcp:400: 00000800:00000200:1.0:1788881099.390996:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:2.0:1788881099.390996:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881099.390996:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881099.390997:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788881099.391003:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881099.391008:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:1.0:1788881099.391009:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000e5360691. 02000000:00000001:0.0:1788881099.391010:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:2.0:1788881099.391010:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881099.391014:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ca22282b. 00000400:00000200:1.0:1788881099.391015:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000010:0.0:1788881099.391016:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008f22742c. 00000100:00000001:2.0:1788881099.391018:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881099.391021:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881099.391025:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000100:00000001:2.0:1788881099.391026:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788881099.391027:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:0.0:1788881099.391028:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000100:00000001:2.0:1788881099.391035:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000010:0.0:1788881099.391037:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000200:3.0:1788881099.391040:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780710016, portal 10 00000400:00000200:1.0:1788881099.391042:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000841b943f 00000100:00000001:2.0:1788881099.391042:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881099.391045:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000010:1.0:1788881099.391048:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000841b943f. 00000100:00000001:2.0:1788881099.391053:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881099.391054:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938051456 : -126823771500160 : ffff8ca78a269180) 00000100:00000001:1.0:1788881099.391055:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788881099.391056:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780710144, portal 10 00000100:00000001:0.0:1788881099.391063:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000200:1.0:1788881099.391063:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881115 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881099.391067:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938056832 : -126823771494784 : ffff8ca78a26a680) 00000100:00000040:3.0:1788881099.391073:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788881099.391076:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881099.391076:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881099.391081:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881115 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881099.391090:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881099.391095:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881099.391106:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1788881099.391106:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881099.391111:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000040:0.0:1788881099.391112:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788881099.391117:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ee6cf6a3. 00000100:00000200:0.0:1788881099.391123:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780710144, offset 0 00000400:00000200:0.0:1788881099.391131:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881099.391143:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788881099.391148:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788881099.391154:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788881099.391155:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788881099.391160:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881099.391162:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:3.0:1788881099.391164:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d9477561. 00000400:00000200:0.0:1788881099.391168:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:3.0:1788881099.391169:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780710016, offset 0 00000400:00000200:0.0:1788881099.391175:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [625:625:256:4294967295] 192.168.204.111@tcp LPNI seq info [625:625:8:4294967295] 00000400:00000200:3.0:1788881099.391176:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881099.391190:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788881099.391197:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788881099.391199:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:0.0:1788881099.391202:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000005f53e16f. 00000400:00000200:3.0:1788881099.391203:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788881099.391208:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:3.0:1788881099.391211:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:0.0:1788881099.391215:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788881099.391217:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788881099.391220:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [626:626:255:4294967295] 192.168.204.111@tcp LPNI seq info [626:626:7:4294967295] 00000800:00000200:0.0:1788881099.391224:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788881099.391229:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000005f53e16f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788881099.391231:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788881099.391253:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881099.391256:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10673 usecs (11 jiffies) 00000100:00000001:0.0:1788881099.391262:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306730508 00000800:00000200:3.0:1788881099.391265:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881099.391267:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306735567 00000100:00000001:0.0:1788881099.391269:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306730508 00000800:00000010:3.0:1788881099.391270:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000074daafaf. 00000100:00000001:0.0:1788881099.391271:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306735567 00000100:00000001:0.0:1788881099.391272:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306730508 00000100:00000001:0.0:1788881099.391281:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306735567 00000800:00000200:3.0:1788881099.391281:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788881099.391286:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306730508 00000100:00000001:0.0:1788881099.391290:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306735567 00000800:00000200:3.0:1788881099.391290:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788881099.391294:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1788881099.391295:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000074daafaf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788881099.391298:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881099.391300:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881099.391302:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881099.391304:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 138455 usecs (139 jiffies) 00000100:00000001:0.0:1788881099.391305:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881099.391310:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306730636 00000100:00000001:0.0:1788881099.391311:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881099.391313:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306735567 00000100:00000001:0.0:1788881099.391314:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881099.391316:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306730636 00000100:00000001:0.0:1788881099.391317:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881099.391319:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306735567 00000100:00000001:0.0:1788881099.391320:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881099.391322:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306730636 00000100:00000001:0.0:1788881099.391323:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881099.391326:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306735567 00000100:00000001:0.0:1788881099.391327:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881099.391329:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788881099.391330:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306730636 00000100:00000001:3.0:1788881099.391334:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306735567 00000800:00000010:1.0:1788881099.391335:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000005f53e16f. 00000100:00000001:3.0:1788881099.391336:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881099.391340:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881099.391342:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881099.391344:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881099.391352:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881099.391370:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881099.391373:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ee6cf6a3 00000400:00000010:1.0:1788881099.391375:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ee6cf6a3. 00000100:00000001:1.0:1788881099.391378:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881099.391382:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881099.391389:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881099.391391:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881099.391397:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881099.391398:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788881099.391411:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881099.391414:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000074daafaf. 00000400:00000200:1.0:1788881099.391417:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881099.391420:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881099.391422:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d9477561 00000400:00000010:1.0:1788881099.391424:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d9477561. 00000100:00000001:1.0:1788881099.391428:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881099.391431:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881099.391438:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881099.391440:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881099.391445:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881099.391447:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788881099.392217:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881099.392227:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881099.392230:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881099.392251:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881099.392259:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881099.392269:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b13000 00000400:00000200:1.0:1788881099.392277:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1449 [1] + 224 00000800:00000001:1.0:1788881099.392282:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881099.392288:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881099.392289:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881099.392292:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881099.392297:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881099.392300:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881099.392309:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881099.392326:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881099.392332:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881099.392335:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881099.392367:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881099.392370:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788881099.392375:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788881099.392377:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788881099.392379:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000400:00000200:2.0:1788881099.392383:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788881099.392385:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:2.0:1788881099.392387:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881099.392393:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881099.392402:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881099.392410:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881099.392413:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881099.392416:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788881099.392420:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788881099.392423:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788881099.392426:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881099.392429:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881099.392432:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:2.0:1788881099.392436:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=27 v=5 (5 5 5 5) 00000800:00000001:1.0:1788881099.392441:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788881099.392443:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=27 v=5 (1 1 1 1) 00000800:00000001:1.0:1788881099.392446:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788881099.392449:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:1.0:1788881099.392453:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881099.392453:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788881099.392464:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788881099.392469:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881099.392472:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:2.0:1788881099.392476:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881099.392483:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881099.392492:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13100 00000100:00000001:2.0:1788881099.392509:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788881099.392517:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1455 [1] + 224 00000100:00000001:2.0:1788881099.392517:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881099.392522:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881099.392527:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000040:2.0:1788881099.392530:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788881099.392542:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000094565713 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780709888:192.168.204.111@tcp:400: 00000800:00000001:1.0:1788881099.392547:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881099.392550:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881099.392555:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788881099.392555:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881099.392562:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000094565713 x1875776780709888/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881099.392574:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881099.392576:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788881099.392582:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:1.0:1788881099.392584:0:201280:0:(events.c:78:reply_in_callback()) Process entered 02000000:00000010:2.0:1788881099.392587:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000002974b115. 00000100:00000200:1.0:1788881099.392593:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881099.392595:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788881099.392598:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788881099.392601:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:2.0:1788881099.392610:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788881099.392615:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000ac392bfe. 00000100:00000040:1.0:1788881099.392621:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881099.392626:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788881099.392630:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788881099.392635:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000094565713. 00000100:00000001:2.0:1788881099.392642:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788881099.392644:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881099.392646:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881099.392650:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881099.392651:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881099.392655:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881099.392658:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881099.392660:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881099.392662:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881099.392672:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881099.392675:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881099.392678:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881099.392684:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788881099.392688:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881099.392693:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788881099.392693:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13080 00000400:00000001:0.0:1788881099.392699:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881099.392701:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788881099.392703:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000400:00000200:0.0:1788881099.392707:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:0.0:1788881099.392709:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881099.392711:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788881099.392716:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881099.392720:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1451 [1] + 224 00000800:00000001:1.0:1788881099.392730:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788881099.392736:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788881099.392739:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881099.392744:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881099.392750:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:1.0:1788881099.392753:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881099.392754:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881099.392757:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881099.392760:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881099.392760:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788881099.392764:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881099.392766:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881099.392767:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881099.392768:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881099.392775:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=27 v=5 (5 5 5 5) 00000100:00001000:0.0:1788881099.392779:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=111 v=5 (1 1 0 0) 00000100:00000040:1.0:1788881099.392780:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881099.392794:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881099.392795:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881099.392800:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881099.392801:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00010000:00000001:0.0:1788881099.392804:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000800:00000001:1.0:1788881099.392806:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00080000:0.0:1788881099.392812:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788881099.392820:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881099.392821:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881099.392823:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788881099.392823:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788881099.392825:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000400:00000001:3.0:1788881099.392826:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788881099.392827:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000400:00000001:3.0:1788881099.392827:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:0.0:1788881099.392828:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881099.392829:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ca22282b. 00000100:00000040:0.0:1788881099.392831:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881099.392831:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000400:00000010:3.0:1788881099.392833:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:3.0:1788881099.392835:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788881099.392838:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:3.0:1788881099.392838:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881099.392840:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881099.392842:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881099.392844:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788881099.392845:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881099.392852:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c09231ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780710144:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881099.392852:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881099.392856:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881099.392858:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788881099.392858:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881099.392859:0:201287:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:0.0:1788881099.392860:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c09231ec x1875776780710144/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881099.392861:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881099.392862:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881099.392863:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881099.392865:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881099.392866:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00001000:3.0:1788881099.392868:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=17 v=5 (5 5 5 5) 02000000:00000001:0.0:1788881099.392869:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00001000:3.0:1788881099.392870:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=17 v=5 (1 1 1 1) 02000000:00000010:0.0:1788881099.392871:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008f22742c. 00000100:00000001:3.0:1788881099.392873:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881099.392874:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881099.392875:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788881099.392877:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881099.392879:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788881099.392880:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:0.0:1788881099.392881:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788881099.392882:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000020:00000001:0.0:1788881099.392883:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788881099.392883:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788881099.392884:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000040:3.0:1788881099.392885:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788881099.392889:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881099.392891:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000da26fd2f. 00000100:00000001:3.0:1788881099.392892:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881099.392893:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:0.0:1788881099.392894:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788881099.392895:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881099.392897:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881099.392900:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881099.392901:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c09231ec. 00000100:00000001:0.0:1788881099.392904:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00100000:3.0:1788881099.392904:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000077a1dd13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780710016:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881099.392905:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881099.392907:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881099.392907:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881099.392909:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881099.392910:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000077a1dd13 x1875776780710016/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881115 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881099.392911:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881099.392912:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881099.392916:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788881099.392917:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788881099.392917:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788881099.392918:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008545eb31. 00000100:00000001:0.0:1788881099.392919:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881099.392920:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881099.392921:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881099.392926:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788881099.392928:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881099.392930:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000131c1a55. 02000000:00000001:3.0:1788881099.392932:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881099.392933:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881099.392934:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000077a1dd13. 00000100:00000001:3.0:1788881099.392936:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881099.392937:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881099.392939:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881099.392942:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881099.392943:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881099.392944:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881099.392946:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881099.454331:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881099.454336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881099.454341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881099.454343:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881099.454345:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881099.454347:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881099.454349:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881099.454351:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881099.454352:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881099.454353:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881099.454411:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881099.454415:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881099.454416:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881100.478321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881100.478329:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881100.478335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881100.478347:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881100.478402:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881100.478407:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881100.478410:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881101.502336:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881101.502343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881101.502348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881101.502348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881101.502400:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881101.502404:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881101.502407:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881102.526380:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881102.526390:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881102.526398:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881102.526399:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881102.526445:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881102.526450:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881102.526453:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881103.550327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881103.550333:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881103.550338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881103.550339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881103.550386:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881103.550391:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881103.550395:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881104.510341:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306735567, at 4306735616 00000100:00000040:1.0:1788881104.510341:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306735567, at 4306735616 00000100:00000040:2.0:1788881104.510341:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306735567, at 4306735616 00000100:00000010:2.0:1788881104.510357:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000780e164. 00000100:00000010:1.0:1788881104.510359:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000003053b935. 00000100:00000010:0.0:1788881104.510362:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ce680e22. 00000100:00000040:3.0:1788881104.510370:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881104.510382:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881104.510395:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000008eea6566. 00000020:00000040:3.0:1788881104.510402:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:3.0:1788881104.510408:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881104.510411:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788881104.510418:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881104.510430:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000074daafaf. 00000100:00000001:3.0:1788881104.510439:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881104.510453:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788881104.510466:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881104.510469:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881104.510476:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000008eea6566] to pc [ptlrpcd_00_01+1] req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881104.510507:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881104.510510:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881104.510514:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788881104.510518:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c1ccbaa0. 00000020:00000040:3.0:1788881104.510521:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:2.0:1788881104.510523:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881104.510528:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881104.510530:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788881104.510536:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1788881104.510539:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881104.510541:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881104.510552:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000100:00000001:2.0:1788881104.510555:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 02000000:00000001:3.0:1788881104.510556:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788881104.510557:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881104.510561:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881104.510562:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788881104.510567:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788881104.510570:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788881104.510571:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000032f1ef9f. 00000100:00100000:2.0:1788881104.510574:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780710272:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881104.510582:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788881104.510582:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881104.510590:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788881104.510593:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881104.510593:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788881104.510599:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788881104.510606:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008d1fdc54. 02000000:00000001:2.0:1788881104.510609:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881104.510610:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881104.510613:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881104.510616:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 00000400:00000010:2.0:1788881104.510622:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000200:2.0:1788881104.510627:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780710272, portal 25 00000100:00000040:3.0:1788881104.510629:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1ccbaa0] to pc [ptlrpcd_00_02+2] req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881104.510630:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788881104.510632:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643431808 : -126823066119808 : ffff8ca7b431d180) 00000100:00000040:2.0:1788881104.510642:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881120 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881104.510661:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788881104.510665:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788881104.510668:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000793778c8. 00000100:00000200:2.0:1788881104.510671:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780710272, offset 0 00000100:00000001:3.0:1788881104.510678:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881104.510679:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788881104.510688:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788881104.510694:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788881104.510699:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000010:3.0:1788881104.510699:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000d7dcc0ad. 00000400:00000200:2.0:1788881104.510700:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000020:00000040:3.0:1788881104.510707:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881104.510712:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788881104.510712:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1788881104.510713:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881104.510718:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788881104.510719:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881104.510722:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000200:2.0:1788881104.510723:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 02000000:00000001:3.0:1788881104.510726:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000040:0.0:1788881104.510728:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788881104.510731:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881104.510732:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788881104.510742:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000010:3.0:1788881104.510745:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000024dd99c. 00000100:00000001:3.0:1788881104.510755:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881104.510756:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [627:627:256:4294967295] 192.168.204.111@tcp LPNI seq info [627:627:8:4294967295] 00000100:00000001:0.0:1788881104.510759:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788881104.510767:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788881104.510770:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881104.510781:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:2.0:1788881104.510790:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:0.0:1788881104.510799:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881104.510801:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881104.510810:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881104.510811:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881104.510812:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:2.0:1788881104.510827:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000f9812869. 00000100:00100000:0.0:1788881104.510832:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780710400:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788881104.510832:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d7dcc0ad] to pc [ptlrpcd_00_03+3] req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788881104.510844:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788881104.510847:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788881104.510853:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881104.510856:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881104.510863:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 02000000:00000001:0.0:1788881104.510865:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:2.0:1788881104.510868:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000f9812869 type 1, nob 320 niov 1 nkiov 1 02000000:00000010:0.0:1788881104.510870:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a6d90e67. 02000000:00000001:0.0:1788881104.510874:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881104.510876:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881104.510879:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000100:00000040:3.0:1788881104.510882:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881104.510883:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881104.510891:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306740687 00000100:00000040:3.0:1788881104.510891:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:0.0:1788881104.510893:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000200:0.0:1788881104.510898:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780710400, portal 10 00000100:00000010:2.0:1788881104.510899:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000000780e164. 00000100:00000001:0.0:1788881104.510908:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881104.510911:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643438976 : -126823066112640 : ffff8ca7b431ed80) 00000100:00000001:2.0:1788881104.510911:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000040:3.0:1788881104.510913:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12073) 00000100:00000001:2.0:1788881104.510921:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306740687 00000100:00000040:0.0:1788881104.510926:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000010:2.0:1788881104.510929:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000b63100a0. 00000100:00000001:2.0:1788881104.510935:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788881104.510937:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788881104.510943:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306735567, at 4306735616 00000100:00000040:0.0:1788881104.510944:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881104.510944:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306740687 00000400:00000010:0.0:1788881104.510951:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000ee6cf6a3. 00000100:00000010:2.0:1788881104.510956:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000488d1158. 00000100:00000200:0.0:1788881104.510957:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780710400, offset 0 00000100:00000010:3.0:1788881104.510958:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000a0959d87. 00000100:00000001:2.0:1788881104.510962:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788881104.510971:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306740687 00000400:00000200:0.0:1788881104.510975:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:2.0:1788881104.510977:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000eb1a14f4. 00000800:00000200:1.0:1788881104.510979:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881104.510984:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000f9812869. 00000100:00000001:2.0:1788881104.510984:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:0.0:1788881104.510985:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788881104.510993:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788881104.510993:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1788881104.510996:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788881104.510999:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881104.511001:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788881104.511004:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881104.511005:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788881104.511008:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788881104.511009:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788881104.511018:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:2.0:1788881104.511018:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881104.511021:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000400:00000200:0.0:1788881104.511026:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [628:628:255:4294967295] 192.168.204.111@tcp LPNI seq info [628:628:7:4294967295] 00000100:00000001:2.0:1788881104.511026:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881104.511029:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881104.511033:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881104.511036:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881104.511042:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788881104.511049:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788881104.511049:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000793778c8 00000400:00000010:1.0:1788881104.511052:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000793778c8. 00000800:00000010:0.0:1788881104.511055:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003806e31f. 00000100:00000001:1.0:1788881104.511057:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881104.511063:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881120 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788881104.511074:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:1.0:1788881104.511074:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881104.511078:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881120 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1788881104.511083:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:1.0:1788881104.511086:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881104.511088:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788881104.511093:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003806e31f type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788881104.511099:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881104.511101:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881104.511102:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788881104.511106:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881104.511107:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881104.511113:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 137736 usecs (138 jiffies) 00000100:00000001:1.0:1788881104.511114:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788881104.511117:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881104.511118:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306735754 02000000:00000001:1.0:1788881104.511120:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788881104.511123:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881104.511124:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306740687 02000000:00000001:1.0:1788881104.511126:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881104.511128:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306735754 00000100:00100000:1.0:1788881104.511129:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780710528:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881104.511134:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306740687 00000100:00000001:1.0:1788881104.511135:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788881104.511139:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306735754 02000000:00000001:1.0:1788881104.511139:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788881104.511142:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306740687 02000000:00000001:1.0:1788881104.511143:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881104.511147:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306735754 02000000:00000001:1.0:1788881104.511147:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:0.0:1788881104.511152:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306740687 02000000:00000010:1.0:1788881104.511152:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000094e066a1. 00000100:00000001:0.0:1788881104.511155:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788881104.511156:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881104.511160:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881104.511161:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001ee767d5. 00000100:00000001:0.0:1788881104.511164:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1788881104.511166:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000494fa13c. 00000100:00000001:0.0:1788881104.511167:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788881104.511169:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780710528, portal 10 00000100:00000001:0.0:1788881104.511173:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881104.511173:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881104.511177:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881104.511177:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250643428224 : -126823066123392 : ffff8ca7b431c380) 00000100:00000001:0.0:1788881104.511183:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881104.511185:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881104.511186:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881104.511190:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881104.511194:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881104.511194:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788881104.511199:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881104.511201:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000487fae38. 00000100:00000200:1.0:1788881104.511203:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780710528, offset 0 00000400:00000200:1.0:1788881104.511207:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:1.0:1788881104.511210:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:1.0:1788881104.511215:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788881104.511219:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:1.0:1788881104.511223:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788881104.511227:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:1.0:1788881104.511230:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [629:629:255:4294967295] 192.168.204.111@tcp LPNI seq info [629:629:7:4294967295] 00000800:00000200:3.0:1788881104.511230:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:1.0:1788881104.511258:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:1.0:1788881104.511275:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:1.0:1788881104.511278:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000a371c7dc. 00000800:00000200:1.0:1788881104.511281:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:1.0:1788881104.511283:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:1.0:1788881104.511285:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000a371c7dc type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788881104.511287:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881104.511289:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10628 usecs (11 jiffies) 00000100:00000001:1.0:1788881104.511291:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306735628 00000100:00000001:1.0:1788881104.511292:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306740687 00000100:00000001:1.0:1788881104.511294:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306735628 00000100:00000001:1.0:1788881104.511295:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306740687 00000100:00000001:1.0:1788881104.511297:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306735628 00000100:00000001:1.0:1788881104.511298:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306740687 00000100:00000001:1.0:1788881104.511299:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306735628 00000100:00000001:1.0:1788881104.511300:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306740687 00000100:00000001:1.0:1788881104.511301:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881104.511302:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881104.511306:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881104.511307:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881104.511310:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881104.511312:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000800:00000010:3.0:1788881104.511313:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003806e31f. 00000100:00000001:1.0:1788881104.511315:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881104.511318:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881104.511322:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881104.511324:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788881104.511325:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881104.511338:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881104.511345:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000ee6cf6a3 00000400:00000010:3.0:1788881104.511348:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000ee6cf6a3. 00000100:00000001:3.0:1788881104.511353:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881104.511356:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881104.511365:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881104.511367:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881104.511373:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881104.511375:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788881104.511418:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788881104.511421:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000a371c7dc. 00000400:00000200:3.0:1788881104.511425:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788881104.511428:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881104.511431:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000487fae38 00000400:00000010:3.0:1788881104.511433:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000487fae38. 00000100:00000001:3.0:1788881104.511435:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788881104.511438:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881104.511446:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881104.511448:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881104.511453:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881104.511455:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788881104.512357:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881104.512367:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881104.512370:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881104.512371:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881104.512377:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881104.512385:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b13180 00000400:00000200:1.0:1788881104.512389:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1461 [1] + 224 00000800:00000001:1.0:1788881104.512394:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881104.512399:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881104.512401:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881104.512404:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881104.512408:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881104.512411:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881104.512418:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881104.512430:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881104.512436:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881104.512440:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881104.512442:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881104.512445:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881104.512448:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881104.512451:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13200 00000400:00000200:1.0:1788881104.512454:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1469 [1] + 224 00000800:00000001:1.0:1788881104.512457:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881104.512460:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881104.512462:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881104.512465:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881104.512466:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881104.512468:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881104.512473:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881104.512474:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881104.512476:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788881104.512483:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788881104.512486:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000001:2.0:1788881104.512487:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788881104.512491:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000400:00000200:2.0:1788881104.512495:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788881104.512498:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:2.0:1788881104.512502:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:1.0:1788881104.512504:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881104.512507:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:2.0:1788881104.512508:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881104.512513:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881104.512517:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:2.0:1788881104.512521:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1788881104.512523:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881104.512526:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788881104.512529:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000100:00000001:2.0:1788881104.512533:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:0.0:1788881104.512534:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000100:00000001:2.0:1788881104.512537:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:0.0:1788881104.512538:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881104.512542:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788881104.512543:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788881104.512546:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000200:0.0:1788881104.512547:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881104.512550:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788881104.512554:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881104.512557:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881104.512558:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788881104.512564:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=32 v=5 (5 5 5 5) 00000100:00001000:2.0:1788881104.512573:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=32 v=5 (1 1 1 1) 00000100:00000001:0.0:1788881104.512575:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881104.512578:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881104.512578:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788881104.512580:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881104.512581:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881104.512583:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881104.512586:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00010000:00000001:2.0:1788881104.512586:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:0.0:1788881104.512588:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788881104.512590:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881104.512591:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881104.512596:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881104.512597:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=22 v=5 (5 5 5 5) 00000100:00001000:0.0:1788881104.512602:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=22 v=5 (1 1 1 1) 00000100:00000001:0.0:1788881104.512605:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000040:2.0:1788881104.512605:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881104.512608:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881104.512612:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788881104.512615:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881104.512619:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:2.0:1788881104.512621:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788881104.512622:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881104.512627:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788881104.512627:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881104.512631:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881104.512631:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881104.512642:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881104.512643:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881104.512663:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881104.512667:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00100000:2.0:1788881104.512668:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000008eea6566 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780710272:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881104.512670:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881104.512677:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881104.512678:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881104.512683:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00100000:0.0:1788881104.512691:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c1ccbaa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780710400:192.168.204.111@tcp:400: 00000100:00000040:2.0:1788881104.512691:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000008eea6566 x1875776780710272/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788881104.512700:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881104.512702:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881104.512707:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788881104.512711:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000040:0.0:1788881104.512713:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c1ccbaa0 x1875776780710400/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788881104.512714:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788881104.512716:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000800:00000001:1.0:1788881104.512722:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 02000000:00000010:2.0:1788881104.512722:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008d1fdc54. 00000800:00000001:1.0:1788881104.512730:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881104.512732:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788881104.512732:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:0.0:1788881104.512737:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000020:00000001:2.0:1788881104.512738:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788881104.512742:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 02000000:00000010:0.0:1788881104.512748:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a6d90e67. 00000400:00000200:1.0:1788881104.512748:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000020:00000001:2.0:1788881104.512748:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788881104.512752:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000074daafaf. 02000000:00000001:0.0:1788881104.512761:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 02000000:00000001:2.0:1788881104.512764:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788881104.512768:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788881104.512769:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000008eea6566. 00000020:00000001:0.0:1788881104.512770:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881104.512775:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000400:00000200:1.0:1788881104.512778:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13280 00000020:00000001:0.0:1788881104.512779:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:2.0:1788881104.512779:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000010:0.0:1788881104.512782:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000032f1ef9f. 00000100:00000001:2.0:1788881104.512784:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881104.512788:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881104.512789:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881104.512793:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788881104.512793:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1788881104.512794:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1471 [1] + 224 00000100:00000010:0.0:1788881104.512798:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c1ccbaa0. 00000100:00000001:2.0:1788881104.512799:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881104.512807:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788881104.512807:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:1.0:1788881104.512808:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881104.512809:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881104.512812:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881104.512814:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881104.512819:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881104.512820:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881104.512822:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881104.512823:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881104.512829:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881104.512829:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881104.512844:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881104.512849:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881104.512856:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881104.512867:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881104.512884:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881104.512892:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881104.512895:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881104.512917:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881104.512920:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788881104.512924:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788881104.512925:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788881104.512927:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001ee767d5. 00000400:00000200:2.0:1788881104.512931:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000494fa13c 00000400:00000010:2.0:1788881104.512933:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000494fa13c. 00000100:00000001:2.0:1788881104.512935:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881104.512940:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881104.512949:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881104.512955:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881104.512957:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881104.512959:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:2.0:1788881104.512961:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788881104.512962:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788881104.512963:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881104.512965:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788881104.512969:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=32 v=5 (5 5 5 5) 00000100:00001000:2.0:1788881104.512972:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=116 v=5 (1 1 0 0) 00000100:00000001:2.0:1788881104.512974:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788881104.512975:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788881104.512977:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788881104.512980:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788881104.512986:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881104.512996:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788881104.512997:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:2.0:1788881104.512999:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788881104.513000:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881104.513003:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881104.513009:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881104.513011:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881104.513012:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788881104.513014:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788881104.513021:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000d7dcc0ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780710528:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881104.513025:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881104.513027:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881104.513029:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000d7dcc0ad x1875776780710528/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881120 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881104.513034:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788881104.513035:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788881104.513037:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000094e066a1. 02000000:00000001:2.0:1788881104.513040:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788881104.513041:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788881104.513042:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:2.0:1788881104.513044:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1788881104.513045:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000024dd99c. 02000000:00000001:2.0:1788881104.513047:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788881104.513049:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788881104.513049:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000d7dcc0ad. 00000100:00000001:2.0:1788881104.513052:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788881104.513053:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881104.513054:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881104.513057:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881104.513058:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881104.513060:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881104.513061:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881104.574388:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881104.574398:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881104.574407:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881104.574408:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881104.574477:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881104.574482:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881104.574485:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881105.598309:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881105.598319:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881105.598326:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881105.598327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881105.598360:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881105.598365:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881105.598368:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881106.622327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881106.622337:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881106.622340:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881106.622344:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881106.622349:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881106.622351:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881106.622352:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881106.622354:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881106.622356:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881106.622357:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881106.622382:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881106.622388:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881106.622392:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881107.646321:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881107.646328:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881107.646334:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881107.646335:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881107.646386:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881107.646391:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881107.646394:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881108.670391:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881108.670400:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881108.670406:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881108.670407:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881108.670469:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881108.670473:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881108.670475:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788881109.630336:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306740687, at 4306740736 00000100:00000040:0.0:1788881109.630341:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306740687, at 4306740736 00000100:00000040:2.0:1788881109.630341:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306740687, at 4306740736 00000100:00000040:1.0:1788881109.630350:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881109.630363:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881109.630365:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000488d1158. 00000100:00000010:0.0:1788881109.630367:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000000780e164. 00000100:00000010:1.0:1788881109.630371:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ca5e4c87. 00000100:00000010:3.0:1788881109.630371:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000eb1a14f4. 00000020:00000040:1.0:1788881109.630378:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788881109.630382:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881109.630385:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881109.630390:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881109.630396:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000030024c2c. 00000100:00000001:1.0:1788881109.630404:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881109.630413:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881109.630425:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881109.630426:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881109.630430:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ca5e4c87] to pc [ptlrpcd_00_00+0] req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881109.630450:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881109.630451:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881109.630454:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881109.630455:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000c27d4622. 00000020:00000040:1.0:1788881109.630457:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881109.630458:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881109.630460:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881109.630461:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881109.630466:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000b016e4f4. 00000100:00000001:1.0:1788881109.630469:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881109.630472:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881109.630477:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881109.630478:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881109.630480:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881109.630480:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881109.630483:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c27d4622] to pc [ptlrpcd_00_01+1] req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788881109.630488:0:201286:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788881109.630490:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881109.630494:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881109.630504:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881109.630506:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881109.630506:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881109.630512:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881109.630515:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788881109.630518:0:201285:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000040:1.0:1788881109.630518:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788881109.630519:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881109.630524:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788881109.630531:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000040:2.0:1788881109.630534:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881109.630537:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:1.0:1788881109.630538:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002980f9ff. 00000100:00000001:0.0:1788881109.630544:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881109.630548:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000020:00000040:1.0:1788881109.630550:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881109.630554:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881109.630554:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881109.630559:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881109.630560:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881109.630561:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788881109.630561:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881109.630565:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788881109.630568:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881109.630570:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788881109.630574:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881109.630575:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881109.630575:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881109.630588:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000006df406c6. 00000100:00100000:2.0:1788881109.630589:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780710656:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881109.630599:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881109.630602:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881109.630604:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1788881109.630613:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881109.630617:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:2.0:1788881109.630618:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881109.630627:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788881109.630628:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881109.630635:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788881109.630640:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:0.0:1788881109.630646:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881109.630651:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000010:2.0:1788881109.630652:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008e3e56e7. 00000100:00100000:0.0:1788881109.630656:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780710784:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881109.630657:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881109.630662:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881109.630667:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788881109.630670:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002980f9ff] to pc [ptlrpcd_00_02+2] req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881109.630672:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788881109.630674:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881109.630675:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 02000000:00000001:0.0:1788881109.630682:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788881109.630690:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000001f6835d8. 00000400:00000010:2.0:1788881109.630691:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 02000000:00000001:0.0:1788881109.630693:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881109.630699:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000100:00000200:2.0:1788881109.630701:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780710656, portal 25 00000400:00000010:0.0:1788881109.630703:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000001:2.0:1788881109.630705:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000200:0.0:1788881109.630708:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780710784, portal 10 00000100:00000001:1.0:1788881109.630708:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881109.630709:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938050560 : -126823771501056 : ffff8ca78a268e00) 00000100:00000001:3.0:1788881109.630721:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881109.630722:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788881109.630723:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881109.630726:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788881109.630730:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881125 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881109.630731:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938049664 : -126823771501952 : ffff8ca78a268a80) 00000100:00000001:3.0:1788881109.630732:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788881109.630745:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788881109.630747:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881109.630751:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788881109.630766:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12078) 00000100:00000040:1.0:1788881109.630791:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306740687, at 4306740736 00000100:00000001:2.0:1788881109.630803:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:1.0:1788881109.630812:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000b63100a0. 00000100:00000001:0.0:1788881109.630817:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788881109.630822:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881109.630827:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000010:2.0:1788881109.630829:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009c31f261. 00000100:00000001:3.0:1788881109.630831:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881109.630833:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788881109.630839:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000200:2.0:1788881109.630840:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780710656, offset 0 00000400:00000010:0.0:1788881109.630841:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000048d2df12. 02000000:00000001:3.0:1788881109.630843:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788881109.630852:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780710784, offset 0 02000000:00000001:3.0:1788881109.630852:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881109.630867:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00100000:3.0:1788881109.630872:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780710912:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881109.630875:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:0.0:1788881109.630883:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788881109.630884:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788881109.630891:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881109.630892:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788881109.630900:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788881109.630905:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000fdf771c7. 00000400:00000200:2.0:1788881109.630907:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:3.0:1788881109.630909:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881109.630916:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:3.0:1788881109.630917:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ca22282b. 00000400:00000200:2.0:1788881109.630924:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000010:3.0:1788881109.630927:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000400:00000200:2.0:1788881109.630931:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:3.0:1788881109.630937:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780710912, portal 10 00000400:00000200:2.0:1788881109.630939:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [630:630:256:4294967295] 192.168.204.111@tcp LPNI seq info [630:630:8:4294967295] 00000100:00000001:3.0:1788881109.630945:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881109.630949:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938061312 : -126823771490304 : ffff8ca78a26b800) 00000100:00000040:3.0:1788881109.630961:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788881109.630968:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:3.0:1788881109.630986:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:0.0:1788881109.630994:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000200:2.0:1788881109.630995:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000040:3.0:1788881109.630999:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881109.631004:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000010:3.0:1788881109.631014:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f6a79a10. 00000800:00000010:2.0:1788881109.631016:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003139f6ba. 00000100:00000001:1.0:1788881109.631017:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:3.0:1788881109.631023:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780710912, offset 0 00000100:00000001:1.0:1788881109.631025:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881109.631027:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:2.0:1788881109.631040:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788881109.631040:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000200:0.0:1788881109.631041:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788881109.631051:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788881109.631054:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [631:631:255:4294967295] 192.168.204.111@tcp LPNI seq info [631:631:7:4294967295] 00000800:00000200:2.0:1788881109.631059:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:2.0:1788881109.631064:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003139f6ba type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788881109.631070:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:0.0:1788881109.631077:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000800:00000010:0.0:1788881109.631082:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003806e31f. 00000100:00000001:2.0:1788881109.631088:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788881109.631092:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:2.0:1788881109.631096:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306745807 00000800:00000200:0.0:1788881109.631101:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:0.0:1788881109.631104:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003806e31f type 1, nob 320 niov 1 nkiov 1 00000100:00000010:2.0:1788881109.631104:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000488d1158. 00000100:00000001:0.0:1788881109.631111:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881109.631115:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000040:0.0:1788881109.631118:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 137025 usecs (138 jiffies) 00000100:00000001:2.0:1788881109.631123:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306745807 00000100:00000001:0.0:1788881109.631125:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306740874 00000100:00000010:2.0:1788881109.631129:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000059817523. 00000100:00000001:0.0:1788881109.631131:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306745807 00000100:00000001:2.0:1788881109.631134:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788881109.631136:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306740874 00000100:00000001:0.0:1788881109.631143:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306745807 00000100:00000001:2.0:1788881109.631143:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306745807 00000400:00000200:3.0:1788881109.631147:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:2.0:1788881109.631149:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001e508d7a. 00000100:00000001:0.0:1788881109.631150:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306740874 00000100:00000001:2.0:1788881109.631157:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1788881109.631172:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306745807 00000100:00000001:0.0:1788881109.631173:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306745807 00000400:00000200:3.0:1788881109.631176:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000010:2.0:1788881109.631178:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ea9971ca. 00000100:00000001:0.0:1788881109.631179:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306740874 00000100:00000001:2.0:1788881109.631184:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:2.0:1788881109.631194:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881109.631195:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306745807 00000400:00000200:3.0:1788881109.631201:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788881109.631202:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881109.631203:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881109.631208:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881109.631211:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881109.631211:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881109.631218:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881109.631220:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881109.631313:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788881109.631319:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003139f6ba. 00000400:00000200:2.0:1788881109.631324:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881109.631332:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:3.0:1788881109.631348:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788881109.631352:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:3.0:1788881109.631356:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [632:632:254:4294967295] 192.168.204.111@tcp LPNI seq info [632:632:6:4294967295] 00000400:00000200:3.0:1788881109.631369:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788881109.631375:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000200:2.0:1788881109.631376:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009c31f261 00000400:00000010:2.0:1788881109.631382:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009c31f261. 00000100:00000001:2.0:1788881109.631387:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881109.631392:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881125 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881109.631400:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881109.631406:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881125 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881109.631411:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881109.631413:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000010:3.0:1788881109.631428:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000800:00000200:3.0:1788881109.631441:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000800:00000200:3.0:1788881109.631449:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000800:00000200:3.0:1788881109.631453:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788881109.631458:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881109.631462:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10586 usecs (11 jiffies) 00000100:00000001:3.0:1788881109.631466:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306740748 00000100:00000001:3.0:1788881109.631468:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306745807 00000100:00000001:3.0:1788881109.631470:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306740748 00000100:00000001:3.0:1788881109.631473:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306745807 00000800:00000200:2.0:1788881109.631475:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788881109.631476:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306740748 00000800:00000010:2.0:1788881109.631478:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003806e31f. 00000400:00000200:2.0:1788881109.631485:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788881109.631486:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306745807 00000400:00000200:2.0:1788881109.631494:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:3.0:1788881109.631495:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306740748 00000400:00000200:2.0:1788881109.631498:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000048d2df12 00000400:00000010:2.0:1788881109.631502:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000048d2df12. 00000100:00000001:3.0:1788881109.631502:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306745807 00000100:00000001:2.0:1788881109.631507:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788881109.631509:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000200:2.0:1788881109.631512:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881109.631523:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881109.631526:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881109.631532:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881109.631533:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1788881109.631549:0:201278:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788881109.631551:0:201278:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:2.0:1788881109.631555:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881109.631559:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881109.631562:0:201278:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f6a79a10 00000400:00000010:2.0:1788881109.631564:0:201278:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f6a79a10. 00000100:00000001:2.0:1788881109.631568:0:201278:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1788881109.631571:0:201278:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881109.631579:0:201278:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881109.631582:0:201278:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881109.631589:0:201278:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881109.631591:0:201278:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788881109.631592:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881109.631598:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881109.631601:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881109.631608:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881109.631642:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881109.631645:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881109.631646:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881109.631648:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881109.631656:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881109.633263:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881109.633272:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881109.633276:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881109.633278:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881109.633286:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881109.633299:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b13300 00000400:00000200:1.0:1788881109.633309:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1479 [1] + 224 00000800:00000001:1.0:1788881109.633315:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881109.633319:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881109.633321:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881109.633326:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881109.633332:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881109.633337:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881109.633346:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881109.633366:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881109.633373:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881109.633377:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881109.633378:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881109.633380:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881109.633383:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881109.633388:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13400 00000400:00000200:1.0:1788881109.633391:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1489 [1] + 224 00000800:00000001:1.0:1788881109.633394:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881109.633398:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881109.633399:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788881109.633401:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1788881109.633403:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881109.633405:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881109.633406:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881109.633410:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788881109.633410:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788881109.633413:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788881109.633416:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000100:00000040:1.0:1788881109.633420:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788881109.633420:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:2.0:1788881109.633423:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:2.0:1788881109.633428:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881109.633434:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881109.633436:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881109.633441:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:2.0:1788881109.633449:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788881109.633451:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881109.633456:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881109.633460:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881109.633461:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788881109.633463:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881109.633464:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881109.633468:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788881109.633468:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788881109.633469:0:201280:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 02000000:00000001:2.0:1788881109.633472:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788881109.633475:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000400:00000001:3.0:1788881109.633475:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:1.0:1788881109.633478:0:201280:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13380 00000100:00000001:2.0:1788881109.633478:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788881109.633479:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:2.0:1788881109.633482:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788881109.633486:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=37 v=5 (5 5 5 5) 00000100:00001000:2.0:1788881109.633489:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=37 v=5 (1 1 1 1) 00000100:00000001:2.0:1788881109.633490:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788881109.633491:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788881109.633493:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788881109.633495:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881109.633496:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881109.633499:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881109.633504:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881109.633506:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788881109.633509:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788881109.633509:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ca22282b. 00000400:00000200:3.0:1788881109.633514:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000100:00000040:2.0:1788881109.633515:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1788881109.633518:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:3.0:1788881109.633524:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00100000:2.0:1788881109.633527:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000ca5e4c87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780710656:192.168.204.111@tcp:400: 00000400:00000200:1.0:1788881109.633528:0:201280:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x147d [1] + 224 00000100:00000200:3.0:1788881109.633531:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:1.0:1788881109.633537:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788881109.633537:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881109.633541:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000800:00000001:1.0:1788881109.633543:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881109.633546:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000040:2.0:1788881109.633547:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ca5e4c87 x1875776780710656/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788881109.633549:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881109.633553:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881109.633559:0:201280:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881109.633564:0:201280:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881109.633571:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788881109.633574:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788881109.633577:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008e3e56e7. 00000100:00000001:3.0:1788881109.633579:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:2.0:1788881109.633583:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788881109.633584:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881109.633586:0:201280:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:2.0:1788881109.633587:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000100:00000001:3.0:1788881109.633587:0:201287:0:(client.c:1580:after_reply()) Process entered 00000020:00000040:2.0:1788881109.633589:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 02000000:00000001:3.0:1788881109.633591:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881109.633594:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000020:00000001:2.0:1788881109.633595:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788881109.633597:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788881109.633598:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000030024c2c. 02000000:00000001:3.0:1788881109.633602:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881109.633604:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788881109.633606:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788881109.633606:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ca5e4c87. 00000100:00001000:3.0:1788881109.633607:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=37 v=5 (5 5 5 5) 00000100:00000001:1.0:1788881109.633609:0:201280:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881109.633610:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00001000:3.0:1788881109.633612:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=121 v=5 (1 1 0 0) 00000100:00000001:2.0:1788881109.633613:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881109.633614:0:201280:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788881109.633617:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881109.633619:0:201280:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881109.633620:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788881109.633622:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881109.633623:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881109.633625:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881109.633626:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:2.0:1788881109.633630:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881109.633634:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:3.0:1788881109.633636:0:201287:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788881109.633645:0:201287:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881109.633648:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788881109.633651:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881109.633652:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881109.633654:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881109.633658:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881109.633659:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788881109.633664:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881109.633666:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:3.0:1788881109.633669:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788881109.633670:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000400:00000200:0.0:1788881109.633681:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:0.0:1788881109.633682:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881109.633685:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788881109.633690:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881109.633747:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:0.0:1788881109.633752:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881109.633752:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881109.633755:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881109.633764:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881109.633798:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00100000:3.0:1788881109.633800:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002980f9ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780710912:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881109.633805:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881109.633806:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881109.633811:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:3.0:1788881109.633812:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:0.0:1788881109.633819:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000040:3.0:1788881109.633819:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002980f9ff x1875776780710912/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881109.633825:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881109.633829:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881109.633832:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788881109.633843:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=27 v=5 (5 5 5 5) 00000100:00000001:3.0:1788881109.633843:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788881109.633847:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00001000:0.0:1788881109.633851:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=27 v=5 (1 1 1 1) 02000000:00000010:3.0:1788881109.633852:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000fdf771c7. 00000100:00000001:0.0:1788881109.633862:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 02000000:00000001:3.0:1788881109.633864:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:0.0:1788881109.633865:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881109.633869:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00010000:00000001:0.0:1788881109.633870:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000020:00000040:3.0:1788881109.633872:0:201287:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00010000:00000001:0.0:1788881109.633876:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881109.633878:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881109.633882:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000006df406c6. 00000100:00000001:0.0:1788881109.633883:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788881109.633887:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:3.0:1788881109.633888:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1788881109.633892:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000001:3.0:1788881109.633893:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788881109.633896:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788881109.633897:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002980f9ff. 00000100:00000001:3.0:1788881109.633906:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:0.0:1788881109.633910:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881109.633911:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881109.633916:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881109.633925:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881109.633928:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881109.633928:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881109.633935:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881109.633936:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881109.633939:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881109.633940:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881109.633947:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881109.633956:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000c27d4622 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780710784:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881109.633971:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881109.633973:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881109.633976:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000c27d4622 x1875776780710784/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881125 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881109.633982:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881109.633984:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881109.633987:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000001f6835d8. 02000000:00000001:0.0:1788881109.633990:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788881109.633992:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881109.633993:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881109.633996:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881109.633997:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000b016e4f4. 02000000:00000001:0.0:1788881109.634001:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881109.634003:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881109.634004:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000c27d4622. 00000100:00000001:0.0:1788881109.634007:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881109.634008:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881109.634010:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881109.634015:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881109.634016:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881109.634018:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881109.634020:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881109.694288:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881109.694294:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881109.694298:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881109.694300:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881109.694302:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881109.694305:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881109.694307:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881109.694309:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881109.694310:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881109.694312:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881109.694375:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881109.694380:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881109.694383:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881110.718327:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881110.718334:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881110.718341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881110.718343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881110.718383:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788881110.718387:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788881110.718390:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881111.742329:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881111.742340:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881111.742347:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881111.742349:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881111.742392:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881111.742395:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881111.742398:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881112.766368:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881112.766378:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881112.766386:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881112.766387:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881112.766418:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788881112.766423:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788881112.766426:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881113.790359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881113.790370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881113.790384:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881113.790385:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881113.790467:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881113.790490:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881113.790493:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788881114.750313:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306745807, at 4306745856 00000100:00000040:1.0:1788881114.750318:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788881114.750321:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306745807, at 4306745856 00000100:00000040:0.0:1788881114.750322:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306745807, at 4306745856 00000100:00000001:1.0:1788881114.750325:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788881114.750328:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001e508d7a. 00000100:00000010:3.0:1788881114.750331:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ea9971ca. 00000100:00000010:0.0:1788881114.750333:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000488d1158. 00000100:00000010:1.0:1788881114.750335:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 00000020:00000040:1.0:1788881114.750339:0:200894:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788881114.750343:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881114.750345:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881114.750349:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881114.750355:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c93f60a2. 00000100:00000001:1.0:1788881114.750360:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881114.750369:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881114.750377:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881114.750378:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881114.750383:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_00_03+3] req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881114.750401:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881114.750402:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881114.750405:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881114.750407:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000663051e5. 00000020:00000040:1.0:1788881114.750408:0:200894:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881114.750410:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881114.750411:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881114.750412:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881114.750414:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f0a0057b. 00000100:00000001:1.0:1788881114.750416:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881114.750417:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881114.750420:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788881114.750422:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881114.750426:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788881114.750427:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881114.750428:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881114.750431:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881114.750432:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881114.750434:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881114.750437:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000663051e5] to pc [ptlrpcd_00_00+0] req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881114.750462:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881114.750463:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881114.750466:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788881114.750468:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881114.750469:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881114.750470:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881114.750472:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1788881114.750475:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780711040:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881114.750479:0:200894:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1788881114.750480:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000010:1.0:1788881114.750481:0:200894:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e2be23b2. 00000020:00000040:1.0:1788881114.750484:0:200894:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881114.750486:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881114.750486:0:200894:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788881114.750486:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788881114.750490:0:200894:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788881114.750490:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881114.750491:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881114.750494:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788881114.750494:0:200894:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881114.750494:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788881114.750495:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1788881114.750500:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881114.750501:0:200894:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000004e0f6846. 02000000:00000010:2.0:1788881114.750502:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000008e3e56e7. 00000100:00000040:0.0:1788881114.750503:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881114.750506:0:200894:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881114.750507:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881114.750513:0:200894:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000010:2.0:1788881114.750514:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b258a24d. 00000100:00000001:0.0:1788881114.750518:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881114.750520:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881114.750522:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000b216ae47. 00000100:00000001:1.0:1788881114.750523:0:200894:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881114.750525:0:200894:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881114.750528:0:200894:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e2be23b2] to pc [ptlrpcd_00_01+1] req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881114.750528:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780711040, portal 25 00000100:00000001:2.0:1788881114.750533:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:0.0:1788881114.750535:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1788881114.750536:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 02000000:00000001:0.0:1788881114.750538:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881114.750543:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788881114.750543:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881130 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788881114.750548:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780711168:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881114.750548:0:200894:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881114.750553:0:200894:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788881114.750557:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:3.0:1788881114.750558:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881114.750561:0:200894:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 02000000:00000001:0.0:1788881114.750562:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1788881114.750562:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881114.750564:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788881114.750565:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881114.750566:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 02000000:00000001:0.0:1788881114.750568:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:1.0:1788881114.750569:0:200894:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12083) 00000100:00000040:2.0:1788881114.750571:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:0.0:1788881114.750572:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000784e3b9. 00000100:00000040:3.0:1788881114.750574:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881114.750576:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881114.750576:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009c31f261. 00000100:00000200:2.0:1788881114.750581:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780711040, offset 0 00000100:00000040:1.0:1788881114.750583:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306745807, at 4306745856 00000400:00000010:0.0:1788881114.750584:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 00000100:00000010:1.0:1788881114.750591:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000059817523. 00000400:00000010:0.0:1788881114.750592:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 00000100:00000001:3.0:1788881114.750593:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:2.0:1788881114.750595:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881114.750597:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881114.750601:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780711168, portal 10 02000000:00000001:3.0:1788881114.750602:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788881114.750605:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:3.0:1788881114.750606:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881114.750607:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938053248 : -126823771498368 : ffff8ca78a269880) 00000400:00000200:2.0:1788881114.750609:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 02000000:00000001:3.0:1788881114.750613:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881114.750618:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788881114.750618:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780711296:192.168.204.111@tcp:400: 00000400:00000200:2.0:1788881114.750624:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788881114.750626:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788881114.750630:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:2.0:1788881114.750631:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:0.0:1788881114.750633:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788881114.750634:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881114.750639:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:3.0:1788881114.750640:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:0.0:1788881114.750642:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000dc289a80. 00000400:00000200:2.0:1788881114.750643:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000010:3.0:1788881114.750646:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000fdf771c7. 00000400:00000200:2.0:1788881114.750648:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000200:0.0:1788881114.750652:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780711168, offset 0 02000000:00000001:3.0:1788881114.750652:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881114.750658:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [633:633:256:4294967295] 192.168.204.111@tcp LPNI seq info [633:633:8:4294967295] 00000400:00000010:3.0:1788881114.750659:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ca22282b. 00000400:00000200:0.0:1788881114.750664:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788881114.750674:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000400:00000200:2.0:1788881114.750680:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000200:3.0:1788881114.750680:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780711296, portal 10 00000100:00000001:3.0:1788881114.750685:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:2.0:1788881114.750688:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881114.750688:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938055936 : -126823771495680 : ffff8ca78a26a300) 00000400:00000200:0.0:1788881114.750690:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000400:00000200:0.0:1788881114.750695:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:2.0:1788881114.750697:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000003139f6ba. 00000100:00000040:3.0:1788881114.750699:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881114.750700:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:0.0:1788881114.750704:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1788881114.750710:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:0.0:1788881114.750711:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000400:00000200:0.0:1788881114.750718:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [634:634:255:4294967295] 192.168.204.111@tcp LPNI seq info [634:634:7:4294967295] 00000100:00000001:3.0:1788881114.750719:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:2.0:1788881114.750725:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:0.0:1788881114.750729:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:2.0:1788881114.750733:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000003139f6ba type 1, nob 320 niov 1 nkiov 1 00000100:00000040:3.0:1788881114.750734:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:3.0:1788881114.750738:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000d99a9c45. 00000800:00000200:0.0:1788881114.750739:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:3.0:1788881114.750743:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780711296, offset 0 00000800:00000010:0.0:1788881114.750744:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000005f53e16f. 00000100:00000001:2.0:1788881114.750748:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788881114.750751:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788881114.750754:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881114.750760:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306750927 00000800:00000200:0.0:1788881114.750765:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000400:00000200:3.0:1788881114.750765:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000010:2.0:1788881114.750767:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001e508d7a. 00000800:00000200:0.0:1788881114.750771:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000005f53e16f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788881114.750773:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788881114.750774:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788881114.750775:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881114.750780:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000040:0.0:1788881114.750781:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 136328 usecs (137 jiffies) 00000100:00000001:2.0:1788881114.750781:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306750927 00000100:00000001:0.0:1788881114.750786:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306745993 00000100:00000010:2.0:1788881114.750787:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000043ac658c. 00000100:00000001:0.0:1788881114.750791:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306750927 00000100:00000001:2.0:1788881114.750792:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000800:00000200:1.0:1788881114.750793:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:3.0:1788881114.750794:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788881114.750797:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306745993 00000800:00000010:1.0:1788881114.750799:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000003139f6ba. 00000100:00000001:2.0:1788881114.750800:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306750927 00000400:00000200:3.0:1788881114.750801:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000001:0.0:1788881114.750803:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306750927 00000400:00000200:1.0:1788881114.750804:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881114.750809:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306745993 00000400:00000200:3.0:1788881114.750810:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [635:635:254:4294967295] 192.168.204.111@tcp LPNI seq info [635:635:6:4294967295] 00000100:00000010:2.0:1788881114.750812:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001c99cc67. 00000400:00000200:1.0:1788881114.750814:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881114.750824:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000400:00000200:3.0:1788881114.750829:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788881114.750830:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306750927 00000100:00000001:2.0:1788881114.750830:0:201288:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306750927 00000100:00000001:0.0:1788881114.750837:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306745993 00000100:00000010:2.0:1788881114.750837:0:201288:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000ca0fac21. 00000400:00000200:1.0:1788881114.750840:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009c31f261 00000800:00000200:3.0:1788881114.750842:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881114.750844:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009c31f261. 00000100:00000001:2.0:1788881114.750845:0:201288:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000800:00000010:3.0:1788881114.750849:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000d5454cc1. 00000100:00000001:1.0:1788881114.750851:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788881114.750851:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881114.750852:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306750927 00000800:00000200:3.0:1788881114.750857:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788881114.750859:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881114.750859:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881114.750862:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881130 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881114.750864:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881114.750865:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788881114.750869:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788881114.750870:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881114.750871:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788881114.750874:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000d5454cc1 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788881114.750877:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881114.750881:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881114.750881:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881114.750884:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881114.750885:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788881114.750888:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881114.750889:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788881114.750890:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881114.750894:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881114.750894:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788881114.750895:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881130 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881114.750896:0:201286:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10550 usecs (11 jiffies) 00000100:00000001:0.0:1788881114.750898:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881114.750898:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881114.750902:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881114.750903:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306745867 00000100:00000001:2.0:1788881114.750905:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881114.750908:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881114.750909:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306750927 00000100:00000001:3.0:1788881114.750915:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306745867 00000100:00000001:1.0:1788881114.750916:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881114.750919:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306750927 00000100:00000001:1.0:1788881114.750920:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788881114.750924:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306745867 00000100:00000001:3.0:1788881114.750926:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306750927 00000100:00000001:3.0:1788881114.750928:0:201286:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306745867 00000100:00000001:3.0:1788881114.750931:0:201286:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306750927 00000100:00000001:3.0:1788881114.750933:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788881114.750934:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881114.750937:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000005f53e16f. 00000100:00000001:3.0:1788881114.750937:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881114.750940:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881114.750942:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881114.750945:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000dc289a80 00000400:00000010:1.0:1788881114.750946:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000dc289a80. 00000100:00000001:1.0:1788881114.750948:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788881114.750953:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881114.750954:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881114.750960:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881114.750966:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881114.750967:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788881114.750968:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788881114.750971:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881114.750972:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881114.750973:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881114.750976:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881114.750978:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881114.750986:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881114.750988:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788881114.751047:0:201279:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881114.751050:0:201279:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000d5454cc1. 00000400:00000200:1.0:1788881114.751052:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881114.751055:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881114.751057:0:201279:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000d99a9c45 00000400:00000010:1.0:1788881114.751058:0:201279:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000d99a9c45. 00000100:00000001:1.0:1788881114.751060:0:201279:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881114.751062:0:201279:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881114.751078:0:201279:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881114.751080:0:201279:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881114.751083:0:201279:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881114.751084:0:201279:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1788881114.752624:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881114.752633:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881114.752635:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881114.752636:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881114.752641:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881114.752655:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b13480 00000400:00000200:2.0:1788881114.752662:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x1491 [1] + 224 00000800:00000001:2.0:1788881114.752667:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881114.752669:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881114.752671:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881114.752673:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881114.752675:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881114.752678:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881114.752684:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881114.752695:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881114.752700:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881114.752702:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881114.752702:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881114.752703:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881114.752707:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881114.752710:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13500 00000400:00000200:2.0:1788881114.752713:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x1499 [1] + 224 00000800:00000001:2.0:1788881114.752716:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881114.752718:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881114.752719:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788881114.752721:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788881114.752721:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881114.752725:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881114.752726:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881114.752729:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:1.0:1788881114.752730:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881114.752733:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788881114.752735:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b258a24d. 00000100:00000040:2.0:1788881114.752737:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788881114.752739:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000b216ae47 00000400:00000010:1.0:1788881114.752743:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000b216ae47. 00000100:00000001:1.0:1788881114.752746:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788881114.752753:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:1.0:1788881114.752757:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881114.752758:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881114.752762:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:1.0:1788881114.752766:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788881114.752766:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881114.752769:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881114.752776:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881114.752779:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881114.752779:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:1.0:1788881114.752783:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881114.752785:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881114.752787:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788881114.752789:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881114.752790:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13580 02000000:00000001:1.0:1788881114.752795:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881114.752799:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x14a1 [1] + 224 00000100:00000001:0.0:1788881114.752802:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00001000:1.0:1788881114.752802:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=42 v=5 (5 5 5 5) 00000800:00000001:2.0:1788881114.752804:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1788881114.752806:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00001000:1.0:1788881114.752808:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=42 v=5 (1 1 1 1) 00000400:00000001:0.0:1788881114.752811:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:1.0:1788881114.752812:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000800:00000001:2.0:1788881114.752813:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788881114.752815:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000001:1.0:1788881114.752815:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881114.752817:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:0.0:1788881114.752819:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00010000:00000001:1.0:1788881114.752820:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788881114.752823:0:201288:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881114.752826:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000200:2.0:1788881114.752826:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881114.752827:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881114.752830:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881114.752835:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:1.0:1788881114.752836:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881114.752836:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788881114.752841:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881114.752844:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881114.752865:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788881114.752871:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000040:2.0:1788881114.752873:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881114.752875:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881114.752883:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788881114.752906:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:fa4856d5-51a8-499e-b3f8-b477743a3dec:201288:1875776780711040:192.168.204.111@tcp:400: 00000100:00000200:0.0:1788881114.752908:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881114.752911:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881114.752918:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881114.752919:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788881114.752923:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788881114.752925:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881114.752925:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788881114.752927:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780711040/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881114.752930:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881114.752936:0:201285:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788881114.752938:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881114.752939:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881114.752939:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788881114.752941:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1788881114.752942:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881114.752942:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:1.0:1788881114.752945:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000008e3e56e7. 00000100:00000001:0.0:1788881114.752946:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:3.0:1788881114.752946:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788881114.752950:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:0.0:1788881114.752951:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881114.752951:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000010:3.0:1788881114.752952:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ca22282b. 00000020:00000001:1.0:1788881114.752955:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000400:00000200:3.0:1788881114.752957:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000020:00000040:1.0:1788881114.752958:0:201288:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00001000:0.0:1788881114.752959:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=32 v=5 (5 5 5 5) 00000400:00000010:3.0:1788881114.752960:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000020:00000001:1.0:1788881114.752965:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:3.0:1788881114.752965:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00001000:0.0:1788881114.752966:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=32 v=5 (1 1 1 1) 02000000:00000010:1.0:1788881114.752969:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c93f60a2. 00000100:00000200:3.0:1788881114.752971:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881114.752977:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000200:3.0:1788881114.752980:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881114.752981:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:0.0:1788881114.752982:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881114.752985:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788881114.752985:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000001:1.0:1788881114.752990:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00010000:00000001:0.0:1788881114.752991:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000100:00000001:1.0:1788881114.752994:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881114.752995:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:0.0:1788881114.752996:0:201285:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881114.752997:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881114.752997:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881114.753001:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:3.0:1788881114.753002:0:201286:0:(client.c:1580:after_reply()) Process entered 00000100:00000001:1.0:1788881114.753004:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1788881114.753006:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 02000000:00000001:3.0:1788881114.753006:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881114.753007:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881114.753007:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881114.753010:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881114.753012:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788881114.753012:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1788881114.753013:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881114.753015:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881114.753016:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788881114.753017:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=42 v=5 (5 5 5 5) 00000100:00001000:3.0:1788881114.753022:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=126 v=5 (1 1 0 0) 00000100:00000040:0.0:1788881114.753024:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881114.753025:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881114.753028:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881114.753031:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788881114.753036:0:201286:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881114.753039:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881114.753043:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881114.753046:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1788881114.753047:0:201286:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881114.753051:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881114.753057:0:201286:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788881114.753059:0:201286:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788881114.753062:0:201286:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788881114.753077:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881114.753078:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780711168:192.168.204.111@tcp:400: 00000100:00000040:3.0:1788881114.753084:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881114.753085:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881114.753090:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:3.0:1788881114.753093:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000040:0.0:1788881114.753096:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000663051e5 x1875776780711168/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881114.753097:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881114.753099:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881114.753104:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788881114.753112:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000e2be23b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ea27006f-52a8-4007-b69a-dcac8279445a:201286:1875776780711296:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881114.753115:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881114.753118:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881114.753121:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e2be23b2 x1875776780711296/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881130 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881114.753137:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:3.0:1788881114.753138:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881114.753142:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000001:3.0:1788881114.753142:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788881114.753144:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000fdf771c7. 02000000:00000010:0.0:1788881114.753145:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000784e3b9. 02000000:00000001:3.0:1788881114.753147:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881114.753149:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881114.753151:0:201286:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:0.0:1788881114.753152:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881114.753153:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 00000020:00000001:0.0:1788881114.753156:0:201285:0:(genops.c:1238:class_import_put()) Process entered 02000000:00000010:3.0:1788881114.753156:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000004e0f6846. 00000020:00000040:0.0:1788881114.753159:0:201285:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 02000000:00000001:3.0:1788881114.753161:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000020:00000001:0.0:1788881114.753164:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:3.0:1788881114.753164:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000010:0.0:1788881114.753168:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f0a0057b. 00000100:00000010:3.0:1788881114.753168:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e2be23b2. 00000100:00000001:3.0:1788881114.753172:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 02000000:00000001:0.0:1788881114.753174:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788881114.753175:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1788881114.753178:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881114.753179:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000663051e5. 00000100:00000001:3.0:1788881114.753181:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881114.753185:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881114.753186:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881114.753188:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881114.753189:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881114.753195:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881114.753195:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881114.753200:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881114.753201:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881114.753204:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881114.753208:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881114.753211:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881114.814343:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881114.814348:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881114.814354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881114.814355:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881114.814370:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788881114.814375:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788881114.814378:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881115.838381:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881115.838390:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881115.838398:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881115.838399:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881115.838469:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881115.838473:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881115.838476:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881116.862344:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881116.862351:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881116.862353:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881116.862355:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881116.862357:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881116.862359:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881116.862361:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881116.862362:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881116.862365:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881116.862366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881116.862467:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788881116.862470:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788881116.862471:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881117.886328:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881117.886335:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881117.886341:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881117.886343:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881117.886364:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881117.886369:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881117.886372:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881118.910348:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881118.910354:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881118.910360:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881118.910361:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881118.910415:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788881118.910418:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788881118.910420:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881119.870349:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306750927, at 4306750976 00000100:00000010:0.0:1788881119.870366:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001e508d7a. 00000100:00000040:2.0:1788881119.870380:0:192584:0:(connection.c:75:cpu_latency_work()) work item of 00000000cc716df0 (cpu 0) has reached its deadline 4306750927, at 4306750976 00000100:00000040:1.0:1788881119.870390:0:205033:0:(connection.c:75:cpu_latency_work()) work item of 000000006c51c1b9 (cpu 0) has reached its deadline 4306750927, at 4306750976 00000100:00000040:0.0:1788881119.870391:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788881119.870391:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306750927, at 4306750976 00000100:00000010:2.0:1788881119.870397:0:192584:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000001c99cc67. 00000100:00000001:0.0:1788881119.870409:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881119.870412:0:205033:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000043ac658c. 00000100:00000010:3.0:1788881119.870415:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000ca0fac21. 00000100:00000010:0.0:1788881119.870416:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000000802a821. 00000020:00000040:0.0:1788881119.870425:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:0.0:1788881119.870429:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788881119.870431:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788881119.870435:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788881119.870441:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000ae4abc27. 00000100:00000001:0.0:1788881119.870446:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881119.870458:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788881119.870466:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788881119.870467:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881119.870483:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000802a821] to pc [ptlrpcd_00_02+2] req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881119.870503:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881119.870504:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788881119.870506:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788881119.870508:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000937cf312. 00000020:00000040:0.0:1788881119.870509:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881119.870511:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788881119.870512:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788881119.870514:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788881119.870516:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000bd68c00a. 00000100:00000001:0.0:1788881119.870521:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881119.870525:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788881119.870532:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881119.870533:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788881119.870537:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881119.870539:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881119.870543:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881119.870543:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788881119.870544:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000937cf312] to pc [ptlrpcd_00_03+3] req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881119.870551:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881119.870558:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881119.870560:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881119.870562:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881119.870565:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1788881119.870568:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881119.870571:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788881119.870581:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788881119.870585:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000010:0.0:1788881119.870586:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 0000000007f91037. 00000100:00000040:1.0:1788881119.870589:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881119.870591:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788881119.870592:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 02000000:00000001:3.0:1788881119.870600:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788881119.870604:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788881119.870605:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881119.870609:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788881119.870615:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881119.870616:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788881119.870625:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000044c69dd2. 00000100:00000001:1.0:1788881119.870631:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00100000:3.0:1788881119.870631:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780711424:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881119.870633:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881119.870636:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881119.870644:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 02000000:00000001:1.0:1788881119.870645:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:3.0:1788881119.870645:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788881119.870653:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788881119.870656:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788881119.870659:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881119.870663:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881119.870663:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788881119.870672:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000746bfd07. 00000100:00100000:1.0:1788881119.870676:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780711552:192.168.204.111@tcp:400: 02000000:00000001:3.0:1788881119.870678:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881119.870687:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ca22282b. 00000100:00000001:1.0:1788881119.870690:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:3.0:1788881119.870693:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 02000000:00000001:1.0:1788881119.870698:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000200:3.0:1788881119.870700:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780711424, portal 25 02000000:00000001:1.0:1788881119.870702:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881119.870703:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788881119.870704:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788881119.870708:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881119.870709:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849370368 : -126822860181248 : ffff8ca7c0783100) 02000000:00000001:1.0:1788881119.870711:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788881119.870719:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000007f91037] to pc [ptlrpcd_00_00+0] req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788881119.870719:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881135 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:1.0:1788881119.870729:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b17a4842. 02000000:00000001:1.0:1788881119.870739:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881119.870752:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000010:1.0:1788881119.870755:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000004b6baab5. 00000100:00000040:3.0:1788881119.870763:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881119.870769:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000058a716d6. 00000400:00000010:3.0:1788881119.870769:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000003539bb5a. 00000100:00000001:0.0:1788881119.870775:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881119.870782:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000200:1.0:1788881119.870785:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780711552, portal 10 00000100:00000200:3.0:1788881119.870786:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780711424, offset 0 00000100:00000040:0.0:1788881119.870787:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1788881119.870796:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788881119.870800:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849367680 : -126822860183936 : ffff8ca7c0782680) 00000100:00000040:0.0:1788881119.870803:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12088) 00000100:00000001:2.0:1788881119.870809:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:3.0:1788881119.870810:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:2.0:1788881119.870817:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:1.0:1788881119.870819:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881119.870821:0:201285:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:3.0:1788881119.870830:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000040:2.0:1788881119.870832:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881119.870833:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881119.870840:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1788881119.870842:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881119.870848:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788881119.870855:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000010:1.0:1788881119.870862:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000000ae22117. 00000400:00000200:3.0:1788881119.870863:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000200:1.0:1788881119.870872:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780711552, offset 0 00000100:00000001:2.0:1788881119.870875:0:201285:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788881119.870880:0:201285:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881119.870881:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788881119.870888:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788881119.870888:0:201285:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:3.0:1788881119.870894:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:2.0:1788881119.870896:0:201285:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1788881119.870903:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [636:636:256:4294967295] 192.168.204.111@tcp LPNI seq info [636:636:8:4294967295] 02000000:00000001:2.0:1788881119.870905:0:201285:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788881119.870913:0:201285:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780711680:192.168.204.111@tcp:400: 00000400:00000200:3.0:1788881119.870926:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788881119.870931:0:201285:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788881119.870938:0:201285:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000800:00000200:3.0:1788881119.870938:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788881119.870942:0:201285:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881119.870944:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:3.0:1788881119.870946:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000024dd99c. 02000000:00000001:2.0:1788881119.870947:0:201285:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788881119.870954:0:201285:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000e84cf675. 00000400:00000200:1.0:1788881119.870958:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000001:2.0:1788881119.870960:0:201285:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:2.0:1788881119.870981:0:201285:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ffb4c8a2. 00000800:00000200:3.0:1788881119.870982:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:1.0:1788881119.870983:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000010:2.0:1788881119.870989:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000170fdf47. 00000100:00000200:2.0:1788881119.870997:0:201285:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780711680, portal 10 00000400:00000200:1.0:1788881119.870999:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788881119.871006:0:201285:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000800:00000200:3.0:1788881119.871006:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:2.0:1788881119.871012:0:201285:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617250849371264 : -126822860180352 : ffff8ca7c0783480) 00000400:00000200:1.0:1788881119.871014:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788881119.871016:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000024dd99c type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788881119.871026:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [637:637:255:4294967295] 192.168.204.111@tcp LPNI seq info [637:637:7:4294967295] 00000100:00000040:2.0:1788881119.871031:0:201285:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881119.871036:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881119.871049:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306756047 00000400:00000200:1.0:1788881119.871064:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:2.0:1788881119.871064:0:201285:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000010:3.0:1788881119.871064:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000625b9f8b. 00000100:00000001:3.0:1788881119.871078:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000100:00000040:2.0:1788881119.871079:0:201285:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000100:00000001:3.0:1788881119.871085:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306756047 00000400:00000010:2.0:1788881119.871086:0:201285:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000007d208f26. 00000800:00000200:1.0:1788881119.871091:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000200:2.0:1788881119.871098:0:201285:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780711680, offset 0 00000100:00000010:3.0:1788881119.871099:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000029868be. 00000800:00000010:1.0:1788881119.871100:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c93f60a2. 00000100:00000001:3.0:1788881119.871110:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 1 00000100:00000001:3.0:1788881119.871119:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306756047 00000800:00000200:1.0:1788881119.871120:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:2.0:1788881119.871123:0:201285:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000010:3.0:1788881119.871126:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000014c076ca. 00000100:00000001:3.0:1788881119.871130:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 2 00000800:00000200:0.0:1788881119.871133:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:2.0:1788881119.871138:0:201285:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:3.0:1788881119.871140:0:201287:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306756047 00000800:00000010:0.0:1788881119.871143:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000024dd99c. 00000100:00000010:3.0:1788881119.871150:0:201287:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000dc3cc638. 00000800:00000200:1.0:1788881119.871157:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788881119.871158:0:201287:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000400:00000200:0.0:1788881119.871159:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881119.871159:0:201285:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1788881119.871168:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:1.0:1788881119.871170:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c93f60a2 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:0.0:1788881119.871176:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:2.0:1788881119.871176:0:201285:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:3.0:1788881119.871191:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881119.871194:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881119.871197:0:201285:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:3.0:1788881119.871198:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788881119.871206:0:201285:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:1.0:1788881119.871210:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 135635 usecs (136 jiffies) 00000400:00000200:2.0:1788881119.871212:0:201285:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [638:638:254:4294967295] 192.168.204.111@tcp LPNI seq info [638:638:6:4294967295] 00000100:00000001:1.0:1788881119.871216:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306751112 00000100:00000001:1.0:1788881119.871222:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306756047 00000100:00000001:1.0:1788881119.871227:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306751112 00000400:00000200:2.0:1788881119.871228:0:201285:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000400:00000200:0.0:1788881119.871266:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000003539bb5a 00000800:00000200:2.0:1788881119.871267:0:201285:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:0.0:1788881119.871270:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000003539bb5a. 00000100:00000001:1.0:1788881119.871272:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306756047 00000800:00000010:2.0:1788881119.871279:0:201285:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c52725cf. 00000100:00000001:1.0:1788881119.871280:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306751113 00000100:00000001:0.0:1788881119.871281:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:3.0:1788881119.871282:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881119.871291:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306756047 00000100:00000001:3.0:1788881119.871294:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:0.0:1788881119.871295:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881135 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788881119.871304:0:201285:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788881119.871305:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881119.871306:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306751113 00000100:00000001:3.0:1788881119.871315:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881119.871320:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306756047 00000100:00000001:3.0:1788881119.871320:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000200:2.0:1788881119.871331:0:201285:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788881119.871333:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788881119.871333:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881119.871333:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881119.871344:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881119.871348:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881135 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000200:2.0:1788881119.871349:0:201285:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 00000000c52725cf type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788881119.871350:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881119.871363:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881119.871364:0:201285:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881119.871366:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881119.871373:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881119.871374:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000040:2.0:1788881119.871375:0:201285:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10510 usecs (11 jiffies) 00000100:00000001:2.0:1788881119.871385:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306750988 00000100:00000001:1.0:1788881119.871389:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881119.871389:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306756047 00000100:00000001:1.0:1788881119.871393:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788881119.871393:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306750988 00000800:00000200:0.0:1788881119.871395:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788881119.871399:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000010:0.0:1788881119.871405:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c93f60a2. 00000100:00000001:2.0:1788881119.871406:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306756047 00000100:00000001:1.0:1788881119.871407:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:0.0:1788881119.871413:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:2.0:1788881119.871414:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306750988 00000100:00000001:1.0:1788881119.871415:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881119.871428:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881119.871428:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306756047 00000400:00000200:0.0:1788881119.871432:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881119.871441:0:201285:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306750988 00000400:00000200:0.0:1788881119.871442:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000000ae22117 00000400:00000010:0.0:1788881119.871446:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000000ae22117. 00000100:00000001:2.0:1788881119.871447:0:201285:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306756047 00000100:00000001:2.0:1788881119.871451:0:201285:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881119.871457:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:2.0:1788881119.871458:0:201285:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881119.871464:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881119.871465:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881119.871476:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881119.871485:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788881119.871485:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881119.871490:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000040:0.0:1788881119.871497:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881119.871499:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881119.871503:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881119.871507:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881119.871516:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881119.871525:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881119.871527:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788881119.871552:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788881119.871554:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c52725cf. 00000400:00000200:0.0:1788881119.871556:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788881119.871558:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788881119.871560:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000007d208f26 00000400:00000010:0.0:1788881119.871561:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000007d208f26. 00000100:00000001:0.0:1788881119.871562:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788881119.871564:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881119.871569:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881119.871570:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881119.871574:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881119.871575:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788881119.872408:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881119.872423:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881119.872427:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881119.872429:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881119.872440:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881119.872456:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b13600 00000400:00000200:1.0:1788881119.872466:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x14a9 [1] + 224 00000800:00000001:1.0:1788881119.872473:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881119.872482:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881119.872485:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881119.872490:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881119.872496:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881119.872501:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881119.872512:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881119.872532:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881119.872541:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881119.872545:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881119.872565:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881119.872570:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788881119.872575:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788881119.872577:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788881119.872580:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ca22282b. 00000400:00000200:3.0:1788881119.872584:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000400:00000010:3.0:1788881119.872585:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:3.0:1788881119.872588:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788881119.872594:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788881119.872609:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881119.872616:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788881119.872617:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881119.872620:0:201287:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1788881119.872622:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788881119.872625:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788881119.872627:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881119.872629:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788881119.872635:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=47 v=5 (5 5 5 5) 00000100:00001000:3.0:1788881119.872638:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=47 v=5 (1 1 1 1) 00000100:00000001:3.0:1788881119.872641:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788881119.872642:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881119.872644:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788881119.872645:0:201287:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881119.872647:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881119.872650:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881119.872662:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881119.872664:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881119.872665:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881119.872668:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788881119.872672:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000000802a821 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:fa4856d5-51a8-499e-b3f8-b477743a3dec:201287:1875776780711424:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881119.872675:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881119.872676:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881119.872677:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000000802a821 x1875776780711424/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881119.872681:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788881119.872683:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788881119.872685:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000746bfd07. 02000000:00000001:3.0:1788881119.872686:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881119.872688:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881119.872745:0:201287:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000020:00000001:3.0:1788881119.872747:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881119.872749:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000ae4abc27. 02000000:00000001:3.0:1788881119.872752:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881119.872753:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881119.872754:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000000802a821. 00000100:00000001:3.0:1788881119.872756:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881119.872757:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881119.872758:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881119.872762:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881119.872763:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881119.872765:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881119.872766:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881119.872784:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881119.872794:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881119.872796:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881119.872798:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881119.872804:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881119.872812:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13680 00000400:00000200:2.0:1788881119.872817:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x14ad [1] + 224 00000800:00000001:2.0:1788881119.872822:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881119.872826:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881119.872829:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788881119.872832:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:2.0:1788881119.872835:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788881119.872839:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881119.872853:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881119.872867:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788881119.872873:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881119.872877:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788881119.872879:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881119.872881:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881119.872885:0:201278:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:2.0:1788881119.872890:0:201278:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13700 00000400:00000200:2.0:1788881119.872895:0:201278:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x14b9 [1] + 224 00000800:00000001:2.0:1788881119.872899:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788881119.872902:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881119.872906:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788881119.872907:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881119.872911:0:201278:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:1.0:1788881119.872912:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788881119.872916:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1788881119.872919:0:201278:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000010:1.0:1788881119.872920:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000004b6baab5. 00000400:00000200:1.0:1788881119.872925:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000058a716d6 00000400:00000010:1.0:1788881119.872927:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000058a716d6. 00000100:00000001:1.0:1788881119.872931:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788881119.872932:0:201278:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881119.872936:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881119.872939:0:201278:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881119.872974:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788881119.872977:0:201278:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881119.872998:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881119.873001:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881119.873004:0:201288:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1788881119.873006:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881119.873008:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788881119.873010:0:201278:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788881119.873011:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881119.873015:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788881119.873016:0:201278:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788881119.873020:0:201278:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:1.0:1788881119.873024:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=37 v=5 (5 5 5 5) 00000100:00001000:1.0:1788881119.873030:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=37 v=5 (1 1 1 1) 00000100:00000001:1.0:1788881119.873034:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788881119.873036:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881119.873039:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788881119.873044:0:201288:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881119.873048:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000001:0.0:1788881119.873051:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881119.873052:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:1.0:1788881119.873058:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788881119.873059:0:201285:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788881119.873062:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1788881119.873066:0:201285:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788881119.873069:0:201285:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00000040:1.0:1788881119.873071:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788881119.873074:0:201285:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ffb4c8a2. 00000400:00000200:0.0:1788881119.873083:0:201285:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000170fdf47 00000100:00000001:1.0:1788881119.873090:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1788881119.873091:0:201285:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000170fdf47. 00000100:00000001:1.0:1788881119.873101:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881119.873102:0:201285:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788881119.873108:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788881119.873112:0:201285:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881119.873126:0:201285:0:(events.c:102:reply_in_callback()) @@@ unlink req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881119.873132:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881119.873180:0:201285:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881119.873186:0:201285:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:1.0:1788881119.873186:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000937cf312 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780711552:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881119.873195:0:201285:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881119.873201:0:201285:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788881119.873204:0:201285:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788881119.873205:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881119.873206:0:201285:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881119.873219:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 02000000:00000001:0.0:1788881119.873220:0:201285:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881119.873229:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000937cf312 x1875776780711552/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1788881119.873230:0:201285:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=47 v=5 (5 5 5 5) 00000100:00001000:0.0:1788881119.873259:0:201285:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=131 v=5 (1 1 0 0) 00000100:00000001:0.0:1788881119.873261:0:201285:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788881119.873262:0:201285:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788881119.873264:0:201285:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788881119.873267:0:201285:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788881119.873331:0:201285:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881119.873332:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000001:0.0:1788881119.873336:0:201285:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:1.0:1788881119.873337:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000040:0.0:1788881119.873339:0:201285:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:0.0:1788881119.873344:0:201285:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 02000000:00000010:1.0:1788881119.873344:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b17a4842. 00000100:00000001:0.0:1788881119.873349:0:201285:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788881119.873354:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:0.0:1788881119.873355:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:1.0:1788881119.873359:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1788881119.873363:0:201288:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:0.0:1788881119.873366:0:201285:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881119.873372:0:201285:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000020:00000001:1.0:1788881119.873373:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 00000100:00000001:0.0:1788881119.873374:0:201285:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:1.0:1788881119.873379:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000bd68c00a. 00000100:00000040:0.0:1788881119.873383:0:201285:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881119.873389:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00100000:0.0:1788881119.873392:0:201285:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@0000000007f91037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ea27006f-52a8-4007-b69a-dcac8279445a:201285:1875776780711680:192.168.204.111@tcp:400: 02000000:00000001:1.0:1788881119.873393:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788881119.873401:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000937cf312. 00000100:00000001:0.0:1788881119.873403:0:201285:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881119.873408:0:201285:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788881119.873411:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000040:0.0:1788881119.873414:0:201285:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@0000000007f91037 x1875776780711680/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881135 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881119.873416:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881119.873425:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881119.873437:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788881119.873440:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881119.873448:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881119.873455:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881119.873456:0:201285:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881119.873459:0:201285:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788881119.873464:0:201285:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000e84cf675. 02000000:00000001:0.0:1788881119.873467:0:201285:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788881119.873469:0:201285:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1788881119.873471:0:201285:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000020:00000001:0.0:1788881119.873473:0:201285:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881119.873475:0:201285:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000044c69dd2. 02000000:00000001:0.0:1788881119.873477:0:201285:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788881119.873478:0:201285:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788881119.873479:0:201285:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 0000000007f91037. 00000100:00000001:0.0:1788881119.873480:0:201285:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788881119.873481:0:201285:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881119.873482:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881119.873485:0:201285:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881119.873486:0:201285:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881119.873487:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881119.873489:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881119.934359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881119.934368:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881119.934374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881119.934378:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881119.934380:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881119.934383:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881119.934386:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881119.934389:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881119.934391:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881119.934394:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788881119.934423:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788881119.934431:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788881119.934436:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881120.958362:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881120.958370:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881120.958375:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881120.958377:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881120.958437:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788881120.958442:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788881120.958444:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881121.982297:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881121.982305:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881121.982311:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881121.982312:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881121.982376:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881121.982380:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881121.982383:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881123.006332:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881123.006339:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881123.006345:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881123.006346:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881123.006445:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788881123.006453:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788881123.006458:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881124.030380:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881124.030390:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881124.030396:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881124.030398:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881124.030461:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881124.030466:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881124.030470:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788881124.990304:0:207049:0:(connection.c:75:cpu_latency_work()) work item of 0000000037056c79 (cpu 0) has reached its deadline 4306756047, at 4306756096 00000100:00000010:0.0:1788881124.990324:0:207049:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000625b9f8b. 00000100:00000040:3.0:1788881124.990336:0:207781:0:(connection.c:75:cpu_latency_work()) work item of 000000006a58f0d2 (cpu 0) has reached its deadline 4306756047, at 4306756096 00000100:00000010:3.0:1788881124.990343:0:207781:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000dc3cc638. 00000100:00000040:1.0:1788881124.990344:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881124.990352:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881124.990360:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000eb1c98ea. 00000020:00000040:1.0:1788881124.990366:0:53635:0:(genops.c:1231:class_import_get()) import 000000003ccc7c76 refcount=4 obd=MGC192.168.204.111@tcp 00000100:00000001:1.0:1788881124.990372:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881124.990376:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788881124.990381:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788881124.990390:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000c93f60a2. 00000100:00000001:1.0:1788881124.990399:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881124.990412:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging fa4856d5-51a8-499e-b3f8-b477743a3dec->MGS req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788881124.990424:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788881124.990427:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881124.990433:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000eb1c98ea] to pc [ptlrpcd_00_01+1] req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881124.990457:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881124.990460:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788881124.990464:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788881124.990467:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000002b387587. 00000020:00000040:1.0:1788881124.990470:0:53635:0:(genops.c:1231:class_import_get()) import 0000000095473c20 refcount=4 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881124.990473:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788881124.990475:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788881124.990476:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:1.0:1788881124.990477:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881124.990480:0:201285:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881124.990481:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 02000000:00000010:1.0:1788881124.990483:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000f0a0057b. 00000100:00000001:0.0:1788881124.990485:0:201286:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1788881124.990486:0:201285:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881124.990489:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881124.990492:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881124.990498:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0000_UUID req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788881124.990504:0:201286:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788881124.990508:0:201286:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881124.990513:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788881124.990516:0:201286:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788881124.990516:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881124.990520:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002b387587] to pc [ptlrpcd_00_02+2] req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788881124.990527:0:201286:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881124.990533:0:201286:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788881124.990537:0:201286:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780711808:192.168.204.111@tcp:400: 00000100:00000001:0.0:1788881124.990543:0:201286:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788881124.990552:0:201286:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:1.0:1788881124.990552:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881124.990555:0:201286:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881124.990557:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788881124.990558:0:201286:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788881124.990564:0:201286:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000482afcf7. 00000100:00000001:1.0:1788881124.990564:0:53635:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:3.0:1788881124.990564:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:0.0:1788881124.990569:0:201286:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788881124.990569:0:53635:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000663051e5. 00000100:00000001:3.0:1788881124.990570:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881124.990574:0:201287:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000020:00000040:1.0:1788881124.990575:0:53635:0:(genops.c:1231:class_import_get()) import 00000000b98b8a4e refcount=4 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:1.0:1788881124.990578:0:53635:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 00000400:00000010:0.0:1788881124.990580:0:201286:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000a2a2ca94. 02000000:00000001:1.0:1788881124.990580:0:53635:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:2.0:1788881124.990581:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:3.0:1788881124.990583:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1788881124.990584:0:53635:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881124.990587:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788881124.990588:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000009d1fb96a. 02000000:00000010:1.0:1788881124.990590:0:53635:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000004e0f6846. 00000100:00000001:1.0:1788881124.990596:0:53635:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881124.990598:0:201287:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000200:0.0:1788881124.990600:0:201286:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780711808, portal 25 00000100:00000001:3.0:1788881124.990602:0:201287:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881124.990603:0:53635:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging ea27006f-52a8-4007-b69a-dcac8279445a->lustre-MDT0001_UUID req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788881124.990605:0:201286:0:(client.c:3275:ptlrpc_request_addref()) Process entered 02000000:00000001:3.0:1788881124.990608:0:201287:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788881124.990609:0:201286:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938060416 : -126823771491200 : ffff8ca78a26b480) 02000000:00000001:3.0:1788881124.990611:0:201287:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788881124.990618:0:201287:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788881124.990621:0:201286:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881140 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881124.990623:0:53635:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00100000:3.0:1788881124.990625:0:201287:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780711936:192.168.204.111@tcp:400: 00000100:00000001:1.0:1788881124.990626:0:53635:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788881124.990636:0:53635:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000663051e5] to pc [ptlrpcd_00_03+3] req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881124.990636:0:201287:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788881124.990640:0:201287:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1788881124.990642:0:201286:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:3.0:1788881124.990644:0:201287:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788881124.990649:0:201287:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000040:0.0:1788881124.990653:0:201286:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000010:3.0:1788881124.990654:0:201287:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000b31f2b6a. 00000400:00000010:0.0:1788881124.990657:0:201286:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000f4d84419. 02000000:00000001:3.0:1788881124.990658:0:201287:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788881124.990663:0:201286:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875776780711808, offset 0 00000100:00000001:1.0:1788881124.990665:0:53635:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881124.990665:0:201287:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 00000000ca22282b. 00000100:00000040:1.0:1788881124.990669:0:53635:0:(pinger.c:229:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:0.0:1788881124.990671:0:201286:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000040:1.0:1788881124.990676:0:53635:0:(pinger.c:247:ptlrpc_pinger_process_import()) ea27006f-52a8-4007-b69a-dcac8279445a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788881124.990680:0:53635:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12093) 00000400:00000010:3.0:1788881124.990682:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000819f63c4. 00000100:00000001:2.0:1788881124.990684:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881124.990688:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:3.0:1788881124.990688:0:201287:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780711936, portal 10 00000400:00000200:0.0:1788881124.990689:0:201286:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000001:2.0:1788881124.990691:0:201288:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788881124.990693:0:201287:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788881124.990696:0:201287:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938057728 : -126823771493888 : ffff8ca78a26aa00) 00000100:00000040:2.0:1788881124.990697:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788881124.990699:0:201286:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788881124.990707:0:201286:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000100:00000001:2.0:1788881124.990709:0:201288:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000040:3.0:1788881124.990709:0:201287:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881124.990711:0:201288:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881124.990714:0:201286:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 02000000:00000001:2.0:1788881124.990716:0:201288:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:0.0:1788881124.990719:0:201286:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 02000000:00000001:2.0:1788881124.990720:0:201288:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788881124.990725:0:201288:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788881124.990727:0:201286:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [639:639:256:4294967295] 192.168.204.111@tcp LPNI seq info [639:639:8:4294967295] 00000100:00100000:2.0:1788881124.990732:0:201288:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780712064:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881124.990734:0:201287:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788881124.990738:0:201288:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788881124.990741:0:201288:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:3.0:1788881124.990741:0:201287:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 02000000:00000001:2.0:1788881124.990744:0:201288:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788881124.990747:0:201287:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000e78d7264. 00000400:00000200:0.0:1788881124.990748:0:201286:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 02000000:00000001:2.0:1788881124.990748:0:201288:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:3.0:1788881124.990753:0:201287:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780711936, offset 0 02000000:00000010:2.0:1788881124.990754:0:201288:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 0000000099f3ac40. 02000000:00000001:2.0:1788881124.990758:0:201288:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788881124.990759:0:201286:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788881124.990762:0:201288:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at 000000001a8af943. 00000800:00000010:0.0:1788881124.990764:0:201286:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000044c69dd2. 00000400:00000200:3.0:1788881124.990765:0:201287:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000400:00000010:2.0:1788881124.990774:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000c013629f. 00000800:00000200:0.0:1788881124.990776:0:201286:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000400:00000200:3.0:1788881124.990776:0:201287:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000100:00000200:2.0:1788881124.990778:0:201288:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875776780712064, portal 10 00000100:00000001:2.0:1788881124.990782:0:201288:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1788881124.990783:0:201287:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788881124.990784:0:201288:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446617249938053248 : -126823771498368 : ffff8ca78a269880) 00000400:00000200:3.0:1788881124.990788:0:201287:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000800:00000200:0.0:1788881124.990791:0:201286:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000040:2.0:1788881124.990791:0:201288:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788881124.990794:0:201287:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000100:00000001:2.0:1788881124.990802:0:201288:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788881124.990804:0:201286:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000044c69dd2 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:3.0:1788881124.990804:0:201287:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000100:00000040:2.0:1788881124.990808:0:201288:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.111@tcp 00000400:00000200:3.0:1788881124.990812:0:201287:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [640:640:255:4294967295] 192.168.204.111@tcp LPNI seq info [640:640:7:4294967295] 00000400:00000010:2.0:1788881124.990813:0:201288:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000032b7316b. 00000100:00000200:2.0:1788881124.990817:0:201288:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875776780712064, offset 0 00000400:00000200:2.0:1788881124.990823:0:201288:0:(lib-move.c:5122:LNetPut()) LNetPut -> 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881124.990826:0:201286:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881124.990828:0:201287:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000800:00000200:3.0:1788881124.990833:0:201287:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000001:0.0:1788881124.990834:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 0 is 4306761167 00000400:00000200:2.0:1788881124.990837:0:201288:0:(lib-move.c:2778:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.204.111@tcp local destination 00000800:00000010:3.0:1788881124.990837:0:201287:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 0000000032f1ef9f. 00000100:00000010:0.0:1788881124.990841:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000006050354c. 00000400:00000200:2.0:1788881124.990844:0:201288:0:(lib-move.c:2409:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000200:3.0:1788881124.990847:0:201287:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:0.0:1788881124.990850:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 0 00000400:00000200:2.0:1788881124.990850:0:201288:0:(lib-move.c:1590:lnet_get_best_ni()) selected best_ni 192.168.204.11@tcp 00000400:00000200:2.0:1788881124.990856:0:201288:0:(lib-move.c:1143:lnet_select_peer_ni()) 192.168.204.11@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788881124.990856:0:201287:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:0.0:1788881124.990858:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 1 is 4306761167 00000400:00000200:2.0:1788881124.990861:0:201288:0:(lib-move.c:1212:lnet_select_peer_ni()) sd_best_lpni = 192.168.204.111@tcp 00000800:00000200:3.0:1788881124.990861:0:201287:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 0000000032f1ef9f type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1788881124.990876:0:201288:0:(lib-move.c:1710:lnet_handle_send()) 192.168.204.11@tcp NI seq info: [641:641:254:4294967295] 192.168.204.111@tcp LPNI seq info [641:641:6:4294967295] 00000100:00000001:3.0:1788881124.990877:0:201287:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881124.990878:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306756047 00000100:00000001:0.0:1788881124.990885:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 2 is 4306761167 00000100:00000040:3.0:1788881124.990885:0:201287:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: using a timeout of 134947 usecs (135 jiffies) 00000100:00000001:3.0:1788881124.990893:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306756231 00000100:00000001:0.0:1788881124.990894:0:201286:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4306756047 00000400:00000200:2.0:1788881124.990896:0:201288:0:(lib-move.c:1826:lnet_handle_send()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp:) -> 192.168.204.111@tcp(192.168.204.111@tcp:192.168.204.111@tcp) : PUT try# 0 00000100:00000001:0.0:1788881124.990901:0:201286:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS new deadline estimation for cpu 3 is 4306761167 00000800:00000200:1.0:1788881124.990901:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:3.0:1788881124.990901:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306761167 00000800:00000200:2.0:1788881124.990905:0:201288:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.204.111@tcp 00000100:00000010:0.0:1788881124.990909:0:201286:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000001d590ced. 00000800:00000010:1.0:1788881124.990909:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000044c69dd2. 00000100:00000001:3.0:1788881124.990909:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306756231 00000800:00000010:2.0:1788881124.990910:0:201288:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 000000006aca3ff7. 00000100:00000001:0.0:1788881124.990914:0:201286:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.204.111@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788881124.990916:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306761167 00000400:00000200:1.0:1788881124.990917:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788881124.990919:0:201286:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:2.0:1788881124.990919:0:201288:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000b4b9d165] -> 12345-192.168.204.111@tcp (5) 00000100:00000001:3.0:1788881124.990923:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306756231 00000100:00000001:0.0:1788881124.990924:0:201286:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881124.990930:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788881124.990930:0:201288:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.204.111@tcp ip 192.168.204.111:988 00000100:00000001:3.0:1788881124.990930:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306761167 00000400:00000200:1.0:1788881124.990931:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:0.0:1788881124.990933:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788881124.990938:0:201288:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet 000000006aca3ff7 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788881124.990938:0:201287:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306756231 00000400:00000200:1.0:1788881124.990940:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000f4d84419 00000100:00000001:3.0:1788881124.990943:0:201287:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306761167 00000400:00000010:1.0:1788881124.990945:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000f4d84419. 00000100:00000001:0.0:1788881124.990946:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881124.990946:0:201288:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881124.990947:0:201287:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788881124.990949:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788881124.990953:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788881124.990953:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000040:2.0:1788881124.990953:0:201288:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: using a timeout of 10471 usecs (11 jiffies) 00000100:00000001:3.0:1788881124.990954:0:201287:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881124.990957:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788881124.990961:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788881124.990961:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881140 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881124.990961:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881124.990962:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 0 is 4306756107 00000100:00000001:0.0:1788881124.990967:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881124.990968:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881124.990969:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 0, keep current deadline 4306761167 00000100:00000001:2.0:1788881124.990976:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 1 is 4306756107 00000100:00000001:3.0:1788881124.990976:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881124.990980:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 1, keep current deadline 4306761167 00000100:00000001:2.0:1788881124.990983:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 2 is 4306756107 00000100:00000001:3.0:1788881124.990983:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788881124.990984:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:2.0:1788881124.990986:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 2, keep current deadline 4306761167 00000100:00000001:3.0:1788881124.990988:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:1.0:1788881124.990990:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881140 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881124.990991:0:201288:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS new deadline estimation for cpu 3 is 4306756107 00000100:00000001:3.0:1788881124.990991:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788881124.990994:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881124.990995:0:201288:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8ca787527000: PM QoS request active for cpu 3, keep current deadline 4306761167 00000100:00000001:3.0:1788881124.990998:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881124.990999:0:201288:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788881124.991004:0:201288:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881124.991008:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881124.991010:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881124.991011:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788881124.991016:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:2.0:1788881124.991018:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788881124.991020:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788881124.991022:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881124.991023:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788881124.991024:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881124.991025:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788881124.991028:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881124.991032:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 0000000032f1ef9f. 00000400:00000200:1.0:1788881124.991035:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881124.991038:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881124.991040:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000e78d7264 00000400:00000010:1.0:1788881124.991042:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000e78d7264. 00000100:00000001:1.0:1788881124.991044:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881124.991046:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881124.991053:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881124.991055:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881124.991061:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881124.991062:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788881124.991125:0:201280:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788881124.991127:0:201280:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 000000006aca3ff7. 00000400:00000200:1.0:1788881124.991130:0:201280:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881124.991134:0:201280:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:1.0:1788881124.991136:0:201280:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000032b7316b 00000400:00000010:1.0:1788881124.991138:0:201280:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 0000000032b7316b. 00000100:00000001:1.0:1788881124.991140:0:201280:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788881124.991142:0:201280:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881124.991149:0:201280:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788881124.991151:0:201280:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881124.991158:0:201280:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788881124.991160:0:201280:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1788881124.992548:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881124.992561:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881124.992565:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881124.992566:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881124.992573:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000400:00000200:1.0:1788881124.992587:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 25 MB=0x6aa0250b13780 00000400:00000200:1.0:1788881124.992594:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.204.111@tcp of length 224/224 into md 0x14c1 [1] + 224 00000800:00000001:1.0:1788881124.992601:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881124.992605:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881124.992607:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788881124.992611:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000001:1.0:1788881124.992617:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788881124.992622:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881124.992632:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881124.992647:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1788881124.992655:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881124.992658:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788881124.992660:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881124.992664:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1788881124.992668:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788881124.992671:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788881124.992675:0:201286:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1788881124.992675:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13880 00000400:00000200:1.0:1788881124.992682:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x14d1 [1] + 224 00000800:00000001:1.0:1788881124.992686:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881124.992689:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788881124.992690:0:201286:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:1.0:1788881124.992692:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:0.0:1788881124.992693:0:201286:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788881124.992697:0:201286:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000a2a2ca94. 00000400:00000200:1.0:1788881124.992698:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000400:00000200:0.0:1788881124.992704:0:201286:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000009d1fb96a 00000400:00000010:0.0:1788881124.992707:0:201286:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 000000009d1fb96a. 00000100:00000001:0.0:1788881124.992709:0:201286:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:1.0:1788881124.992710:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788881124.992714:0:201286:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788881124.992717:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788881124.992729:0:201286:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788881124.992735:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881124.992752:0:201286:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881124.992755:0:201286:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881124.992759:0:201286:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1788881124.992762:0:201286:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788881124.992763:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788881124.992764:0:201286:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788881124.992768:0:201286:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788881124.992771:0:201286:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881124.992772:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881124.992776:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788881124.992777:0:201286:0:(import.c:2149:obd_at_measure()) add 5 to 000000002a301aa0 time=52 v=5 (5 5 5 5) 00000800:00000001:1.0:1788881124.992779:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788881124.992783:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:0.0:1788881124.992784:0:201286:0:(import.c:2149:obd_at_measure()) add 1 to 000000003cbfdd57 time=52 v=5 (1 1 1 1) 00000100:00000001:0.0:1788881124.992790:0:201286:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000400:00000200:1.0:1788881124.992790:0:201279:0:(lib-move.c:4617:lnet_parse()) TRACE: 192.168.204.11@tcp(192.168.204.11@tcp) <- 192.168.204.111@tcp : PUT - for me 00000100:00000001:0.0:1788881124.992793:0:201286:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881124.992795:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00010000:00000001:0.0:1788881124.992798:0:201286:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00000400:00000200:1.0:1788881124.992799:0:201279:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.204.111@tcp of length 224 into portal 10 MB=0x6aa0250b13800 00000100:00000001:2.0:1788881124.992801:0:201288:0:(client.c:1977:ptlrpc_check_set()) Process entered 00010000:00000001:0.0:1788881124.992802:0:201286:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788881124.992807:0:201279:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.204.111@tcp of length 224/224 into md 0x14c9 [1] + 224 00000100:00000001:0.0:1788881124.992808:0:201286:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881124.992814:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000400:00000001:2.0:1788881124.992815:0:201288:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000040:0.0:1788881124.992817:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:2.0:1788881124.992818:0:201288:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788881124.992821:0:201288:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 000000001a8af943. 00000800:00000001:1.0:1788881124.992823:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788881124.992825:0:201288:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c013629f 00000400:00000200:1.0:1788881124.992828:0:201279:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:2.0:1788881124.992828:0:201288:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000c013629f. 00000100:00000001:2.0:1788881124.992832:0:201288:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:1.0:1788881124.992837:0:201279:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.204.11@tcp->192.168.204.111@tcp: PUT: OK 00000100:00000200:2.0:1788881124.992838:0:201288:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788881124.992845:0:201279:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1788881124.992851:0:201286:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000200:1.0:1788881124.992856:0:201279:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788881124.992856:0:201288:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881124.992857:0:201286:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788881124.992861:0:201286:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788881124.992880:0:201286:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788881124.992888:0:201288:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881124.992890:0:201288:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881124.992894:0:201288:0:(client.c:1580:after_reply()) Process entered 00000100:00000040:1.0:1788881124.992897:0:201279:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881124.992897:0:201288:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788881124.992900:0:201288:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00100000:0.0:1788881124.992901:0:201286:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000eb1c98ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:fa4856d5-51a8-499e-b3f8-b477743a3dec:201286:1875776780711808:192.168.204.111@tcp:400: 00000100:00000001:2.0:1788881124.992903:0:201288:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881124.992908:0:201288:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788881124.992910:0:201286:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:2.0:1788881124.992914:0:201288:0:(import.c:2149:obd_at_measure()) add 5 to 0000000028cb2d55 time=52 v=5 (5 5 5 5) 00000100:00000001:0.0:1788881124.992915:0:201286:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788881124.992919:0:201286:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000eb1c98ea x1875776780711808/t0(0) o400->MGC192.168.204.111@tcp@192.168.204.111@tcp:26/25 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:2.0:1788881124.992920:0:201288:0:(import.c:2149:obd_at_measure()) add 1 to 0000000040f48a4f time=136 v=5 (1 1 0 0) 00000100:00000001:2.0:1788881124.992926:0:201288:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788881124.992928:0:201288:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788881124.992933:0:201288:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1788881124.992939:0:201288:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881124.992948:0:201286:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788881124.992952:0:201286:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 00010000:00000001:2.0:1788881124.992954:0:201288:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788881124.992955:0:201286:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000482afcf7. 00000100:00000001:1.0:1788881124.992956:0:201279:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788881124.992958:0:201288:0:(client.c:3096:ptlrpc_free_committed()) Process entered 02000000:00000001:0.0:1788881124.992960:0:201286:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000040:2.0:1788881124.992961:0:201288:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000020:00000001:0.0:1788881124.992962:0:201286:0:(genops.c:1238:class_import_put()) Process entered 00000800:00000001:1.0:1788881124.992963:0:201279:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788881124.992963:0:201288:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000020:00000040:0.0:1788881124.992965:0:201286:0:(genops.c:1244:class_import_put()) import 000000003ccc7c76 refcount=3 obd=MGC192.168.204.111@tcp 00000100:00000001:2.0:1788881124.992967:0:201288:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1788881124.992969:0:201279:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788881124.992970:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000020:00000001:0.0:1788881124.992974:0:201286:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1788881124.992977:0:201286:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000c93f60a2. 00000100:00000001:3.0:1788881124.992980:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881124.992981:0:201288:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788881124.992984:0:201286:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1788881124.992984:0:201288:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 02000000:00000001:0.0:1788881124.992986:0:201286:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1788881124.992987:0:201288:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:0.0:1788881124.992989:0:201286:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000eb1c98ea. 00000100:00000001:3.0:1788881124.992991:0:201287:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:2.0:1788881124.992993:0:201288:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788881124.992995:0:201286:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000400:00000001:3.0:1788881124.992996:0:201287:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00000001:0.0:1788881124.992997:0:201286:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788881124.993001:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1788881124.993001:0:201287:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000100:00100000:2.0:1788881124.993004:0:201288:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@00000000663051e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ea27006f-52a8-4007-b69a-dcac8279445a:201288:1875776780712064:192.168.204.111@tcp:400: 00000400:00000010:3.0:1788881124.993005:0:201287:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at 00000000ca22282b. 00000100:00000001:0.0:1788881124.993008:0:201286:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788881124.993011:0:201286:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788881124.993012:0:201287:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000819f63c4 00000100:00000001:0.0:1788881124.993013:0:201286:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881124.993014:0:201288:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000010:3.0:1788881124.993016:0:201287:0:(lib-lnet.h:277:lnet_md_free()) slab-freed 'md': 136 at 00000000819f63c4. 00000100:00000001:0.0:1788881124.993017:0:201286:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881124.993017:0:201288:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788881124.993022:0:201288:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000663051e5 x1875776780712064/t0(0) o400->lustre-MDT0001-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881124.993022:0:201287:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:2.0:1788881124.993028:0:201288:0:(client.c:2780:__ptlrpc_free_req()) Process entered 00000100:00000200:3.0:1788881124.993030:0:201287:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881124.993031:0:201288:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788881124.993034:0:201288:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 0000000099f3ac40. 00000100:00000200:3.0:1788881124.993041:0:201287:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:2.0:1788881124.993049:0:201288:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000100:00000001:3.0:1788881124.993051:0:201287:0:(events.c:163:reply_in_callback()) Process leaving 00000020:00000001:2.0:1788881124.993052:0:201288:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1788881124.993055:0:201288:0:(genops.c:1244:class_import_put()) import 00000000b98b8a4e refcount=3 obd=lustre-MDT0001-mdc-ffff8ca787527000 00000100:00000001:3.0:1788881124.993055:0:201287:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881124.993059:0:201287:0:(client.c:1580:after_reply()) Process entered 00000020:00000001:2.0:1788881124.993060:0:201288:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000001:3.0:1788881124.993062:0:201287:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 02000000:00000010:2.0:1788881124.993064:0:201288:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000004e0f6846. 00000100:00000001:3.0:1788881124.993066:0:201287:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 02000000:00000001:2.0:1788881124.993069:0:201288:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:3.0:1788881124.993069:0:201287:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788881124.993073:0:201288:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 02000000:00000001:3.0:1788881124.993074:0:201287:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788881124.993077:0:201288:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000663051e5. 00000100:00001000:3.0:1788881124.993080:0:201287:0:(import.c:2149:obd_at_measure()) add 5 to 00000000468b166e time=42 v=5 (5 5 5 5) 00000100:00000001:2.0:1788881124.993081:0:201288:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788881124.993083:0:201288:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788881124.993086:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00001000:3.0:1788881124.993087:0:201287:0:(import.c:2149:obd_at_measure()) add 1 to 0000000092db0829 time=42 v=5 (1 1 1 1) 00000100:00000001:2.0:1788881124.993090:0:201288:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881124.993091:0:201287:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788881124.993093:0:201288:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881124.993094:0:201287:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788881124.993098:0:201288:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788881124.993100:0:201288:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788881124.993103:0:201287:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788881124.993107:0:201287:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881124.993110:0:201287:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788881124.993112:0:201287:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8ca787527000: skip recheck: last_committed 0 00000100:00000001:3.0:1788881124.993114:0:201287:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788881124.993115:0:201287:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788881124.993119:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881124.993125:0:201287:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788881124.993127:0:201287:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788881124.993128:0:201287:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788881124.993131:0:201287:0:(lustre_net.h:2426:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788881124.993149:0:201287:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@000000002b387587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ea27006f-52a8-4007-b69a-dcac8279445a:201287:1875776780711936:192.168.204.111@tcp:400: 00000100:00000001:3.0:1788881124.993153:0:201287:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881124.993155:0:201287:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788881124.993157:0:201287:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@000000002b387587 x1875776780711936/t0(0) o400->lustre-MDT0000-mdc-ffff8ca787527000@192.168.204.111@tcp:12/10 lens 224/224 e 0 to 0 dl 1788881140 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788881124.993164:0:201287:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788881124.993165:0:201287:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788881124.993167:0:201287:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000b31f2b6a. 02000000:00000001:3.0:1788881124.993170:0:201287:0:(sec.c:1914:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788881124.993172:0:201287:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1788881124.993173:0:201287:0:(genops.c:1244:class_import_put()) import 0000000095473c20 refcount=3 obd=lustre-MDT0000-mdc-ffff8ca787527000 00000020:00000001:3.0:1788881124.993175:0:201287:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1788881124.993176:0:201287:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000f0a0057b. 02000000:00000001:3.0:1788881124.993179:0:201287:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788881124.993180:0:201287:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788881124.993181:0:201287:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000002b387587. 00000100:00000001:3.0:1788881124.993183:0:201287:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788881124.993184:0:201287:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881124.993186:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788881124.993189:0:201287:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788881124.993191:0:201287:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788881124.993192:0:201287:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788881124.993194:0:201287:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881125.054354:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881125.054363:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881125.054376:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881125.054377:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881125.054429:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788881125.054434:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788881125.054437:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881126.078338:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881126.078345:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881126.078350:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881126.078351:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881126.078403:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881126.078406:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881126.078408:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881127.102296:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881127.102303:0:205033:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1788881127.102306:0:205033:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1788881127.102309:0:205033:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881127.102314:0:205033:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1788881127.102316:0:205033:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881127.102318:0:205033:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881127.102321:0:205033:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881127.102325:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881127.102327:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881127.102380:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788881127.102383:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788881127.102385:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881128.126359:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881128.126366:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881128.126374:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881128.126376:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788881128.126425:0:203399:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788881128.126428:0:203399:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788881128.126430:0:203399:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1788881129.150323:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881129.150329:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788881129.150335:0:205033:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1788881129.150336:0:205033:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788881129.150413:0:203398:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788881129.150420:0:203398:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788881129.150423:0:203398:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 118776 lines, 118776 kept, 0 dropped, 0 bad.